Skip to content

Add support for new /v2/query endoint#1897

Draft
Viicos wants to merge 2 commits intomainfrom
vp/new-query-endpoint
Draft

Add support for new /v2/query endoint#1897
Viicos wants to merge 2 commits intomainfrom
vp/new-query-endpoint

Conversation

@Viicos
Copy link
Copy Markdown
Member

@Viicos Viicos commented May 4, 2026

The first commit switches the existing client to use the /v2/query endpoint. The second commit adds support for NDJSON streaming.

I went over a couple solutions to make the switch to the new endpoint. Here are some requirements to keep in mind:

  • Behavior should not silently change. For instance, we can't silently have a min_timestamp set when existing usage did not set any.
  • Having existing usage of the client deprecated is fine, but disruption should ideally happen only once.

The first solution would be to introduce new methods, leaving the existing ones unchanged (still targeting /v1/query):

  • The existing methods would still have to be marked as deprecated, and it is also unclear how to name the new ones (query_json_v2() isn't ideal, because if/when we remove the old ones, we end up with this v2 naming in the method).
  • The benefit is we can freely change the data structure to directly match the /v2/query result structure (e.g. no _transform_fields_for_backwards_compatibility(), etc).

The second solution is to adapt the existing methods, while still preserving backwards compatibility:

  • Omitting min_timestamp is deprecated, and will be required in the future (internally, we use 2020-01-01 as a min_timestamp).
  • The data structure is mapped to preserve backwards compatibility (_map_v2_result()). This remaps the columns field, and only renames data -> rows (so no concerns of memory/CPU usage client side).
  • Only concern: /v2/query's default limit when from 100 to 1000, and we rely on the remote default here. I feel it is ok to potentially return more results; could this break existing usage? Anyone could be relying on the limit of 100 in some way?

Imo 2 is the best option, so this is the one I implemented.


For the query stream method, I'm wondering if a utility to filter (in a type safe way?) message types would be useful. Maybe having users filtering with if message['type'] = ... is enough.

@adriangb
Copy link
Copy Markdown
Member

adriangb commented May 4, 2026

Why not both?

  1. Adapt the current methods. We can leave them around longer that way.
  2. We add new methods / new clients that don't do (somewhat expensive) transformations.

@Viicos
Copy link
Copy Markdown
Member Author

Viicos commented May 4, 2026

Adapt the current methods. We can leave them around longer that way.

Apart from query_json() (which we can also not deprecate and keep),  the existing ones I modified aren't meant to go away.

We add new methods / new clients that don't do (somewhat expensive) transformations.

How would the new ones be named? As per the PR body, I'm not a fan of v2 being in method names. Also the data transformation added in this PR will not be expensive, mostly because the data isn't deeply modified, just the top level key name is changed. In a major version, we could introduce a breaking change that renames the key names to match the /v2/query endpoint verbatim. It will be a one off migration for users, and I don't think it will cause any issues as the SDK is most likely not being really used by other libraries which would need to maintain compatibility with both.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants