Backend migration: When adding multiple groupBy terms, only the last group works

[idastambuk]

We had a bug report in one of the comments to the migration PR: #375 (comment) related to groupBy clause:

Hi, I think updating the plugin to v2.15.0 also broke some queries for me. All queries where I used 'group by' on multiple terms, only the last group by would work. Reverting to v2.14.7 fixed the issue. Not sure if it related to this change or not.
Grafana v10.4.2
Opensearch v2.13.0
Plugin v2.15.0

[difi80211g]

I was having this issue too. I just updated to 2.15.1 and now it i set a metric of count and group by a term it will not show me anything.

[idastambuk]

Hi @LoLZeS666 and @difi80211g from initial debugging, I managed to reproduce the error. It looks like it's caused by the number of buckets created being higher than maximum for Open Search. If you set the min_interval in the Query options for the panel to be higher (1h for example - @difi80211g I see in your panel it's 1m), does this return data?

[difi80211g]

I changed it to 1h as you suggested but still do not see anything.

[LoLZeS666]

I was grouping by a term called name, which gives a result of around 150 rows. My opensearch cluster has buckets set at 100000. So I don't think this is related to the bucket limit.
Reverting the plugin fixed the issue.

[idastambuk]

Hi again, @difi80211g, I have a few questions on your bug:

1. Can you copy the query from the QueryInspector? I tried to reproduce the bug with one terms grouping but the query works fine for me.
2. Did the same query work before 12.15.0?
3. When you upgraded to 12.15.0, did the query error or did it show no data?
4. Just to double check, it shows No Data in 12.15.1?
5. If you select Raw Data from the Metric dropdown, do you get back any results?

Thanks for helping us debug this!

[idastambuk]

@LoLZeS666 I managed to reproduce your problem, and I indeed see only the latest term aggregation. Adding this to our pipeline.

For whoever picks this up: We're building the query correctly and getting all the aggregations from OpenSearch, but not processing the response correctly.
Example query to reproduce in local with test data: