ad hoc filters not applied to raw_data queries

[gulecroc]

What happened:

Use a table with raw data.

When adding a filter from ad hoc filter, the table must filter lines.

Or when adding a filter from the table (clic on button "+" to add the key/value as an ad hoc filter).

I have 2 problems :

• when adding filter from the ad hoc filter on the top, there is no value Edit: this has been separated to another issue now
• when adding filter from the table value (clic on button "+" to add the key/value filter), the table is not filtered

What you expected to happen:

The table content is filtered with ad hoc filter.

How to reproduce it (as minimally and precisely as possible):

Create opensearch data :

DELETE /grafana_table_raw_data_ad_hoc_filters

PUT grafana_table_raw_data_ad_hoc_filters

POST /grafana_table_raw_data_ad_hoc_filters/_bulk
{"index":{}}
{"name": "test0", "filter1": "value1", "@timestamp" : "2023-07-26"}
{"index":{}}
{"name": "test1", "filter1": "value1", "@timestamp" : "2023-07-26"}
{"index":{}}
{"name": "test2", "filter1": "value1", "@timestamp" : "2023-07-26"}
{"index":{}}
{"name": "test3", "filter1": "value2", "@timestamp" : "2023-07-26"}
{"index":{}}
{"name": "test4", "filter1": "value2", "@timestamp" : "2023-07-26"}
{"index":{}}
{"name": "test5", "filter1": "value2", "@timestamp" : "2023-07-26"}
{"index":{}}
{"name": "test6", "filter1": "value3", "@timestamp" : "2023-07-26"}
{"index":{}}
{"name": "test7", "filter1": "value3", "@timestamp" : "2023-07-26"}
{"index":{}}
{"name": "test8", "filter1": "value3", "@timestamp" : "2023-07-26"}
{"index":{}}
{"name": "test9", "filter1": "value4", "@timestamp" : "2023-07-26"}

Add grafana opensearch datasource from this index.

Create a simple dashboard with table and raw data :

Select on the button "+" near a table line value to create the ad hoc filter.

Anything else we need to know?:
This bug was introduced in v2.7.0

Environment:

• Grafana version: v9.5.6 (859a2654d3)
• Plugin version: 2.8.1

[fridgepoet]

Hi @gulecroc, thanks for reporting this issue. I can reproduce the issues.
Adding some information this part of the issue:

when adding filter from the ad hoc filter on the top, there is no value

Clicking on the value should show a list of populated possible values. It shows "No Options found" (Seems to be current version and also v2.6.2)

In the meantime you can roll back to v2.6.2 of the plugin and it should work again.

[gulecroc]

Hi @fridgepoet , thank you for your reply.

I create an elasticsearch datasource with opensearch URL and it works.
If I create the filter from the top, I have option values only from "key.keyword", not from "key".
If I create the filter from selecting value in the table, the filter key name is "key" :

[gulecroc]

I rollback the plugin to v2.6.2.

The filter from selecting value in the table works well.
But adding filter at the top does not show values, using "key" or "key.keyword" :

[fridgepoet]

Thanks @gulecroc, on any previous version of the opensearch plugin, were you ever able to show the values from adding the filters?

[gulecroc]

We are migrating from opendistro (we used elasticsearch datasource) to opensearch (with opensearch datasource), so I don't kown if it works previously on opensearch datasource plugin

[fridgepoet]

Thanks again @gulecroc. From my investigation, I think the values were not showing in any previous plugin version either.

I propose that this issue deal with "when adding filter from the table value (clic on button "+" to add the key/value filter), the table is not filtered."
This is something we can solve with #225 . It will revert the behavior to the same as what you're seeing in v2.6.2. (Please note ad hoc filtering will not return data for nested documents as this is not supported in the Lucene query language.)

Then I think we can open another issue to deal with "when adding filter from the ad hoc filter on the top, there is no value." The workaround for that is to simply write in the value you want. We will work on populating the list with the possible values in another PR.

[gulecroc]

I @fridgepoet , sorry for the delay I was on vacation.
No problem to create multiples issues.
Thank you for your work !

[fridgepoet]

no worries @gulecroc, hope you had a nice vacation! thanks so much for reporting issues!