Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEATURE] Support searches across raw data indices and rollup indices together #1213

Open
sarthakaggarwal97 opened this issue Jul 26, 2024 · 3 comments · May be fixed by #1268
Open

[FEATURE] Support searches across raw data indices and rollup indices together #1213

sarthakaggarwal97 opened this issue Jul 26, 2024 · 3 comments · May be fixed by #1268
Labels
enhancement New request

Comments

@sarthakaggarwal97
Copy link
Contributor

Is your feature request related to a problem?
Currently, we do not support searches if the same request has both raw indices and the indices with associated rollup job. We return the exception upon validation if both types of indices are present in the request.

What solution would you like?
Support searches across both all indices in order to query / aggregate results irrespective of whether the rollup job is assigned or not to a particular index.
@sarthakaggarwal97 sarthakaggarwal97 changed the title [FEATURE] Support searches across raw data indices and rollup indices together. [FEATURE] Support searches across raw data indices and rollup indices together Jul 26, 2024
@mgodwan
Copy link
Member

mgodwan commented Jul 31, 2024

Is there any historical reason around why we are not supporting this? Is this just because of query rewrite we trigger via rollup interceptor?

Also, we would need to test this flow and verify that the aggregations across indices (raw, rolled up) yield correct results.

@dblock dblock removed the untriaged label Aug 19, 2024
@dblock
Copy link
Member

dblock commented Aug 19, 2024

Catch All Triage - 1, 2, 3

@manojfaria
Copy link

manojfaria commented Sep 18, 2024 •
edited
Loading

In the absence of this feature, users of opensearch have to evaluate the option of issuing two separate queries i.e. one query to the raw index, and one query to the rollup-indices and then merge the two results in their application code.

Some customers/users of OpenSearch are reporting this double querying (once to raw, and additional query to rollup indices) as a major hurdle, as this workaround of separately querying raw and rollup indices requires significant code change to their product/app.

@tandonks tandonks linked a pull request Sep 27, 2024 that will close this issue
Open
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Allowing non-rollup and rollup indices to be searched together tandonks/index-management
4 participants
@dblock @manojfaria @sarthakaggarwal97 @mgodwan