Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backend: Migrate PPL Logs queries to the backend #200

Closed
Tracked by #192
idastambuk opened this issue Jun 12, 2023 · 1 comment · Fixed by #259
Closed
Tracked by #192

Backend: Migrate PPL Logs queries to the backend #200

idastambuk opened this issue Jun 12, 2023 · 1 comment · Fixed by #259

Comments

@idastambuk
Copy link
Contributor

idastambuk commented Jun 12, 2023

Prerequisite: #226

PPL Logs are processed into dataFrames on the FE, and as timeSeries on the BE. They should probably be separated out.
ES plugin doesn't support PPL queries, but maybe the Logs PR for Lucene could be something to look at for reference.

Some general guidelines are:

  • Create visualization to compare between frontend and backend flows, preferably in a shared Cloud dashboard e.g. clouddatasources.grafana.net
  • Unit tests in the frontend should be copied or updated in the backend to assert the same behavior and add unit tests when there are none.
    • Check Elasticsearch repo for related tests – helpful when there are no OpenSearch tests in the frontend or backend
  • We should match OpenSearch's backend behavior to the current frontend behavior, but be aware it may not always make sense. Elasticsearch's current behavior is also another resource for how OpenSearch should behave.
    • Let's be pragmatic about this – our foremost goal is the migration of OpenSearch's frontend behavior. We can aim for some Elasticsearch feature parity, but only within reason.
  • Often there is a PR in the Elasticsearch backend data source migration Elasticsearch: Convert the datasource to be a backend datasource grafana#54011 which is 75% of what OpenSearch will need
  • Migrate or implement: i.e. make any changes to code in backend
  • Use the dashboard created above to compare visualization results (data frame comparison)
    • Can also inspect data frame on a more detailed basis

Note: (Similarly to Lucene Logs of the FE) It looks like PPL Logs are not completely correctly processed on the Frontend, as the int field is passed as string. This isn't the case on the BE, as seen from the screenshot.
Screenshot 2023-06-12 at 1 31 26 PM
Screenshot 2023-06-12 at 1 32 18 PM

@sarahzinger sarahzinger moved this from Incoming to Next in AWS Datasources Jun 12, 2023
@sarahzinger sarahzinger moved this from Next to Backlog in AWS Datasources Jun 12, 2023
@sarahzinger sarahzinger moved this from Backlog to Next in AWS Datasources Jul 24, 2023
@sarahzinger
Copy link
Member

maybe we can also fix this one: #212 with this ticket

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

3 participants