Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[system tests] Take advantage of the runtime mappings to validate the runtime fields present in the docs #2208

Open
mrodm opened this issue Nov 6, 2024 · 0 comments
Labels
Team:Ecosystem Label for the Packages Ecosystem team

Comments

@mrodm
Copy link
Contributor

mrodm commented Nov 6, 2024

Relates #1229
Follows #2206

In order to try to be agnostic to the structure of the documents ingested to run the validation in system tests. It would be helpful to run validations comparing the mapping definitions instead.

Use the mappings found in the preview template or in the data stream API to validate the runtime fields present in the hits returned by Elasticsearch as part of the system tests.

# preview mappings API
POST /_index_template/_simulate/<index_template_name>

# Example
POST /_index_template/_simulate/logs-microsoft_dhcp.log

# actual data stream mappings API
GET /<data_stream_test>/_mapping/

Related documentation about runtime fields:
https://www.elastic.co/guide/en/elasticsearch/reference/current/runtime-mapping-fields.html

For now, the errors that could be raised as part of this validation should not make fail the validation process, therefore:

  • Ignore at this step mappings not found in the preview, to be done in the next step.
  • Enable via environment variable this mapping or show just just warning messages.

To be tested:

  • Run these validations in stack 7.x
  • Run these validations in stack 8.x
  • Run these validations in input and integration packages.
  • Run these validations in Stacks with LogsDB enabled (synthetics).
@mrodm mrodm added the Team:Ecosystem Label for the Packages Ecosystem team label Nov 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Ecosystem Label for the Packages Ecosystem team
Projects
None yet
Development

No branches or pull requests

1 participant