Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[system tests] Validate fields are documented with dynamic templates if needed #2207

Open
mrodm opened this issue Nov 6, 2024 · 0 comments
Assignees
Labels
Team:Ecosystem Label for the Packages Ecosystem team

Comments

@mrodm
Copy link
Contributor

mrodm commented Nov 6, 2024

Follows #2206

In order to try to be agnostic to the structure of the documents ingested to run the validation in system tests. It would be helpful to run validations comparing the mapping definitions instead.

For those mappings that can not be validated against the preview mappings in #2206, it is also needed to validate if they match with any of the dynamic templates found in the data stream.

Mappings and dynamic templates can be retrieved from these APIs:

  • Mappings and dynamic templates installed by Fleet before ingesting any doc.
    • These preview mappings can be retrieved using this simulated API:
      POST /_index_template/_simulate/<index_template_name>
      
      # Example
      POST /_index_template/_simulate/logs-microsoft_dhcp.log
      
  • Mappings and dynamic templates that are present after ingesting the docs as part of the system tests.
    • These mappings can be retrieved using this API:
      GET /<data_stream_test>/_mapping/
      

For now, the errors that could be raised as part of this validation should not make fail the validation process, therefore:

  • Ignore at this step mappings not found in the preview, to be done in the next step.
  • Enable via environment variable this mapping or show just just warning messages.

To be tested:

  • Run these validations in stack 7.x
  • Run these validations in stack 8.x
  • Run these validations in input and integration packages.
  • Run these validations in Stacks with LogsDB enabled (synthetics).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Ecosystem Label for the Packages Ecosystem team
Projects
None yet
Development

No branches or pull requests

1 participant