Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enrich otel logs to match beat process logs #7062

Merged
merged 13 commits into from
Mar 11, 2025

Conversation

khushijain21
Copy link
Contributor

@khushijain21 khushijain21 commented Feb 27, 2025

What does this PR do?

This PR adds component.* fields for components run by the OTelManager. For more reasoning check https://github.com/elastic/ingest-dev/issues/5139

Checklist

  • I have read and understood the pull request guidelines of this project.
  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in ./changelog/fragments using the changelog tool
  • I have added an integration test or an E2E test

How to test this PR locally

Logs from normal mode

{"log.level":"info","@timestamp":"2025-02-24T10:59:43.243Z","message":"X","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"filestream-monitoring","type":"filestream"},"log":{"source":"filestream-monitoring"},"log.logger":"centralmgmt.V2-manager","log.origin":{"file.line":505,"file.name":"management/managerV2.go","function":"github.com/elastic/beats/v7/x-pack/libbeat/management.(*BeatV2Manager).unitListen"},"service.name":"filebeat","ecs.version":"1.6.0","ecs.version":"1.6.0"}

Logs from components run by Otel Manager

{"log.level":"error","@timestamp":"2025-02-27T17:24:15.152+0530","log.logger":"modules","message"X","log":{"source":"elastic-agent"},"kind":"receiver","name":"filebeatreceiver/_agent-component/filestream-default","data_type":"logs","log":{"source":"filestream-default"},"component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"filestream-default"},"service.name":"Khushis-MacBook-Pro.local","ecs.version":"1.6.0","ecs.version":"1.6.0"}

Some Differences:

  • We have two log.source fields in the latter. It is not misleading, but two fields with same key can be confusing to users.

Related issues

Copy link
Contributor

mergify bot commented Feb 27, 2025

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b withfields upstream/withfields
git merge upstream/main
git push upstream withfields

Copy link
Contributor

mergify bot commented Feb 27, 2025

This pull request does not have a backport label. Could you fix it @khushijain21? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-./d./d is the label that automatically backports to the 8./d branch. /d is the digit
  • backport-active-all is the label that automatically backports to all active branches.
  • backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
  • backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

@khushijain21 khushijain21 changed the title Enrich otel logs to match beat process logs in hybrid mode [DO NOT MERGE] Enrich otel logs to match beat process logs in hybrid mode Feb 27, 2025
@khushijain21 khushijain21 changed the title [DO NOT MERGE] Enrich otel logs to match beat process logs in hybrid mode Enrich otel logs to match beat process logs Mar 7, 2025
@khushijain21 khushijain21 added backport-8.x Automated backport to the 8.x branch with mergify backport-9.0 Automated backport to the 9.0 branch skip-changelog labels Mar 7, 2025
@pierrehilbert pierrehilbert added the Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team label Mar 7, 2025
@elasticmachine
Copy link
Contributor

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

swiatekm
swiatekm previously approved these changes Mar 7, 2025
Copy link
Contributor

@swiatekm swiatekm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, had two minor questions.

pkoutsovasilis
pkoutsovasilis previously approved these changes Mar 7, 2025
@khushijain21 khushijain21 dismissed stale reviews from pkoutsovasilis and swiatekm via 2092dbb March 10, 2025 11:09
@khushijain21 khushijain21 requested a review from swiatekm March 10, 2025 14:48
Copy link
Contributor

mergify bot commented Mar 10, 2025

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b withfields upstream/withfields
git merge upstream/main
git push upstream withfields

@elasticmachine
Copy link
Contributor

💛 Build succeeded, but was flaky

Failed CI Steps

History

cc @khushijain21

@khushijain21 khushijain21 removed the backport-9.0 Automated backport to the 9.0 branch label Mar 11, 2025
@khushijain21 khushijain21 merged commit ca05561 into elastic:main Mar 11, 2025
13 checks passed
mergify bot pushed a commit that referenced this pull request Mar 11, 2025
* Add  fields to otel logs

(cherry picked from commit ca05561)

# Conflicts:
#	go.mod
#	go.sum
khushijain21 added a commit that referenced this pull request Mar 11, 2025
khushijain21 pushed a commit that referenced this pull request Mar 11, 2025
)

* Enrich otel logs to match beat process logs  (#7062)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-8.x Automated backport to the 8.x branch with mergify skip-changelog Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants