Skip to content

[aws] Update Grok pattern to support new HTTP ELB log format #13944

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
devamanv wants to merge 3 commits into from
Closed

[aws] Update Grok pattern to support new HTTP ELB log format #13944

devamanv wants to merge 3 commits into from
+62 −4

Conversation

@devamanv
Copy link
Contributor

@devamanv devamanv commented May 20, 2025 •
edited
Loading

Proposed commit message

The PR contains changes to modify the ELB HTTP Grok pattern to support the new log format, which is as follows:

http 2025-05-01T11:24:32.748149Z app/internal-service-alb/abcd1234efgh5678 127.0.0.1:57273 - -1 -1 -1 200 - 0 272 "- http://internal-service-alb.example.com:80-/ " "-" - - - "-" "-" "-" - 2025-05-01T11:24:32.720000Z "-" "-" "-" "-" "-" "-" "-" TID_00000000000000000000000000000000

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

  • All pipeline tests should still pass

Related issues

@devamanv devamanv requested review from a team as code owners May 20, 2025 07:53
@devamanv devamanv added enhancement New feature or request Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] labels May 20, 2025
@elasticmachine
Copy link

elasticmachine commented May 20, 2025 •
edited
Loading

💔 Build Failed

Failed CI Steps

History

ritalwar
ritalwar reviewed May 20, 2025
View reviewed changes
packages/aws/changelog.yml
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "3.3.2"
Copy link
Contributor

@ritalwar ritalwar May 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- version: "3.3.2"
- version: "3.4.0"

Since this is an enhancement, should we consider increasing the minor version instead of the patch version, which is typically for bug fixes?

efd6
efd6 reviewed May 21, 2025
View reviewed changes
Copy link
Contributor

@efd6 efd6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is probably worth adding a note to the commit message indicating the origin of the test sample.

packages/aws/data_stream/elb_logs/_dev/test/pipeline/test-alb.log
2024-10-04T17:05:15.514108Z 192.168.131.39 36280 443 TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 - "CN=amazondomains.com,O=endEntity,L=Seattle,ST=Washington,C=US" NotBefore=2023-09-21T22:43:21Z;NotAfter=2026-06-17T22:43:21Z FEF257372D5C14D4 Failed:ClientCertUntrusted
2024-11-29T13:45:24.599544Z 172.31.43.26 58206 80 - - - "-" - - - TID_16132ed0b4112148
2024-11-29T13:45:24.599544Z 172.31.43.26 58206 80 - - - "-" - - - TID_16132ed0b4112148
http 2025-05-01T11:24:32.748149Z app/internal-service-alb/abcd1234efgh5678 127.0.0.1:57273 - -1 -1 -1 200 - 0 272 "- http://internal-service-alb.example.com:80-/ " "-" - - - "-" "-" "-" - 2025-05-01T11:24:32.720000Z "-" "-" "-" "-" "-" "-" "-" TID_00000000000000000000000000000000
Copy link
Contributor

@efd6 efd6 May 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a final new line.

packages/aws/changelog.yml
# newer versions go on top
- version: "3.3.2"
changes:
- description: Update Grok pattern to support new ELB HTTP log format.
Copy link
Contributor

@efd6 efd6 May 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"new" is a term that goes stale. Is there a version number or documentation that details this?

@botelastic
Copy link

botelastic bot commented Jun 20, 2025

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

@botelastic botelastic bot added the Stalled label Jun 20, 2025
@botelastic
Copy link

botelastic bot commented Jul 20, 2025

Hi! This PR has been stale for a while and we're going to close it as part of our cleanup procedure. We appreciate your contribution and would like to apologize if we have not been able to review it, due to the current heavy load of the team. Feel free to re-open this PR if you think it should stay open and is worth rebasing. Thank you for your contribution!

@botelastic botelastic bot closed this Jul 20, 2025
@agithomas agithomas reopened this Jul 21, 2025
@botelastic botelastic bot removed the Stalled label Jul 21, 2025
@agithomas
Copy link
Contributor

agithomas commented Jul 21, 2025

@devamanv , Could you check the open comments and address them. It will be good to have the support for the missing log format.

@botelastic
Copy link

botelastic bot commented Aug 20, 2025

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

@botelastic botelastic bot added the Stalled label Aug 20, 2025
@botelastic
Copy link

botelastic bot commented Sep 19, 2025

Hi! This PR has been stale for a while and we're going to close it as part of our cleanup procedure. We appreciate your contribution and would like to apologize if we have not been able to review it, due to the current heavy load of the team. Feel free to re-open this PR if you think it should stay open and is worth rebasing. Thank you for your contribution!

@botelastic botelastic bot closed this Sep 19, 2025
@andy-werderman
Copy link

andy-werderman commented Sep 24, 2025

Hello, can this be re-opened?

@efd6
Copy link
Contributor

efd6 commented Sep 24, 2025

@andy-werderman I would recommend just opening a new PR with the change.

@devamanv
Copy link
Contributor Author

devamanv commented Jan 4, 2026 •
edited
Loading

I have opened a new PR, and will try to get it merged ASAP.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@efd6 efd6 efd6 left review comments

@ritalwar ritalwar ritalwar left review comments

Assignees

No one assigned

Labels

enhancement New feature or request Integration:aws AWS Stalled Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations]

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@devamanv @elasticmachine @agithomas @andy-werderman @efd6 @ritalwar @andrewkroh