Skip to content

test: add independent log parser fixtures - #76

Open
GioLealhmx wants to merge 4 commits into
weilixiong:mainfrom
GioLealhmx:agent/log-parser-independent-fixtures
Open

test: add independent log parser fixtures#76
GioLealhmx wants to merge 4 commits into
weilixiong:mainfrom
GioLealhmx:agent/log-parser-independent-fixtures

Conversation

@GioLealhmx

Copy link
Copy Markdown

/claim #5

Summary

Fixes #5 by adding independent, fixture-backed coverage for the legacy log aggregator parsers. The new fixtures are handwritten production-like samples for JSON, plain text, nginx access logs, and malformed/unsupported input, so the tests no longer depend on data generated by the parser itself.

The independent nginx fixture exposed two small parser bugs, which this PR fixes: nginx lines now reach the nginx parser before the generic text fallback, and remote_user now maps to the nginx remote user field instead of the ident field.

Changes

  • Added independent log fixtures under tools/tests/fixtures/log_aggregator/ for JSON, text, nginx, and malformed lines.
  • Added tools/tests/test_log_aggregator_fixtures.py using only the Python standard library/unittest.
  • Validated parser outputs for timestamp, level, service, message, nginx status, remote address, remote user, and CLI JSON report shape.
  • Reordered parser fallback so NginxLogParser runs before TextLogParser.
  • Fixed nginx remote_user extraction to use the correct regex group.
  • Hardened build.py so missing optional toolchains are reported as module failures instead of aborting before diagnostics are generated.

Testing

  • python -m py_compile build.py tools/log_aggregator.py — PASS
  • python -m unittest discover -s tools/tests -p "test_*.py" -v — PASS, 5 tests
  • python build.py — ran and generated required diagnostics: diagnostic/build-a17cdea5.logd and diagnostic/build-a17cdea5-metadata.json. Exit code was 1 because this Windows environment lacks several unrelated toolchains; the diagnostic metadata records 2 passed modules (market, compliance) and 8 missing-toolchain failures.

Checklist

  • Relevant modules affected by these changes build locally
  • Tests pass locally
  • Diagnostic build log is committed in this PR
  • Documentation has been updated, if applicable
  • Configuration or schema changes are documented, if applicable
  • No generated build artifacts are committed, except the required diagnostic build log
  • Changes are scoped to the PR purpose and avoid unrelated cleanup
  • Security, privacy, and error-handling implications have been considered

  • I would like to request that my diagnostic build log is removed before merging

@GioLealhmx

Copy link
Copy Markdown
Author

Hi maintainers — the required GitHub Actions workflow for this fork PR is currently showing action_required, so it looks like it needs maintainer approval before CI can run.

Local verification is documented in the PR body:

  • python -m py_compile build.py tools/log_aggregator.py — PASS
  • python -m unittest discover -s tools/tests -p "test_*.py" -v — PASS
  • python build.py — generated the required diagnostic artifacts; unrelated modules that need unavailable local toolchains are recorded in the diagnostic metadata

Could you please approve the workflow run when convenient? I’ll follow up if CI reports anything actionable.

1 similar comment
@GioLealhmx

Copy link
Copy Markdown
Author

Hi maintainers — the required GitHub Actions workflow for this fork PR is currently showing action_required, so it looks like it needs maintainer approval before CI can run.

Local verification is documented in the PR body:

  • python -m py_compile build.py tools/log_aggregator.py — PASS
  • python -m unittest discover -s tools/tests -p "test_*.py" -v — PASS
  • python build.py — generated the required diagnostic artifacts; unrelated modules that need unavailable local toolchains are recorded in the diagnostic metadata

Could you please approve the workflow run when convenient? I’ll follow up if CI reports anything actionable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[$25 BOUNTY] [Python] Add independent log parser fixtures

1 participant