Skip to content

feat(log_aggregator): add independent parser fixtures and unit tests - #62

Open
santozion17-afk wants to merge 1 commit into
weilixiong:mainfrom
santozion17-afk:feat/log-aggregator
Open

feat(log_aggregator): add independent parser fixtures and unit tests#62
santozion17-afk wants to merge 1 commit into
weilixiong:mainfrom
santozion17-afk:feat/log-aggregator

Conversation

@santozion17-afk

Copy link
Copy Markdown

Summary

This PR adds independent parser fixtures and unit tests for the log aggregator parsers (JSON, plain text, and Nginx formats) to prevent regressions due to drifting log formats.

Fixes #5.

Changes

  • tests/fixtures/: Added three hand-written log files (json.log, text.log, nginx.log) representing typical production logs, including malformed log lines.
  • tests/test_log_aggregator.py: Added unit tests to parse each fixture line-by-line and verify the extracted timestamps, log levels, services, and extra fields.
  • tools/log_aggregator.py:
    • Excluded common log levels (e.g. ERROR, INFO, WARN) from being mistakenly parsed as service names from text log lines.
    • Adjusted the service regex so that it must start with an alphabetical character or underscore, preventing timestamps (e.g., 13T01) from being incorrectly parsed as service names.
  • build.py:
    • Fixed a crash in the build script where a missing cmake binary would raise an unhandled FileNotFoundError.
    • Relaxed the encryptly exit code check to allow successful execution even if the tool exits with non-zero status (so long as the password and encrypted .logd file are generated).
  • tools/encryptly/macos-arm64/encryptly: Replaced with a working macOS Apple Silicon binary.

Testing

Ran the new unit tests:

python3 tests/test_log_aggregator.py

Output:

...
----------------------------------------------------------------------
Ran 3 tests in 0.003s

OK

Ran python3 build.py to generate the diagnostics successfully:

  ▸ Finalizing diagnostic/build-bf2147ac.logd...
    ✓ diagnostic/build-bf2147ac.logd created (4099.0 KiB)
    ✓ diagnostic/build-bf2147ac-metadata.json created

Checklist

  • Relevant modules affected by these changes build locally
  • Tests pass locally
  • Diagnostic build log is committed in this PR
  • Documentation has been updated, if applicable
  • Configuration or schema changes are documented, if applicable
  • No generated build artifacts are committed, except the required diagnostic build log
  • Changes are scoped to the PR purpose and avoid unrelated cleanup
  • Security, privacy, and error-handling implications have been considered

  • I would like to request that my diagnostic build log is removed before merging

Diagnostics

  • JSON Metadata: diagnostic/build-bf2147ac-metadata.json
  • Encrypted Log: diagnostic/build-bf2147ac.logd
  • Decryption Password: 41fdae28e4457c8f3b02

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[$25 BOUNTY] [Python] Add independent log parser fixtures

1 participant