Skip to content

[ BOUNTY] Fix HighMemoryUsage self-dividing alert expression - #6

Open
Xiaofei-Z wants to merge 2 commits into
weilixiong:mainfrom
Xiaofei-Z:fix-memory-alert-ratio
Open

[ BOUNTY] Fix HighMemoryUsage self-dividing alert expression#6
Xiaofei-Z wants to merge 2 commits into
weilixiong:mainfrom
Xiaofei-Z:fix-memory-alert-ratio

Conversation

@Xiaofei-Z

Copy link
Copy Markdown

Summary

Closes #1. Fixes the HighMemoryUsage Prometheus alert that divided process_resident_memory_bytes by itself (always evaluating to 1), making the alert unreliable.

Changes

  • tools/monitoring_setup.py: replaced process_resident_memory_bytes / process_resident_memory_bytes > 0.9 with process_resident_memory_bytes / machine_memory_bytes > 0.9 — matches the existing instance:memory_usage:ratio recording rule already in the file
  • Added validate_alert_expressions() regression guard that detects any self-dividing metric expression (metric / metric), runnable via the module's validation path

Testing

  • python3 tools/monitoring_setup.py --validateOK: no self-dividing alert expressions
  • Reverse test: temporarily restoring the self-dividing expr → validator correctly flags it
  • Confirmed machine_memory_bytes is valid PromQL and consistent with the recording rule at line 149
  • python3 build.py run; diagnostic JSON attached (build-2b54872c.json)
  • Existing dry-run output remains compatible

Checklist

  • HighMemoryUsage no longer self-divides
  • Replacement uses a meaningful denominator, valid PromQL
  • Validation path catches self-dividing expressions
  • Existing monitoring setup behavior preserved
  • Diagnostic build log committed
  • Changes scoped to monitoring setup path

@Xiaofei-Z

Copy link
Copy Markdown
Author

Hi @weilixiong, submitted this for the bounty. Fixed the self-dividing expression (now uses machine_memory_bytes, consistent with the existing recording rule) and added a validator to catch this regression. Happy to address any feedback. Thanks!

Feige added 2 commits June 19, 2026 19:06
- Replace process_resident_memory_bytes / process_resident_memory_bytes
  with process_resident_memory_bytes / machine_memory_bytes
- Add validate_alert_expressions() regression guard

Closes weilixiong#1
@Xiaofei-Z
Xiaofei-Z force-pushed the fix-memory-alert-ratio branch from 5d2a498 to c40b2b3 Compare June 19, 2026 11:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[$30 BOUNTY] [Python] Fix HighMemoryUsage alert ratio expression

1 participant