fix: correct HighMemoryUsage alert self-dividing expression

[FUstomato]

Summary

Fix the HighMemoryUsage alert rule expression in tools/monitoring_setup.py which contained a self-dividing PromQL expression (process_resident_memory_bytes / process_resident_memory_bytes) that always evaluates to 1, making the alert completely ineffective. Also add a new validate_self_dividing_expressions() function with --validate CLI integration to prevent similar issues in the future.

Changes

1. Fix HighMemoryUsage alert expression (tools/monitoring_setup.py)

Before (buggy):

"expr": "process_resident_memory_bytes / process_resident_memory_bytes > 0.9",

After (fixed):

"expr": "process_resident_memory_bytes / node_memory_MemTotal_bytes > 0.9",

The original expression divides process_resident_memory_bytes by itself, always yielding ~1.0, which means the alert would never fire regardless of actual memory usage. The fix uses node_memory_MemTotal_bytes (standard node_exporter metric) as the denominator to compute the actual ratio of process memory to total system memory.

2. Add validate_self_dividing_expressions() function

New validation function that detects PromQL self-dividing patterns (e.g., X / X) in both alert rules and recording rules. It correctly:

• Handles nested PromQL functions (rate, sum, histogram_quantile, etc.)
• Strips label matchers {...} and time range selectors [...]
• Ignores by (...)/without (...) grouping clauses (label names are not metrics)
• Splits only on top-level / operators (respects parentheses and braces)

3. Enhanced --validate CLI flag

The existing --validate flag now runs the self-dividing expression check before connectivity checks. If issues are found, exits with code 1 to integrate with CI/CD pipelines.

Testing

• Verified the fix: python3 tools/monitoring_setup.py --validate reports "PASS: No self-dividing expressions found."
• Verified detection: the validator correctly flags the original buggy expression as an issue
• Verified no false positives: the validator correctly passes all other 12 alert rules and 6 recording rules

Checklist

• Relevant modules affected by these changes build locally
• Tests pass locally (validation script exits 0 with fixed code)
• Diagnostic build log is committed in this PR
• Documentation has been updated, if applicable (inline comments explain the fix)
• Configuration or schema changes are documented, if applicable
• No generated build artifacts are committed, except the required diagnostic build log
• Changes are scoped to the PR purpose and avoid unrelated cleanup
• Security, privacy, and error-handling implications have been considered (N/A - monitoring config change only)

---

• I would like to request that my diagnostic build log is removed before merging