feat(metrics): preserve canonical engine load sections

[lucifer1004]

Motivation

Engine GetLoads responses carry canonical load sections (memory, queues, aggregate) that were dropped at the gRPC boundary, and engine load gauges were only polled when a load-aware routing policy was active — coupling engine observability to routing configuration.

What this changes

• SchedulerLoadSnapshot gains memory/queues snapshots and WorkerLoadResponse gains version + an optional aggregate section, with mappings for SGLang, vLLM (aggregate None), and TokenSpeed engines.
• Every successful routing-owned load poll is always re-exported as smg_engine_* Prometheus gauges; --engine-metrics is narrowed to only force polling when no load-aware policy is active. No additional engine RPC is introduced.
• Worker eviction now sentinels smg_engine_* series whenever a load was ever published, not only under the flag.
• Usage::with_cached_tokens(0) records an explicit zero (cold miss) instead of omitting cache accounting.

Tests

New conversion test (conversion_preserves_version_sections_and_aggregate), explicit-zero builder test, and the metrics-endpoint integration test; openai-protocol 93/0, smg metrics/policies/monitor lib suites 285/0, engine_metrics_test 1/1.

[coderabbitai]

Warning

Review limit reached

@lucifer1004, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 20 minutes

Limit details: You’ve used all 2 included reviews currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

Wait for the limit to reset, then comment @coderabbitai review or push new commits to the PR.

An organization admin can change what happens after included review limits in Billing.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: b28af46e-085a-4870-a5f7-0a65fecce56c

📥 Commits

Reviewing files that changed from the base of the PR and between c1f671e and aca2c31.

📒 Files selected for processing (13)

• crates/grpc_client/src/sglang_scheduler.rs
• crates/grpc_client/src/tokenspeed_scheduler.rs
• crates/grpc_client/src/vllm_engine.rs
• crates/protocols/src/common.rs
• crates/protocols/src/worker.rs
• model_gateway/src/config/types.rs
• model_gateway/src/main.rs
• model_gateway/src/observability/metrics.rs
• model_gateway/src/policies/least_load.rs
• model_gateway/src/policies/power_of_two.rs
• model_gateway/src/routers/grpc/zmq_client.rs
• model_gateway/src/worker/monitor.rs
• model_gateway/tests/engine_metrics_test.rs

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[lucifer1004]

Note for reviewers: this PR and #2227 (TokenSpeed sampling seeds) both touch crates/grpc_client/src/tokenspeed_scheduler.rs (and one line of zmq_client.rs). Disjoint hunks; the second to land may need a trivial rebase.