fix(metrics): record streamed calls by their final outcome

[nachiketb-nvidia]

What

Defer routed-call metrics until a streaming answer finishes or is dropped. Stream and decode errors count as failures, and an abandoned open stream cannot count as a success. Record each outcome once and remove the server's duplicate Prometheus error increment.

Why

Fixes SWITCH-1441. An HTTP 200 stream was counted as successful before a later in-band error arrived, contradicting /v1/stats.

Notes for reviewers

85 added production lines. Extends one existing HTTP regression with three assertions; no new test functions. Stream events and buffered responses remain unchanged. Metrics include the time spent consuming the stream.

The local HTTP regression failed on unchanged main because a failed stream incremented switchyard_requests_total. It now verifies no success or successful-call latency is recorded, total errors increase, and the per-model error is counted exactly once. The reported bug no longer reproduces through the server.

Validation: targeted server stream tests (14 passed), buffered stats control (1 passed), client observability stream tests (3 passed, including cancellation), and classifier accounting control (1 passed). Formatting, workspace clippy, and diff checks passed. Only local mock providers; no full suite or live provider calls.

Landing note: this independent main-based PR overlaps #773 in run.rs. The combined resolution calls the stream-outcome observer at each completion candidate and counts algorithm-supplied answers once. That resolution passed both regressions plus classifier-accounting and streaming-success controls in a temporary worktree.

Summary by CodeRabbit

• Bug Fixes
  • Improved usage and performance metrics for streamed LLM responses.
  • Streaming requests are now recorded as successful only after normal completion.
  • Stream interruptions, decoding errors, and abandoned streams are correctly counted as failures.
  • Prevented failed streaming requests from incorrectly increasing request and latency metrics.
  • Corrected error reporting to avoid duplicate error counts.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: de54ab91-b2c7-4966-854e-2692bf4fefb4

📥 Commits

Reviewing files that changed from the base of the PR and between 0a32f56 and f64fd31.

📒 Files selected for processing (4)

• crates/libsy-llm-client/src/metrics.rs
• crates/libsy-llm-client/src/run.rs
• crates/switchyard-server/src/usage_metrics.rs
• crates/switchyard-server/tests/server.rs

💤 Files with no reviewable changes (1)

• crates/switchyard-server/src/usage_metrics.rs

Included review availability: Your plan provides up to 12 included reviews per hour; 8 remain after this review.

---

Walkthrough

Changes

The client now tracks routed request metrics through streamed response completion or drop. It records stream failures explicitly, updates terminal metric APIs, removes duplicate server error counting, and extends failed-stream metric tests.

Stream-aware metrics

Layer / File(s)	Summary
Stream outcome observation crates/libsy-llm-client/src/metrics.rs	The client wraps streamed responses and records success on completion, failure on stream or decode errors, and failure when an open stream is dropped.
Metric API integration crates/libsy-llm-client/src/metrics.rs, crates/libsy-llm-client/src/run.rs	Terminal metric functions accept explicit success values. run uses observe_routed_request for routed request metrics.
Server metric validation crates/switchyard-server/src/usage_metrics.rs, crates/switchyard-server/tests/server.rs	Stream errors no longer increment the OpenTelemetry error counter directly. Tests verify unchanged request and latency metrics and an increased total-error metric.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Merge Risk: ⚪ Minimal · up to f64fd

Streaming request metrics retain successful completion handling and record recognized stream failures without duplicate server error increments. No actionable merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 71.43% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 3 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: recording streamed call metrics according to their final outcome.

• Fix all pre-merge checks with AI

---

_{A rabbit watched the stream run bright
Metrics waited through the night
Drops now mark a failed trail
Success arrives when chunks prevail
Errors count without a veil}

_{Comment @coderabbitai help to get the list of available commands.}

[ayushag-nv]

looks solid.