Skip to content

feat(metrics): surface turn-level spend on turn_metrics and /v1/admin/metrics - #645

Open
yzxcj797 wants to merge 1 commit into
yc-software:mainfrom
yzxcj797:feat/turn-metrics-spend-586c
Open

feat(metrics): surface turn-level spend on turn_metrics and /v1/admin/metrics#645
yzxcj797 wants to merge 1 commit into
yc-software:mainfrom
yzxcj797:feat/turn-metrics-spend-586c

Conversation

@yzxcj797

@yzxcj797 yzxcj797 commented Aug 21, 2026

Copy link
Copy Markdown

Summary

Follow-up to #586's second half: the budget now debits the LlmCallUsage the harnesses meter (#592, #638), but an operator planning which models to run qm on still has to join spend out of session_llm_requests by hand — turn_metrics carried the cache columns and no cost, no output tokens. This lands turn-level spend where the rest of the turn telemetry already lives.

Implementation

  • Harness turn results gain costUsage { outputTokens, costUsd } alongside the existing cacheUsage:
    • pi sums its per-call stats (sumCostUsage next to sumCacheUsage, both return sites);
    • claude reports the SDK's running total_cost_usd delta with usageTotals.output; the finalResult fallback branch keeps output tokens but reports cost 0 — the SDK result carries no cost total, and turnsWithKnownCost distinguishes that downstream;
    • mock reports its deterministic usage so tests observe the pipeline.
  • The orchestrator lands both fields on TurnMetricSample; turn_metrics grows output_tokens BIGINT and cost_usd DOUBLE PRECISION via the sink's ALTER TABLE ADD COLUMN IF NOT EXISTS migration, so existing deployments migrate in place.
  • GET /v1/admin/metrics returns a spend block: samples, turnsWithKnownCost, costUsdTotal, outputTokensTotal — the scope-level number an operator plans against, next to the existing cache block.

Testing

  • New test/turn-metrics-cost.test.ts drives a mock-harness turn through the full app and asserts the spend block on the metrics endpoint, plus the empty-scope shape. Differential: both tests fail with the source changes stashed (no spend block).
  • Neighbors green (metrics-sink, cache-observability, budget: 18/18); tsc --noEmit clean.

Follow-up to #586


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

…/metrics

The budget now debits the LlmCallUsage the harnesses meter, but an
operator planning which models to run qm on still had to join spend out
of session_llm_requests by hand: turn_metrics carried the cache columns
and no cost, no output tokens.

Harness turn results gain costUsage {outputTokens, costUsd} alongside
cacheUsage -- pi sums its per-call stats, claude reports the SDK's
running cost total with the fallback branch flagging unknown cost, and
the mock harness reports its deterministic usage. The orchestrator
lands both on TurnMetricSample, turn_metrics grows output_tokens and
cost_usd (ALTER TABLE ADD COLUMN IF NOT EXISTS, so existing deployments
migrate in place), and /v1/admin/metrics returns a spend block:
samples, turnsWithKnownCost, costUsdTotal, outputTokensTotal.

Follow-up to yc-software#586
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant