Surface prompt caching that was requested and never delivered

[aktasbatuhan]

Problem

A host can accept a cache marker on an API layer that does not implement caching, return zero cached tokens, and raise no error. The run then reports uncached cost and latency as though they were properties of the tier or the model.

serving_metrics already avoids this for Anthropic, by measuring it on its native Messages API rather than the OpenAI-compatible layer that "would score Anthropic at 0% cache". Nothing detected the same shape anywhere else.

The live case

Doubleword's caching guide states cache_control is supported on /v1/chat/completions and "isn't supported yet" on /v1/responses. Its async/flex tier is reachable from either API. The doubleword provider block is type: flex, which is the Responses API, and four candidate models route through it.

Measured 2026-09-14 on DeepSeek V4.1 Flash, one account, one byte-identical 4k-token prompt repeated six times per route:

API	Tier	Cached input	Latency
chat completions	realtime	57.8%	0.8-3.8s
chat completions	flex	58.1%	25-28s
responses	flex	0%	159-241s
responses	default	0%	1.1-6.2s

Caching tracks the API, not the tier. A flex run through chat completions pays async rates and still caches; a flex run through responses pays async rates on every input token.

Change

• cache_effectiveness() compares caching requested against caching observed and warns when a route asked on every call and received none. Usage is read in the Responses, Chat Completions and Anthropic shapes, because a missed field name silently reads as a 0% cache.
• compound.yaml records the Doubleword finding against the provider block, so a flex run through responses is not mistaken for evidence that the tier cannot cache.

Tests

549 passed, 3 skipped. New tests cover requested-and-denied, requested-and-served, never-requested, and all three usage shapes.

🤖 Generated with Claude Code

[aktasbatuhan]

Codex review applied

High — the parser inverted its own purpose. Native Anthropic Messages reports input_tokens alongside cache_read_input_tokens, so branching on input_tokens sent that shape down the Responses path and scored Anthropic at 0% cache. Anthropic is the provider the module docstring exists to protect. Its input_tokens also excludes cache reads, so the total was undercounted. Verified by Codex: {"input_tokens":10,"cache_read_input_tokens":90,"cache_creation_input_tokens":5} returned 0% and a warning instead of 85.71%.

The original test passed because it invented the schema (prompt_tokens + cache_read_input_tokens, which no provider emits). Tests now use payloads captured from live runs, plus a cold-write case so a real 0% is not confused with a parse failure.

Low — explicit zero treated as missing. A nested cached_tokens of 0 no longer falls back to a stale top-level counter, so a route that stopped caching is not reported as still caching.

Low — warning overstated. It now reports how many calls asked rather than claiming all of them.

Medium — not wired into shipped reporting. Partially addressed: the helper now accepts cache_marked, which serving cells already emit, so no cell changes are needed. A tracked report caller is still not in this PR.

cache_strategy: none withdrawn. Codex traced it as inert for these routes: the field is read only for direct/<name> (providers_registry.py:286), where this host already defaults to none; doubleword/flex and doubleword/realtime never consult it. TypeScript accepts it via z.looseObject but never forwards it. Declaring it implied enforcement that does not exist, so the measurement is documented as a comment instead.

On the warning threshold, Codex's recommendation is taken: "never observed" stays the strong diagnostic, since partial delivery is explained by cold starts, prefix differences and eviction. The docstring now says to group by host, API and model, because an aggregate can hide one wholly uncached route.

551 passed, 3 skipped.