Fix Piebald cached input accounting

[mike1858]

Summary

• separate tiered cache pricing from provider input-token semantics
• normalize Piebald OpenAI-style cached reads out of input tokens before reporting/costing
• add focused Piebald tests for OpenAI, tiered OpenAI, Anthropic, and saturating normalization

Verification

• cargo test piebald --quiet
• cargo build --quiet
• cargo test --quiet
• cargo clippy --quiet -- -D warnings
• cargo doc --quiet
• cargo fmt --all --quiet

Summary by CodeRabbit

• Bug Fixes

  • Fixed token counting for cached reads to properly account for provider-specific semantics
  • Improved cache token cost calculations across different AI model providers

• Tests

  • Added comprehensive test coverage for token normalization behavior

[coderabbitai]

Caution

Review failed

Pull request was closed or merged during review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: ef0c8fbb-3c9e-4f6a-ad4b-0a55940eae8f

📥 Commits

Reviewing files that changed from the base of the PR and between 33860af and b480f20.

📒 Files selected for processing (2)

• src/analyzers/piebald.rs
• src/models.rs

---

📝 Walkthrough

Walkthrough

This PR adds provider-specific token accounting semantics to distinguish how different LLM providers report cached input tokens. OpenAI includes cache reads in raw input counts; Anthropic does not. A new InputTokenSemantics enum and normalization logic adjust token counts accordingly during message analysis, and CachingSupport::Google is renamed to CachingSupport::Tiered for clarity.

Changes

Provider-specific token semantics for cache reads

Layer / File(s)	Summary
Token Semantics Type System and ModelInfo src/models.rs	InputTokenSemantics enum with ExcludesCache and IncludesCacheRead variants is introduced alongside CachingSupport::Tiered rename; ModelInfo gains a new serde-defaulted input_token_semantics field. Test imports are updated.
Model Registration and Semantics Derivation src/models.rs	Helper input_token_semantics_for_model derives semantics from model name patterns (gpt-*, o* → IncludesCacheRead); default models are registered with appropriate semantics and updated caching variant references; test fixtures populate the new field.
CachingSupport Refactor and Validation/Cache-Cost Logic src/models.rs	Validation and cache-cost calculation match arms handle the renamed CachingSupport::Tiered variant; free-tier model construction and test data are updated; tiered-model lookup test assertion is corrected to match the new variant name.
Input Token Normalization in Message Analysis src/analyzers/piebald.rs, src/models.rs	Imports include InputTokenSemantics and get_model_info; normalize_input_tokens helper conditionally subtracts cache-read tokens based on model semantics with saturating subtraction; convert_messages uses normalized counts; tests validate behavior across OpenAI base, OpenAI tiered (gpt-5.5), and Anthropic models.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• Piebald-AI/splitrail#136: Both PRs modify src/analyzers/piebald.rs's convert_messages to enrich ConversationMessage parsing—main PR adjusts input_tokens via model cache-read semantics, while PR #136 adds per-message tool_calls counts into the stats flow.
• Piebald-AI/splitrail#151: The main PR's new InputTokenSemantics and normalize_input_tokens logic in models.rs/analyzers/piebald.rs directly impacts how GPT-5.5 cached-input token counts are interpreted for pricing, which aligns with the retrieved PR's GPT-5.5 long-context cached-input pricing/tiered updates.

Poem

🐰 A token walks in, unsure if cache-read's in—
We ask the provider: does your count include the spin?
OpenAI says yes, Anthropic says no,
Normalize it down, let the costs fairly flow!
Tiered caching renamed, semantics refined—
Provider-aware pricing, well-designed! 🥕

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'Fix Piebald cached input accounting' directly reflects the main change: normalizing cached input token accounting for Piebald OpenAI-style inputs.
Docstring Coverage	✅ Passed	Docstring coverage is 90.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix-piebald-cache-token-semantics

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[mike1858]

Additional context on why this change exists:

Piebald persists provider-native token usage into its messages table. That means different providers have different semantics for input_tokens relative to cached input:

• OpenAI-style usage reports input_tokens as uncached_input + cached_input, with cached_tokens/cache_read_tokens as a subset.
• Anthropic-style usage reports input_tokens separately from cache_read_input_tokens / cache_creation_input_tokens.

Splitrail’s normalized stats model expects input_tokens and cached_tokens to be additive buckets. So for Piebald rows backed by OpenAI-style usage, Splitrail must subtract cache reads before reporting/costing:

normalized_input = reported_input - cache_read
cached = cache_read + cache_write

The first version of this fix tried to infer that behavior from CachingSupport::OpenAI, but that missed gpt-5.4 / gpt-5.5 because those models use tiered cached-input pricing and were represented as CachingSupport::Google(...). That variant was really “tiered cache pricing,” not “Google provider semantics,” which made the name misleading and caused the normalization to skip large GPT usage.

This PR separates those concepts:

• CachingSupport::Tiered(...) now describes tiered cached-input pricing.
• InputTokenSemantics now describes whether a model’s reported input includes cache reads.
• The Piebald analyzer uses InputTokenSemantics::IncludesCacheRead to normalize OpenAI-style rows before computing stats and cost.

Concrete example:

OpenAI/Piebald row:
  input_tokens = 1000
  cache_read_tokens = 300

Before:
  Splitrail input = 1000
  Splitrail cached = 300
  input + cached = 1300

After:
  Splitrail input = 700
  Splitrail cached = 300
  input + cached = 1000

This fixes the inflated input/cost accounting for tiered GPT models like gpt-5.5, while preserving Anthropic-style accounting where input and cache tokens are already separate.