feat(pricing): add models.dev-backed tier-aware costs - #458
Merged
Conversation
Use models.dev as the primary provider-scoped pricing catalog while retaining LiteLLM and OpenRouter fallbacks. Add conflict-safe matching, ETag caching, indexed lookup, and conditional source fetching to reduce sync latency and network work. Preserve manual confirmation for ambiguous pricing and avoid stale-cache fallback after source failures.
Update model price synchronization guidance in all supported locales. Identify models.dev as the preferred source and LiteLLM and OpenRouter as fallbacks. Keep the displayed behavior aligned with the Manager Server implementation without changing the frontend API contract.
Document source priority, provider-scoped identity handling, and conflict confirmation. Clarify which models.dev cost fields are currently mapped into billing rules. Note that tiers, Fast Mode, and reasoning metadata remain available but are not yet billed automatically.
Create SQLite tables for context and service-tier price rules and durable pricing rollups. Add indexes, state initialization, and cascade coverage required by later repositories. This schema-only change leaves application behavior unchanged until subsequent commits.
Extend model prices with normalized context and service-tier rules. Persist, validate, and reload those rules transactionally through the model-price repository. This establishes the price-book contract used by synchronization and cost calculation.
Classify usage by resolved pricing model, context threshold, and effective service tier. Apply explicit context and service-tier rates while preserving legacy long-context behavior. Add query-plan and cost regression coverage for mixed pricing bands.
Persist hourly and account pricing-band aggregates with migration-aware rebuild state. Expose transaction-consistent snapshots and block price mutations during multi-query reads. This keeps analytics fast while preserving a coherent price book during concurrent synchronization.
Parse context and Priority/Fast pricing rules from models.dev and notify rollup refreshes. Bound source latency, retain valid ETag data, reject unusable catalogs, and preserve failed-source prices. This keeps models.dev preferred without losing last-known-good pricing during outages.
Merge pricing-band rollups with pending raw usage in the shared hourly reader. Preserve core usage metrics while exposing classified rows to cost-aware consumers. This centralizes the optimized data path used by dashboard and monitoring services.
Start a pricing rollup worker and wake it after usage, migration, and price changes. Integrate bounded catch-up and failure recording into the collector fanout. This keeps pricing aggregates current without blocking request processing.
Use consistent price snapshots and classified rollups when calculating dashboard costs. Handle Priority/Fast, Flex/Batch, and long-context bands without collapsing mixed aggregates. Add correctness and performance coverage for dashboard refresh paths.
Price monitoring timelines, model totals, and account history from classified pricing bands. Use snapshot-safe price reads and durable account rollups for large histories. Add regression and benchmark coverage for mixed service tiers and context lengths.
Display synchronized context and service-tier rules and warn when manual prices replace them. Calculate mixed Priority/Fast and long-context usage consistently with the backend. Update API types, tests, styling, and all supported locales.
Document models.dev context and service-tier pricing rules in both manuals. Explain last-known-good fallback behavior and how manual replacement clears synchronized rules. Clarify Priority/Fast, Flex/Batch, and long-context cost semantics.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Use models.dev as the preferred public model-price catalog with LiteLLM and OpenRouter fallbacks, bounded synchronization, and last-known-good retention.
Add context-tier and explicit Fast/Priority pricing across backend rollups, Dashboard, Monitoring, and frontend cost estimation.
Scope
Changes
User Impact
Cost estimates now distinguish models.dev context tiers and explicit Fast/Priority prices instead of collapsing all usage into one base rate. Existing prices remain available during source or network failures, and the Model Prices page exposes synchronized advanced rules. Provider billing and CPA routing are unchanged.
Compatibility / Runtime Notes
Data / Security Notes
Adds SQLite tables for model-price tiers, pricing rollups, and rollup state. Existing base prices and raw usage remain local.
Catalog synchronization reads public pricing metadata only; it does not send usage events, management keys, credentials, or authentication files to models.dev.
No authentication, secret-storage, or CPA Management Key behavior changes.
Risk / Rollback
Risk level: Medium
Rollback notes:
Revert the PR or deploy the previous binary. The added SQLite tables are additive and may remain safely unused by older versions; existing base model-price rows are preserved. This changes local cost estimation only and does not affect provider billing or request routing.
Verification
Commands / evidence:
Frontend tests passed across 131 files and 1,213 tests. Repeated concurrency/race tests and pricing-rollup performance smoke benchmarks also passed. Manual UI and docs-link checks were not run; tier display and cost behavior are covered by automated tests.
Screenshots / Recordings
Not captured. The visible changes are data-dependent tier badges and a manual-replacement warning on the existing Model Prices page; automated model and UI tests cover their rendering decisions.
Docs
Docs decision:
README and both model-price manuals are updated. Existing Model Prices navigation and deep links remain unchanged. Release notes should be added in the next versioned release PR.
Related
N/A — no dedicated issue is linked.