feat(cost-model): break-even tokens-per-day per cloud provider (closes #39)#67
Merged
Conversation
6 tasks
Adds per-cloud-target break-even analysis answering the core FinOps question: at what daily token volume does on-prem beat the cloud for this workload? - calculator: DailyHardwareCost, CloudCostPerToken (blended by actual in/out ratio), BreakEvenTokensPerDay, PercentOfBreakEven (pure, table-tested). - CostProfile.spec.cloudComparison.targets lets operators pick comparable cloud models; defaults to one mid-tier model per provider when unset; targets are validated against the bundled pricing catalog (unknowns logged, not silently dropped). - UsageReport.status.breakEvenAnalysis (per target: break-even/day, current/day, percent, verdict) + a Break-even % print column. costPerMillionTokens and the cloud-comparison metrics are untouched. - REST: GET /api/v1/break-even. Metrics: infercost_break_even_tokens_per_day and infercost_percent_of_break_even gauges feed a new Grafana 'Percent of Break-even' gauge (green at >=100%). - docs/cloud-comparison.md documents the methodology and target selection. Closes #39 Signed-off-by: Christopher Maher <chris@mahercode.io>
Defilan
force-pushed
the
feat/break-even-calculator
branch
from
June 4, 2026 06:54
08855c0 to
9f654b0
Compare
Defilan
added a commit
that referenced
this pull request
Jun 4, 2026
…) (#68) ## What Rewrites the `UsageReport` Ready-condition message (and adds a matching section to `infercost status`) so the headline cost is never shown without utilization context. ## Why Fixes #41 `Period 2026-04-23: $0.0660 across 3,993 tokens` is correct but misleading — readers divide and conclude "$16/MTok, worse than every cloud." The launch runbook says InferCost's differentiator is *honest framing, including when cloud is cheaper*; the bare string doesn't live up to that. The new framing puts the amortized number next to the utilization that explains it, plus the marginal rate and the break-even comparison: ``` Period 2026-04-23: $0.0660 across 3,993 tokens. Amortized: $16.52/MTok at 4% utilization. Marginal: $0.0040/MTok. Break-even with Anthropic/claude-opus-4-6: 142K tokens/day (you served 6K). ``` ## How - **`internal/report.StatusMessage(UsageReportStatus)`** — a pure, table-tested formatter, shared by the controller and CLI so their framing is identical. Marginal line is omitted when 0 (no energy sampled → "$0.0000" would be misleading); break-even line omitted when there are no targets. - **Controller** — `applyStatusIfChanged` sets the Ready message via the formatter. - **CLI** — `infercost status` gains a "USAGE REPORTS" section printing the same framing per report. - **Tests** — formatter table tests (humanize, commas, conditional lines) + a controller assertion that the message carries the new framing. - **Grafana** — the utilization %, $/MTok panels and the new break-even gauge (#39) already carry this visually; a Prometheus dashboard can't render a CRD condition string, so no message panel is added. ## Note on stacking Builds on **#39 / PR #67** (the break-even line reads `status.breakEvenAnalysis`). This PR is based on `feat/break-even-calculator`; merge #67 first (GitHub will retarget this to `main`). ## Checklist - [x] Status condition message updated in `applyStatusIfChanged` - [x] CLI `infercost status` output matches - [x] Unit tests assert the new phrasing - [x] `make test` + `make lint` pass locally - [x] Signed off (DCO); conventional commit - [x] Grafana: covered by existing panels + break-even gauge (documented above) Signed-off-by: Christopher Maher <chris@mahercode.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds per-cloud-target break-even tokens/day analysis: the daily token volume at which running a workload on-prem costs the same as a comparable cloud model. Above break-even, on-prem is cheaper.
Why
Fixes #39
A flat
$/MTokcan't answer the real FinOps question because on-prem cost-per-token depends on throughput (idle hardware still costs money) while cloud cost-per-token doesn't. Break-even collapses both into one comparable number and tells the operator where on the curve they are.How
DailyHardwareCost(amortization/day + idle electricity/day),CloudCostPerToken(input/output prices blended by the period's actual in/out ratio, 50/50 fallback),BreakEvenTokensPerDay,PercentOfBreakEven.CostProfile.spec.cloudComparison.targetslets operators pick comparable models (a small coder breaks even against a budget model, not a flagship). Defaults to one mid-tier model per provider (gpt-5.4-mini,claude-sonnet-4-6,gemini-2.5-flash) so it works zero-config. Targets are validated against the bundled pricing catalog; unknown ones are logged, not silently dropped.UsageReport.status.breakEvenAnalysis(provider, model, breakEvenTokensPerDay, currentUtilizationTokensPerDay, percentOfBreakEven, verdict) + aBreak-even %print column.costPerMillionTokensand the existing cloud-comparison metrics are untouched.GET /api/v1/break-even. Metrics:infercost_break_even_tokens_per_day+infercost_percent_of_break_even(labels cost_profile/provider/cloud_model) feed a new "Percent of Break-even" Grafana gauge (green at >=100%).docs/cloud-comparison.mddocuments the methodology + target selection;docs/cost-model.mdupdated.Design note: break-even lives in
UsageReport.Status(it needs the period's tokens), and the UsageReport controller now emits its first Prometheus metrics for the gauge.Checklist
make testpasses locallymake lintpasses locally (0 issues)git commit -s) per DCO