Interactive tool for comparing the economics of self-hosted open-weight model inference against frontier API providers.
Live: openshift-psap.github.io/self-hosted-llm-cost-calculator
Calculates the blended cost per million tokens for a self-hosted GPU deployment across 1–100% utilization, and plots it against frontier API costs (Claude, GPT, Gemini, DeepSeek) to show breakeven points.
Key features:
- Live frontier pricing fetched from the OpenRouter API — no API key required
- HuggingFace model search for labeling the self-hosted model
- Workload-aware blended cost — accounts for input:output token ratio and prompt cache hit rate, which heavily influence frontier API costs for agentic workloads
- Single self-hosted cost — unlike frontier APIs that charge differently per token type, self-hosted infrastructure has one fixed cost regardless of whether tokens are input, cached, or output
cost/M = Annual TCO / (total_goodput_tps × seconds_per_year / 1e6)
At utilization U%: cost/M = cost_at_100% / (U / 100)
The cost is the same for all token types because every token — whether prefill (cached or uncached) or decode — consumes compute from the same fixed hardware pool.
blended/M = uncached_input_frac × input_price
+ cached_input_frac × cached_price
+ output_frac × output_price
Where fractions are derived from the serving goodput ratio and cache hit rate. This matters because agentic workloads with high cache hit rates (e.g., 87% for multi-turn sessions) dramatically reduce the effective frontier cost — a naive comparison using list prices overstates the frontier cost.
The defaults model an agentic coding workload served by llm-d on 8 × IBM Cloud 8xH200 nodes (64 H200 GPUs):
| Parameter | Default | Source |
|---|---|---|
| GPU nodes | 8 (8xH200 each) | IBM Cloud |
| Cost/node/month | $11,597.84 | IBM Cloud public pricing |
| Output goodput | 6,000 tok/s | aiperf benchmark (InferenceX CC-Traces Weka, c=128) |
| Input goodput | 13,000 tok/s | aiperf benchmark |
| Cache hit rate | 87% | Multi-turn agentic session prefix reuse |
| Avg ISL | 30,000 tokens | Weka trace dataset |
| Avg OSL | 2,000 tokens | Weka trace dataset |
Single HTML file, no build step. Uses Chart.js 4.x from CDN, vanilla JavaScript, and browser fetch() for API calls (both OpenRouter and HuggingFace support CORS).
Apache-2.0