per-request-cost-telemetry

Per-request billing telemetry contract for an LLM API gateway: a machine-checkable field schema, sample records, and a validator.

Why this exists

Most API gateways show you a monthly total. You cannot reconcile a monthly total against your own request log — you have to take the provider's word for it.

This repo publishes the exact shape of the telemetry record emitted per request by the gateway we operate, so that any customer can:

1. Capture the record attached to each of their own requests.
2. Re-compute cost locally from prompt_tokens, completion_tokens and the published per-model unit price.
3. Cross-check the result against the dashboard balance ledger.

A production customer of ours did exactly this as an independent audit: gateway records, dashboard ledger, and upstream third-party metering all matched. That audit was performed by the customer, on the customer's own tooling, without our involvement — which is the point. If the contract holds, the provider is not needed for verification at all.

Contents

Path	What it is
schema/telemetry.schema.json	JSON Schema for one telemetry record
samples/sample_events.jsonl	Synthetic sample records (all fields populated)
validate.py	Standalone validator (Python 3.9+, stdlib only)

Field contract (summary)

Field	Type	Meaning
request_id	string	Unique id, also returned in the response header
model	string	Model identifier actually served
prompt_tokens	int	Input tokens metered
completion_tokens	int	Output tokens metered
cached_tokens	int	Input tokens served from cache, if any
latency_ms	object	{ gateway_overhead, upstream, total } split
cost_usd	number	Computed cost for this request, 6 decimal places
failover	bool	Whether the request was transparently retried
ts	string	ISO-8601 UTC timestamp

The full contract, including edge cases (streamed responses, cached prompts, failover retries), is in the schema file.

Usage

python3 validate.py samples/sample_events.jsonl
# OK: 6 records, all fields valid, recomputed cost matches cost_usd (6 dp)

The validator also re-computes cost_usd from token counts and a price table you supply, so you can verify arithmetic rather than just shape.

Scope and honesty

• Sample data is synthetic; field semantics are production-accurate.
• latency_ms.upstream varies by model and load; only gateway_overhead is under our direct control. We publish the split precisely because we do not want our overhead blamed for upstream variance.
• No claims are made about third-party pricing stability. Verify unit prices against the dashboard at audit time.

---

Maintained by NovaRouteAI. Questions or discrepancies: open an issue.