A self-hosted tool to health-check any LLM API (latency / TTFT / TPS / concurrency) and verify whether a relay actually serves the model it claims.
The market for LLM APIs is messy: official endpoints, cloud vendors, and all kinds of "relays/resellers". Two things stay invisible to you as a user:
- Opaque performance — vendors won't tell you the real latency, time-to-first-token (TTFT), throughput (TPS), or concurrency behavior of a given key.
- Opaque authenticity — a relay claims it serves
GPT-5.5orClaude Opus 4.8, but may quietly swap in a cheaper model. You have no way to tell.
LLM-API-Lens answers both: enter a key + model, click once, and within a few minutes get a performance report and a "is it being downgraded?" verdict.
Design principle: relays are pluggable.
tokunthat appears in the code is only an example — any OpenAI/Anthropic-compatible endpoint plugs in as a "Source".
1. Performance health-check — single-stream baseline + stepped concurrency load test, measuring TTFT, TPS, end-to-end latency, concurrency degradation, and success rate.
2. Authenticity verification (confidence-tiered, not binary) — a set of forensic probes (identity, knowledge cutoff, capability floor, behavioral fingerprint, self-consistency) compared against a baseline, producing one of:
- ✅ Consistent with claim
⚠️ Inconclusive- ❌ Strongly suspected downgrade
Two comparison modes:
- Without an official key (default) — compared against a built-in fingerprint library baseline (starts from public benchmarks, self-grows over time).
- With an official key (optional, gold standard) — the same probes hit the relay and the official endpoint simultaneously for a live differential; the official results are written back into the library, making it more accurate.
Honest limits: verification is probabilistic. It reliably catches cross-generation/family downgrades (e.g. "5.5" that is really "4"), but cannot catch quantization, occasional model-mixing, or an adversary that routes benchmark queries to the genuine model (unsolvable in a black box; see arXiv 2504.04715). When a model has neither an official key nor a baseline in the library, the tool honestly returns "unverifiable" instead of faking a pass.
3. Cost guard — every test is capped by tokens (default 0.3M) and stops when the cap is reached, so you never overspend by accident.
- Backend: Python + FastAPI + httpx/asyncio + SQLAlchemy (async) + SQLite
- Frontend: React + Vite
- Tests: pytest (all run against a free in-process mock — no spend)
# 1. Backend
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
# 2. Build the frontend (needs Node.js)
cd frontend && npm install && npm run build && cd ..
# 3. Run (serves the frontend automatically)
./run.sh # open http://127.0.0.1:8800Keys are entered in the UI and stored in a local SQLite file — never uploaded.
.env is for local testing only and is excluded by .gitignore.
Run tests: source .venv/bin/activate && pytest
This project uses the PolyForm Noncommercial License 1.0.0 (source-available):
- ✅ Anyone may download, modify, and distribute it for non-commercial purposes (personal, study, research, hobby, etc.), free of charge.
- ❌ Commercial use requires prior permission from, or payment to, the author.
- For a commercial license, contact @luyileo810 on GitHub (open an issue or reach out).
Questions, suggestions, or commercial inquiries are welcome — please open an Issue in this repo, or contact @luyileo810 on GitHub.
给大模型 API 做「性能体检 + 真伪验证」的自托管工具。
设计原则:中转站可插拔 —— 代码里出现的 tokun 只是一个示例,任何 OpenAI / Anthropic 兼容端点都能作为「来源」接入。
市面上大模型 API 来源很杂:官方、云厂商、各种「中转站」。你会遇到两个看不见的问题:
- 性能不透明 —— 厂商不会告诉你某个 Key 的真实延迟、首字时间(TTFT)、 吞吐(TPS)、并发能力到底如何。
- 真伪不透明 —— 中转站宣称给你的是
GPT-5.5/Claude Opus 4.8, 但实际可能偷偷换成更便宜的模型。作为用户你无从分辨。
LLM-API-Lens 就是用来回答这两个问题的:填入一个 Key + 模型,点一下, 几分钟内给出性能报告和「是否被降级」的判断。
1. 性能体检 —— 对目标模型做单流基线 + 阶梯并发压测,测量 TTFT、TPS、 端到端延迟、并发衰减、成功率。
2. 质量验真(置信度分级,不是二元真假) —— 用一组「取证探针」(身份、 知识截止线、能力地板、行为指纹、自洽性)对比模型表现,输出三档结论:
- ✅ 符合声称
⚠️ 存疑- ❌ 高度怀疑降级
两种对照方式:
- 无官方 Key(默认) —— 对照内置「指纹库基线」(公开基准起步,可自生长)。
- 有官方 Key(可选,金标准) —— 同一批探针同时打中转站和官方,实时差分; 官方结果会自动写回指纹库,越用越准。
诚实声明: 验真是概率性的。它能高置信度抓到跨代/跨族降级(如 5.5 实为 4), 但对量化、偶发混用、或对手专门把基准题路由到真模型等情况无能为力 (黑盒下无解,参见 arXiv 2504.04715)。当某模型既无官方 Key、指纹库里又没有它的基线时, 工具会如实返回**「无法验证」**,而不会假装通过。
3. 成本保护 —— 每次测试按 token 设上限(默认 0.3M),触顶即停,避免意外烧钱。
- 后端:Python + FastAPI + httpx/asyncio + SQLAlchemy(async) + SQLite
- 前端:React + Vite
- 测试:pytest(全部跑在内置免费 mock 上,不花钱)
# 1. 后端
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
# 2. 前端构建(需要 Node.js)
cd frontend && npm install && npm run build && cd ..
# 3. 启动(自动托管前端)
./run.sh # 打开 http://127.0.0.1:8800密钥通过界面录入,存于本地 SQLite,不会上传。.env 仅用于本地测试且已被
.gitignore 排除。
跑测试:source .venv/bin/activate && pytest
本项目采用 PolyForm Noncommercial License 1.0.0(源码可见 / source-available):
- ✅ 任何人可免费下载、修改、分发用于非商业目的(个人、学习、研究、爱好等)。
- ❌ 商业使用需事先获得作者授权或付费。
- 想商用?请通过 GitHub 联系作者 @luyileo810(提 issue 或私信)。
有任何问题、建议或商用合作,欢迎在本仓库 提 Issue, 或在 GitHub 上联系 @luyileo810。