Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -482,6 +482,42 @@ curl http://localhost:3001/v1/chat/completions \
}'
```

**Routing strategies (`auto:*`)**

Plain `auto` follows your active fallback chain. Add a suffix to steer a single request instead — no dashboard changes needed:

- `auto:smart` — favor the highest-intelligence models
- `auto:fast` — favor measured speed (throughput and time-to-first-byte)
- `auto:cheap` — budget-leaning; currently the same blend as `balanced` (everything in the pool is already free)
- `auto:reliable` — favor recent success rate
- `auto:balanced` — the default blend (reliability first, speed and intelligence split the rest)

These rank **every enabled model**, ignoring your chain order. Common synonyms resolve too (`auto:fastest`, `auto:speed`, `auto:smartest`, `auto:cheapest`, `auto:budget`, …), and the whole model string is case-insensitive.

```bash
curl http://localhost:3001/v1/chat/completions \
-H "Authorization: Bearer freellmapi-your-unified-key" \
-H "Content-Type: application/json" \
-d '{
"model": "auto:fast",
"messages": [{"role": "user", "content": "hi"}]
}'
```

`auto:<profile-name>` routes through a named profile's chain instead of the active one, so different tools can use different chains through the same key:

```bash
curl http://localhost:3001/v1/chat/completions \
-H "Authorization: Bearer freellmapi-your-unified-key" \
-H "Content-Type: application/json" \
-d '{
"model": "auto:coding",
"messages": [{"role": "user", "content": "Write a binary search in Rust."}]
}'
```

An unknown profile name returns a clear `400` rather than silently falling back. Profiles are the named fallback chains from **Model profiles** (see [Features](#features)) — create and switch them from the dashboard; whichever is active is what plain `auto` uses.

**Streaming**

```python
Expand Down Expand Up @@ -867,6 +903,11 @@ See [CONTRIBUTING.md](./CONTRIBUTING.md) for the full migration CLI and workflow
<a href="https://github.com/noobix"><img src="https://images.weserv.nl/?url=github.com/noobix.png&w=60&h=60&fit=cover&mask=circle" width="60" alt="@noobix" /></a>
<a href="https://github.com/nandukmelath"><img src="https://images.weserv.nl/?url=github.com/nandukmelath.png&w=60&h=60&fit=cover&mask=circle" width="60" alt="@nandukmelath" /></a>
<a href="https://github.com/coffcoe"><img src="https://images.weserv.nl/?url=github.com/coffcoe.png&w=60&h=60&fit=cover&mask=circle" width="60" alt="@coffcoe" /></a>
<a href="https://github.com/NirvanaCh7"><img src="https://images.weserv.nl/?url=github.com/NirvanaCh7.png&w=60&h=60&fit=cover&mask=circle" width="60" alt="@NirvanaCh7" /></a>
<a href="https://github.com/Mohamed3nan"><img src="https://images.weserv.nl/?url=github.com/Mohamed3nan.png&w=60&h=60&fit=cover&mask=circle" width="60" alt="@Mohamed3nan" /></a>
<a href="https://github.com/Arman-Espiar"><img src="https://images.weserv.nl/?url=github.com/Arman-Espiar.png&w=60&h=60&fit=cover&mask=circle" width="60" alt="@Arman-Espiar" /></a>
<a href="https://github.com/MetaMysteries8"><img src="https://images.weserv.nl/?url=github.com/MetaMysteries8.png&w=60&h=60&fit=cover&mask=circle" width="60" alt="@MetaMysteries8" /></a>
<a href="https://github.com/lujun880726"><img src="https://images.weserv.nl/?url=github.com/lujun880726.png&w=60&h=60&fit=cover&mask=circle" width="60" alt="@lujun880726" /></a>

## Terms of Service review

Expand Down
15 changes: 14 additions & 1 deletion client/src/i18n/locales/en.json
Original file line number Diff line number Diff line change
Expand Up @@ -409,7 +409,20 @@
"recentCalls": "Recent calls",
"clientIp": "Client IP",
"clientAgent": "Client app",
"requestedModelHint": "Requested {model}, served by fallback"
"requestedModelHint": "Requested {model}, served by fallback",
"attempts": "Attempts",
"filterAll": "All",
"providerBreakdown": "Per-provider breakdown",
"tokensPerSec": "Tok/s",
"allProviders": "All providers",
"requestDetailTitle": "Request #{id}",
"failoverLadder": "Failover ladder",
"failoverLadderHint": "Each hop this request took through the fallback chain, in order.",
"noAttemptTrace": "No attempt trace was recorded for this request.",
"keyOrdinal": "key #{n}",
"attemptTimingHint": "Start offset from request start · attempt duration",
"requestedModel": "Requested model",
"ttft": "TTFT"
},
"playground": {
"title": "Playground",
Expand Down
15 changes: 14 additions & 1 deletion client/src/i18n/locales/es.json
Original file line number Diff line number Diff line change
Expand Up @@ -392,7 +392,20 @@
"recentCalls": "Llamadas recientes",
"clientIp": "IP del cliente",
"clientAgent": "Aplicación cliente",
"requestedModelHint": "Se solicitó {model}, servido por respaldo"
"requestedModelHint": "Se solicitó {model}, servido por respaldo",
"attempts": "Intentos",
"filterAll": "Todas",
"providerBreakdown": "Desglose por proveedor",
"tokensPerSec": "Tok/s",
"allProviders": "Todos los proveedores",
"requestDetailTitle": "Solicitud #{id}",
"failoverLadder": "Escalera de failover",
"failoverLadderHint": "Cada salto que esta solicitud dio por la cadena de respaldo, en orden.",
"noAttemptTrace": "No se registró ninguna traza de intentos para esta solicitud.",
"keyOrdinal": "clave #{n}",
"attemptTimingHint": "Desfase desde el inicio de la solicitud · duración del intento",
"requestedModel": "Modelo solicitado",
"ttft": "TTFT"
},
"playground": {
"title": "Entorno de pruebas",
Expand Down
15 changes: 14 additions & 1 deletion client/src/i18n/locales/fr.json
Original file line number Diff line number Diff line change
Expand Up @@ -392,7 +392,20 @@
"recentCalls": "Appels récents",
"clientIp": "IP du client",
"clientAgent": "Application cliente",
"requestedModelHint": "{model} demandé, servi par repli"
"requestedModelHint": "{model} demandé, servi par repli",
"attempts": "Tentatives",
"filterAll": "Toutes",
"providerBreakdown": "Détail par fournisseur",
"tokensPerSec": "Tok/s",
"allProviders": "Tous les fournisseurs",
"requestDetailTitle": "Requête #{id}",
"failoverLadder": "Échelle de failover",
"failoverLadderHint": "Chaque étape parcourue par cette requête dans la chaîne de repli, dans l'ordre.",
"noAttemptTrace": "Aucune trace de tentative n'a été enregistrée pour cette requête.",
"keyOrdinal": "clé #{n}",
"attemptTimingHint": "Décalage depuis le début de la requête · durée de la tentative",
"requestedModel": "Modèle demandé",
"ttft": "TTFT"
},
"playground": {
"title": "Bac à sable",
Expand Down
15 changes: 14 additions & 1 deletion client/src/i18n/locales/it.json
Original file line number Diff line number Diff line change
Expand Up @@ -392,7 +392,20 @@
"recentCalls": "Chiamate recenti",
"clientIp": "IP del client",
"clientAgent": "App client",
"requestedModelHint": "Richiesto {model}, servito dal fallback"
"requestedModelHint": "Richiesto {model}, servito dal fallback",
"attempts": "Tentativi",
"filterAll": "Tutte",
"providerBreakdown": "Dettaglio per provider",
"tokensPerSec": "Tok/s",
"allProviders": "Tutti i provider",
"requestDetailTitle": "Richiesta #{id}",
"failoverLadder": "Scala di failover",
"failoverLadderHint": "Ogni passaggio di questa richiesta lungo la catena di fallback, in ordine.",
"noAttemptTrace": "Nessuna traccia dei tentativi registrata per questa richiesta.",
"keyOrdinal": "chiave #{n}",
"attemptTimingHint": "Offset dall'inizio della richiesta · durata del tentativo",
"requestedModel": "Modello richiesto",
"ttft": "TTFT"
},
"playground": {
"title": "Playground",
Expand Down
15 changes: 14 additions & 1 deletion client/src/i18n/locales/pt-BR.json
Original file line number Diff line number Diff line change
Expand Up @@ -392,7 +392,20 @@
"recentCalls": "Chamadas recentes",
"clientIp": "IP do cliente",
"clientAgent": "Aplicativo cliente",
"requestedModelHint": "Solicitado {model}, servido por fallback"
"requestedModelHint": "Solicitado {model}, servido por fallback",
"attempts": "Tentativas",
"filterAll": "Todas",
"providerBreakdown": "Detalhamento por provedor",
"tokensPerSec": "Tok/s",
"allProviders": "Todos os provedores",
"requestDetailTitle": "Requisição #{id}",
"failoverLadder": "Escada de failover",
"failoverLadderHint": "Cada salto que esta requisição deu pela cadeia de fallback, em ordem.",
"noAttemptTrace": "Nenhum rastro de tentativas foi registrado para esta requisição.",
"keyOrdinal": "chave #{n}",
"attemptTimingHint": "Deslocamento desde o início da requisição · duração da tentativa",
"requestedModel": "Modelo solicitado",
"ttft": "TTFT"
},
"playground": {
"title": "Playground",
Expand Down
15 changes: 14 additions & 1 deletion client/src/i18n/locales/zh-CN.json
Original file line number Diff line number Diff line change
Expand Up @@ -392,7 +392,20 @@
"recentCalls": "最近调用",
"clientIp": "客户端 IP",
"clientAgent": "客户端应用",
"requestedModelHint": "请求了 {model},由回退模型提供"
"requestedModelHint": "请求了 {model},由回退模型提供",
"attempts": "尝试次数",
"filterAll": "全部",
"providerBreakdown": "按提供商明细",
"tokensPerSec": "Tok/s",
"allProviders": "全部提供商",
"requestDetailTitle": "请求 #{id}",
"failoverLadder": "故障转移阶梯",
"failoverLadderHint": "该请求沿回退链经过的每一跳,按顺序排列。",
"noAttemptTrace": "此请求未记录尝试轨迹。",
"keyOrdinal": "密钥 #{n}",
"attemptTimingHint": "自请求开始的偏移 · 尝试耗时",
"requestedModel": "请求的模型",
"ttft": "TTFT"
},
"playground": {
"title": "试玩台",
Expand Down
Loading
Loading