Skip to content

README.md

Latest commit

 

History

History
261 lines (185 loc) · 8.75 KB

README.md

File metadata and controls

261 lines (185 loc) · 8.75 KB
Raw

plaudapi

Standard wrappers fail when providers flake. plaudapi heals.

PyPI version Python 3.11+ License: MIT Tests

A unified Python SDK for Anthropic + 350+ OpenRouter models with dynamic routing, automatic failover, and heuristic self-repair. One interface. Zero babysitting.

Install

pip install plaudapi
plaudapi setup          # saves API keys to ~/.plaudapi/.env

Quickstart

from plaudapi import PlaudClient

client = PlaudClient()   # FREE_FIRST by default — no API key required to start
resp = client.chat([{"role": "user", "content": "Explain transformers in 2 sentences."}])
print(resp.content)

That's it. plaudapi picks the best available free model from the live OpenRouter catalog, falls back to a hardcoded default if the catalog is unreachable, and retries transient failures with full-jitter backoff — all without any extra wiring.

Why plaudapi?

Raw SDK plaudapi
Multi-provider One package per provider Single unified interface
Free models Manual catalog lookup FREE_FIRST auto-selects from live catalog
Price optimization Hardcode a model and hope COST_OPTIMIZED sorts 350+ models by live pricing
Failover Write your own try/except loop Built-in jitter retry + strategy fallback
Malformed JSON json.JSONDecodeError crashes Heuristic repair closes unclosed braces/quotes
Context overrun Provider 400 error Pre-flight token estimate raises PlaudContextWindowError
Auth errors Confusing 401 response PlaudAuthError tells you exactly which env var is missing
Streaming Different API per provider client.stream() / client.astream() everywhere

Routing Strategies

Pass strategy= to PlaudClient() to change routing behaviour. All 19 strategies share the same chat() / stream() / achat() / astream() API.

Strategy Default model Notes
FREE_FIRST (default) nvidia/nemotron-3-super-120b-a12b:free Dynamically picks any free model
COST_OPTIMIZED dynamic Sorts live catalog by price ascending
QUALITY_FIRST anthropic/claude-opus-4.6-fast Highest context paid model
ANTHROPIC_DIRECT claude-sonnet-4-6 Native Anthropic API (no OpenRouter)
ANTHROPIC anthropic/claude-sonnet-4.6 via OpenRouter
OPENAI openai/gpt-4o via OpenRouter (62 models)
GOOGLE google/gemini-2.0-flash-001 via OpenRouter (32 models)
META meta-llama/llama-4-maverick via OpenRouter (14 models)
DEEPSEEK deepseek/deepseek-v3.2 via OpenRouter
MISTRAL mistralai/mistral-small-2603 via OpenRouter
XAI x-ai/grok-4.20 via OpenRouter
PERPLEXITY perplexity/sonar-pro Search-grounded
COHERE cohere/command-a via OpenRouter
NOUS nousresearch/hermes-4-405b Hermes family
QWEN qwen/qwen3.5-flash-02-23 Alibaba, via OpenRouter
MINIMAX minimax/minimax-m1 via OpenRouter
KIMI moonshotai/kimi-k2.5 Moonshot AI
ZAI z-ai/glm-5.1 GLM family
NVIDIA nvidia/nemotron-3-super-120b-a12b Nemotron family 
from plaudapi import PlaudClient
from plaudapi.router import RoutingStrategy

# Pin to a specific provider family
client = PlaudClient(strategy=RoutingStrategy.DEEPSEEK)

# Or override the model directly while keeping strategy routing
resp = client.chat(
    messages=[{"role": "user", "content": "Hello"}],
    model="deepseek/deepseek-r1",
)

Streaming

Sync streaming

from plaudapi import PlaudClient

client = PlaudClient(strategy="quality_first")

for chunk in client.stream([{"role": "user", "content": "Write me a haiku."}]):
    print(chunk.delta, end="", flush=True)
    if chunk.done:
        print()  # usage stats available on final chunk

Async streaming

import asyncio
from plaudapi import PlaudClient

async def main():
    client = PlaudClient(strategy="openai")
    async for chunk in client.astream(
        [{"role": "user", "content": "Count to 5, one word per line."}]
    ):
        print(chunk.delta, end="", flush=True)

asyncio.run(main())

Browse the Model Catalog

The catalog is fetched from OpenRouter with a 1-hour TTL cache at ~/.plaudapi/models_cache.json. No key required for reads.

List all models

models = client.list_models()
print(f"{len(models)} models available")

# Async
models = await client.alist_models()

Filter with find_models()

# All free models with at least 128k context
free_large = client.find_models(free=True, min_context=128_000)

# Anthropic models sorted cheapest-first
cheap_anthropic = client.find_models(
    provider="anthropic",
    free=False,
    sort_by="price_asc",
)

# Vision-capable models
vision = client.find_models(modality="image", sort_by="context_desc")

for m in cheap_anthropic[:3]:
    print(m["id"], "-", m["pricing"]["prompt"], "$/token")

find_models() supports: free, provider, min_context, modality, sort_by (price_asc, price_desc, context_desc).

Self-Healing

plaudapi bakes six self-healing behaviours directly into the client. You don't configure them — they just work.

1. Auth pre-check

Before the first network call, plaudapi verifies that the required API key is present. Missing keys raise PlaudAuthError immediately with the exact env var name and the fix:

PlaudAuthError: OPENROUTER_API_KEY is not set or invalid.
Run 'plaudapi setup' or add it to ~/.plaudapi/.env

2. Routing fallback

If the live OpenRouter catalog is unreachable (network error, rate limit, timeout), the router silently falls back to a hardcoded default model for the chosen strategy. Your code never sees the exception.

3. JSON auto-repair

When a model returns malformed JSON — trailing commas, unclosed braces, unterminated strings — repair_json() applies heuristic patches before surfacing the result:

from plaudapi.repair import repair_json

parsed, was_repaired = repair_json('{"key": "value",}')
# parsed = {"key": "value"}, was_repaired = True

4. Retry with full-jitter backoff

Network errors and 429 rate limits are retried up to 3 times using full-jitter exponential backoff (random delay in [0, min(cap, base * 2^attempt)]). Works identically for sync and async code.

5. strict_free guardrail

Set strict_free=True to guarantee your app never accidentally uses a paid model. If no free models are found in the catalog, plaudapi raises PlaudFallbackExhaustedError rather than silently upgrading to a paid tier.

client = PlaudClient(strict_free=True)  # raises if zero free models available

6. Context window pre-check

PlaudRequest.estimated_tokens() counts your message length before sending. If it would exceed the model's context limit, PlaudContextWindowError is raised with the token count and the limit — so you can trim the message rather than burn the round-trip.

PlaudContextWindowError: Request has 142000 tokens but model limit is 128000.
Reduce your message length or choose a model with a larger context window.

Debug Mode

Set debug=True to print per-call telemetry to stdout:

client = PlaudClient(debug=True)
resp = client.chat([{"role": "user", "content": "Hi"}])
# [plaudapi] openai/gpt-4o · 12 in / 38 out · $0.00043 · 623ms · openrouter

Exception Reference

PlaudError                        # base
├── PlaudNetworkError             # timeouts, connection failures
│   └── PlaudRateLimitError       # HTTP 429, retries exhausted
├── PlaudAuthError                # missing/invalid API key
├── PlaudFallbackExhaustedError   # strict_free=True, no free models
└── PlaudLogicError               # bad request shape
    ├── PlaudContextWindowError   # exceeds model context limit
    ├── PlaudModelNotFoundError   # model ID not in catalog (with suggestions)
    └── PlaudContentPolicyError   # HTTP 403, provider safety filter

Catch at whatever level makes sense for your application:

from plaudapi.exceptions import PlaudError, PlaudRateLimitError

try:
    resp = client.chat(messages)
except PlaudRateLimitError:
    # all retries failed — back off at the application level
    ...
except PlaudError as e:
    # catch-all for any plaudapi error
    ...

License

MIT — see LICENSE.