Nadir × Kong Gateway

Intelligent, cost-aware model routing for Kong Gateway and Kong AI Gateway, installed as a Kong plugin with zero application code changes.

The whole UX: one string

A team already calling their gateway keeps 100% of their code and changes one field:

  resp = client.chat.completions.create(
-     model="claude-opus-4-6",
+     model="nadir-auto",
      messages=[...],
  )

nadir-auto is a virtual model. Inside Kong, this plugin looks at the prompt, asks Nadir's decision engine which real model fits (Haiku for simple, Sonnet for mid, Opus for complex), and rewrites the request before it leaves the gateway. Auth, provider keys, streaming, rate limits, and logging stay exactly as Kong already does them. This plugin only supplies the routing decision.

There is no Nadir service to run. The plugin is a thin adapter that calls Nadir's hosted decision API (or your own self-hosted instance). The only thing you install into Kong is the plugin.

How it works

client ──"model: nadir-auto"──▶ Kong Gateway
                                     │  nadir plugin (access phase)
                                     ▼
                             resty.http ──▶ POST /v1/recommend   (no LLM call)
                                     │       ◀── {recommended_model, tier, confidence}
                                     ▼
                        request body rewritten to the chosen model
                                     │
                                     ▼
                            upstream provider (OpenAI / Anthropic / …)

The client sees which model was picked on the response it already gets back:

x-nadir-model: claude-haiku-4-5
x-nadir-tier: simple
x-nadir-confidence: 0.983
x-nadir-routed: true

Design guarantees

Guarantee	How
Fail-open	Any decision error / timeout → request proceeds on default_model. If the body is unreadable or not JSON, the plugin touches nothing. A router outage never takes down inference.
Opt-in	Only requests with model: nadir-auto are routed. A real model name is honored and passed through untouched, so you can adopt on 5% of traffic first.
Low latency	Calls the decision-only endpoint (/v1/recommend), which never invokes an LLM. Bounded by timeout_ms (default 250ms) for the whole call, after which it fails open. The budget is split across connect, send and read rather than applied to each, so the worst case is the number you configured, not three times it.
Transparent	The decision rides back as x-nadir-* response headers, with no separate dashboard to consult.

Install (self-hosted Kong)

# 1. Install the plugin from LuaRocks
luarocks install kong-plugin-nadir
#   or, from a checkout:  luarocks make kong-plugin-nadir-0.1.0-1.rockspec

# 2. Enable it in kong.conf (or via env)
#    plugins = bundled,nadir
export KONG_PLUGINS=bundled,nadir

# 3. Point a route at Nadir (declarative example below)
deck gateway sync deck/kong.yaml

Store the API key in a Kong Vault rather than plaintext. The config field is referenceable, so {vault://env/nadir-api-key} reads it from the NADIR_API_KEY environment variable.

Konnect

The same plugin runs on Konnect via custom plugins: upload handler.lua + schema.lua, then enable it on a route from the UI. No LuaRocks, no kong.conf edit.

Configuration

All fields live on the plugin config (schema.lua):

Field	Default	Meaning
api_url	https://api.getnadir.com	Nadir decision API base
recommend_path	/v1/recommend	Decision endpoint (no LLM call)
api_key	(required)	X-API-Key for the decision call (referenceable)
virtual_model	nadir-auto	The sentinel model that opts into routing
default_model	claude-sonnet-4-6	Fail-open target
timeout_ms	250	Per-request decision budget
models	(unset)	Optional candidate set; defaults to the models on the API key
response_headers	true	Stamp x-nadir-* headers on the response

Try it locally

export NADIR_API_KEY=sk-...     # your Nadir key
export OPENAI_API_KEY=sk-...    # upstream provider key for the demo route
docker compose up

curl -s localhost:8000/v1/chat/completions \
  -H 'content-type: application/json' \
  -H "authorization: Bearer $OPENAI_API_KEY" \
  -d '{"model":"nadir-auto","messages":[{"role":"user","content":"hi"}]}' -i

Pairing with ai-proxy

Two deck examples ship here, and both are verified against Kong 3.7:

• deck/kong.yaml: Nadir in front of an OpenAI-compatible upstream. Nadir rewrites the model in the body and the provider honors it.
• deck/kong-ai-gateway.yaml: Nadir + ai-proxy. Nadir supplies the decision, ai-proxy owns provider auth and streaming. Leave model.name unset so ai-proxy takes the model from the (Nadir-rewritten) body.

The pairing works by construction, not by luck. kong.service.request.set_raw_body() calls ngx.req.set_body_data(), which replaces the nginx request buffer that ai-proxy's kong.request.get_body() later reads, so the rewrite is visible to every plugin that runs after Nadir. Ordering holds because Nadir is PRIORITY 775 and ai-proxy is PRIORITY 770, both in the access phase.

test/verify-aiproxy.sh asserts this end to end, and will tell you if a future ai-proxy version stops honoring the mutated buffer.

Plugin ordering

Nadir sits at PRIORITY 775, deliberately between two bounds:

	Priority	Why it matters
auth plugins (e.g. key-auth)	1250	Run before Nadir, so unauthenticated traffic never spends a decision call
nadir	775	Reads the body, rewrites model
ai-proxy	770	Runs after Nadir, dispatches the chosen model

Streaming

stream: true works on the plain-upstream path: the body rewrite lands, the x-nadir-* headers arrive with the response headers, content-type: text/event-stream is preserved, and frames reach the client incrementally. The plugin registers no body_filter, so it never buffers a response.

One measured caveat, and it is not ours: on Kong 3.7, ai-proxy buffers SSE by itself. test/verify-streaming.sh pins this with a control: ai-proxy with no Nadir in the chain buffers identically to ai-proxy with Nadir, so the plugin is neutral. Measured frame spread across a 5-frame stream:

Chain	Frame spread	Behaviour
nadir → plain upstream	~0.90s	streams incrementally
ai-proxy alone (control)	0.00s	buffered
ai-proxy + nadir	0.00s	buffered, identical to control

If streaming latency matters to you, prefer the plain-upstream path.

Testing

A hermetic harness lives in test/. It mocks both the decision API and the LLM upstream, so it needs no API key and spends nothing:

cd test
docker compose -f docker-compose.test.yml up -d
./run-tests.sh          # 12 assertions: rewrite, opt-in, fail-open, headers
./verify-aiproxy.sh     # proves ai-proxy dispatches Nadir's choice
./verify-streaming.sh   # proves SSE survives, and that buffering isn't ours

Self-hosted decision engine

Point api_url at a self-hosted NadirClaw / Nadir instance instead of api.getnadir.com to keep the decision entirely in-network. NadirClaw is the open-source reference engine; the trained Nadir engine is a drop-in swap on the same endpoint contract.

Status

Validated against Kong 3.7 (DB-less, OSS) on the harness in test/: the plugin loads, the schema is accepted, the body rewrite reaches the upstream with a recalculated content-length, fail-open and opt-in behave as documented, the ai-proxy pairing dispatches the chosen model, and SSE streams incrementally with headers intact.

Not yet exercised against: a live api.getnadir.com decision call (the harness mocks it), Konnect, or Kong Enterprise.

License

PolyForm Noncommercial 1.0.0. Free for personal use, research, experimentation, and education. Commercial use (running it in or for a for-profit business) is covered by your Nadir commercial terms, since the plugin only does anything against a Nadir decision API you already pay for. Questions: getnadir.com.