Skip to content

Let a deployment point model providers at its own gateway - #110

Closed
Restry wants to merge 1 commit into
yc-software:mainfrom
Restry:feat/provider-base-url
Closed

Let a deployment point model providers at its own gateway#110
Restry wants to merge 1 commit into
yc-software:mainfrom
Restry:feat/provider-base-url

Conversation

@Restry

@Restry Restry commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

The in-process Pi harness can only ever reach the vendor endpoints. pi-ai bakes baseUrl into each provider's model record and nothing read an override, so resolveModel always returns https://api.anthropic.com / https://api.openai.com/v1 / https://openrouter.ai/api/v1.

The child-process harnesses are already in a different position. config.ts forwards ANTHROPIC_BASE_URL into claudeProcessEnv and OPENAI_BASE_URL into codexProcessEnv, and each vendor SDK honours them. So on a deployment behind a gateway, Claude Code and Codex work and Pi doesn't — a difference in reach that the harness abstraction is otherwise good at hiding. I hit this running QM against a self-hosted Anthropic-compatible gateway: switching the org to Pi silently pointed every request back at the vendor.

Config now parses ANTHROPIC_BASE_URL, OPENAI_BASE_URL and OPENROUTER_BASE_URL — the same variables the SDKs read — and buildApp hands them to the model registry.

Where the override lands

In builtinModel, not in resolveModel's return. MODEL_REGISTRY clones some entries from a template (claude-opus-5 is a clone of claude-opus-4-8), and cloneModel spreads the template — so overriding the template covers the clone, while overriding only the direct return path would leave clones pointing at the vendor while their template moved. Everything that issues a request resolves through builtinModel, so the agent loop, judge, compaction, title, ack and detect calls all follow one setting.

The value replaces the vendor base URL verbatim. Providers disagree about whether the version prefix belongs in it — anthropic omits /v1 and pi-harness appends /v1/messages, openai includes it — and matching each provider's own convention is what keeps these variables interchangeable with the SDKs that already read them.

Injection rather than a direct env read

pi-models.ts can't import config.ts: config already imports pi-models, so it would be a cycle. And reading process.env from pi-models is what the repo's own lint rule exists to prevent. So config parses, buildApp injects via setProviderBaseUrls, and pi-models keeps the resolved value. Threading it through resolveModel's ~15 call sites across 8 files was the alternative; it seemed a lot of surface for a value that is fixed for the process lifetime.

Admin key validation

putModelProvider posted to a hardcoded vendor URL to check a key. Against a gateway that never sees the vendor's key, that fails a key which actually works, so the admin UI refuses to save it. It now resolves through the same setting.

Verification

npm run typecheck and npm run lint clean; full root suite 3715 tests, 0 failures. New tests cover the parse (including trailing-slash and whitespace normalization) and the resolution, with the clone case pinned explicitly — that one fails if the override moves to resolveModel.

Beyond the suite, this is running on a live single-user deployment pointed at an Anthropic-compatible gateway. With HARNESS=pi the agent completes turns and drives the sandbox through the gateway; both Pi and Claude Code are approved on the same org and both execute against the same gateway with no vendor key present anywhere. Before the change the same instance could only run Claude Code.

No screenshot — this is config plumbing with no rendered surface. The only UI it touches is the admin model-provider form, which behaves as it did except that a gateway-issued key now validates.

Not addressed

secret-schema.ts still requires ANTHROPIC_API_KEY when the base model is Anthropic. A gateway deployment typically carries its token in ANTHROPIC_AUTH_TOKEN (which claudeChildEnv already forwards), so it ends up setting both to the same value. Accepting either felt like a separate call about what the schema is asserting, so I left it.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

pi-ai ships each provider with the vendor endpoint baked into the model
record, and nothing read an override, so the in-process pi harness could
only ever reach api.anthropic.com / api.openai.com / openrouter.ai. The
child-process harnesses were already in a different position: config.ts
forwards ANTHROPIC_BASE_URL and OPENAI_BASE_URL into the claude and codex
child environments, where each vendor SDK honours them. So an operator
running behind a gateway could use Claude Code but not Pi, which is a
difference in reach the harness abstraction is supposed to hide.

Config now parses ANTHROPIC_BASE_URL, OPENAI_BASE_URL and
OPENROUTER_BASE_URL -- the same variables the SDKs use -- and buildApp
hands them to the model registry. The override lands in builtinModel, so
cloned entries inherit it too: claude-opus-5 is a clone of
claude-opus-4-8 and would otherwise keep the vendor endpoint while its
template moved. Everything that issues a request resolves through there,
so the agent loop, judge, compaction, title, ack and detect calls all
follow one setting.

The base URL replaces the vendor value verbatim; providers differ on
whether the version prefix belongs in it (anthropic omits /v1, openai
includes it), and matching each provider's own convention keeps the
variables interchangeable with the SDKs that already read them.

Admin key validation posted to the hardcoded vendor URL, which fails
against a gateway that never sees the vendor's key. It now resolves
through the same setting.
@16francej 16francej added code-pr Code submitted instead of an ADR enhancement New feature or request connector Proposes a new connector/surface/provider labels Aug 4, 2026
pull Bot pushed a commit to danielabelski/qm that referenced this pull request Aug 4, 2026
…tered providers

Two layers of custom model endpoints:

1. ANTHROPIC_BASE_URL / OPENAI_BASE_URL / OPENROUTER_BASE_URL are parsed
   and validated once in config (bad URLs fail at boot), resolved through
   one provider-endpoints module, and applied uniformly: the pi harness,
   the claude/codex child harness process envs, and admin key validation
   (a gateway-issued key validates against the configured endpoint).

2. An org admin can register additional providers that speak the OpenAI
   or Anthropic wire protocol — base URL, API key, and the model ids to
   expose — via PUT /v1/admin/custom-providers/:slug. Registered models
   resolve through the same choke point as built-ins, surface in the
   catalog and model pickers, and serve on the pi and opencode harnesses.
   Keys validate against the registered endpoint (skippable for gateways
   without a models listing), live in the same encrypted store as the
   built-in provider keys, and are write-only. Built-in model ids and
   provider slugs are reserved. The admin portal gets a Custom providers
   card. DeepSeek / Kimi / xAI / a corporate gateway become request
   bodies, not code.

QA hardening: custom keys reach the model runtime for every provider,
models.json materialization is cached per registry version, a corrupt
custom key degrades only its provider, slashed custom ids win the
opencode modelRef lookup, model/name input caps, double-delete 404s,
and the picker refreshes when registrations change.

Addresses the custom-endpoint asks in yc-software#110, yc-software#60, yc-software#116, yc-software#104.
@time-attack

Copy link
Copy Markdown
Collaborator

Thanks for this, @Restry — and sorry for the slow response.

We're closing this PR. Per CONTRIBUTING.md, since coding agents write most of the underlying code now, we'd rather receive feature ideas as a short human-written note in adrs/ — informal is great, just the idea in your own words — and bugs as a plain issue. If we're aligned we're happy to burn our own tokens on the implementation, and we'll credit you as co-author.

We don't want your idea to get lost, so we've captured it in #840 to triage on our side. Please feel free to open an adrs/ note if you feel strongly about the approach. Appreciate you engaging with the repo!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

code-pr Code submitted instead of an ADR connector Proposes a new connector/surface/provider enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants