Let a deployment point model providers at its own gateway - #110
Closed
Restry wants to merge 1 commit into
Closed
Conversation
pi-ai ships each provider with the vendor endpoint baked into the model record, and nothing read an override, so the in-process pi harness could only ever reach api.anthropic.com / api.openai.com / openrouter.ai. The child-process harnesses were already in a different position: config.ts forwards ANTHROPIC_BASE_URL and OPENAI_BASE_URL into the claude and codex child environments, where each vendor SDK honours them. So an operator running behind a gateway could use Claude Code but not Pi, which is a difference in reach the harness abstraction is supposed to hide. Config now parses ANTHROPIC_BASE_URL, OPENAI_BASE_URL and OPENROUTER_BASE_URL -- the same variables the SDKs use -- and buildApp hands them to the model registry. The override lands in builtinModel, so cloned entries inherit it too: claude-opus-5 is a clone of claude-opus-4-8 and would otherwise keep the vendor endpoint while its template moved. Everything that issues a request resolves through there, so the agent loop, judge, compaction, title, ack and detect calls all follow one setting. The base URL replaces the vendor value verbatim; providers differ on whether the version prefix belongs in it (anthropic omits /v1, openai includes it), and matching each provider's own convention keeps the variables interchangeable with the SDKs that already read them. Admin key validation posted to the hardcoded vendor URL, which fails against a gateway that never sees the vendor's key. It now resolves through the same setting.
pull Bot
pushed a commit
to danielabelski/qm
that referenced
this pull request
Aug 4, 2026
…tered providers Two layers of custom model endpoints: 1. ANTHROPIC_BASE_URL / OPENAI_BASE_URL / OPENROUTER_BASE_URL are parsed and validated once in config (bad URLs fail at boot), resolved through one provider-endpoints module, and applied uniformly: the pi harness, the claude/codex child harness process envs, and admin key validation (a gateway-issued key validates against the configured endpoint). 2. An org admin can register additional providers that speak the OpenAI or Anthropic wire protocol — base URL, API key, and the model ids to expose — via PUT /v1/admin/custom-providers/:slug. Registered models resolve through the same choke point as built-ins, surface in the catalog and model pickers, and serve on the pi and opencode harnesses. Keys validate against the registered endpoint (skippable for gateways without a models listing), live in the same encrypted store as the built-in provider keys, and are write-only. Built-in model ids and provider slugs are reserved. The admin portal gets a Custom providers card. DeepSeek / Kimi / xAI / a corporate gateway become request bodies, not code. QA hardening: custom keys reach the model runtime for every provider, models.json materialization is cached per registry version, a corrupt custom key degrades only its provider, slashed custom ids win the opencode modelRef lookup, model/name input caps, double-delete 404s, and the picker refreshes when registrations change. Addresses the custom-endpoint asks in yc-software#110, yc-software#60, yc-software#116, yc-software#104.
Collaborator
|
Thanks for this, @Restry — and sorry for the slow response. We're closing this PR. Per We don't want your idea to get lost, so we've captured it in #840 to triage on our side. Please feel free to open an |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The in-process Pi harness can only ever reach the vendor endpoints.
pi-aibakesbaseUrlinto each provider's model record and nothing read an override, soresolveModelalways returnshttps://api.anthropic.com/https://api.openai.com/v1/https://openrouter.ai/api/v1.The child-process harnesses are already in a different position.
config.tsforwardsANTHROPIC_BASE_URLintoclaudeProcessEnvandOPENAI_BASE_URLintocodexProcessEnv, and each vendor SDK honours them. So on a deployment behind a gateway, Claude Code and Codex work and Pi doesn't — a difference in reach that the harness abstraction is otherwise good at hiding. I hit this running QM against a self-hosted Anthropic-compatible gateway: switching the org to Pi silently pointed every request back at the vendor.Config now parses
ANTHROPIC_BASE_URL,OPENAI_BASE_URLandOPENROUTER_BASE_URL— the same variables the SDKs read — andbuildApphands them to the model registry.Where the override lands
In
builtinModel, not inresolveModel's return.MODEL_REGISTRYclones some entries from a template (claude-opus-5is a clone ofclaude-opus-4-8), andcloneModelspreads the template — so overriding the template covers the clone, while overriding only the direct return path would leave clones pointing at the vendor while their template moved. Everything that issues a request resolves throughbuiltinModel, so the agent loop, judge, compaction, title, ack and detect calls all follow one setting.The value replaces the vendor base URL verbatim. Providers disagree about whether the version prefix belongs in it — anthropic omits
/v1and pi-harness appends/v1/messages, openai includes it — and matching each provider's own convention is what keeps these variables interchangeable with the SDKs that already read them.Injection rather than a direct env read
pi-models.tscan't importconfig.ts: config already imports pi-models, so it would be a cycle. And readingprocess.envfrom pi-models is what the repo's own lint rule exists to prevent. So config parses,buildAppinjects viasetProviderBaseUrls, and pi-models keeps the resolved value. Threading it throughresolveModel's ~15 call sites across 8 files was the alternative; it seemed a lot of surface for a value that is fixed for the process lifetime.Admin key validation
putModelProviderposted to a hardcoded vendor URL to check a key. Against a gateway that never sees the vendor's key, that fails a key which actually works, so the admin UI refuses to save it. It now resolves through the same setting.Verification
npm run typecheckandnpm run lintclean; full root suite 3715 tests, 0 failures. New tests cover the parse (including trailing-slash and whitespace normalization) and the resolution, with the clone case pinned explicitly — that one fails if the override moves toresolveModel.Beyond the suite, this is running on a live single-user deployment pointed at an Anthropic-compatible gateway. With
HARNESS=pithe agent completes turns and drives the sandbox through the gateway; both Pi and Claude Code are approved on the same org and both execute against the same gateway with no vendor key present anywhere. Before the change the same instance could only run Claude Code.No screenshot — this is config plumbing with no rendered surface. The only UI it touches is the admin model-provider form, which behaves as it did except that a gateway-issued key now validates.
Not addressed
secret-schema.tsstill requiresANTHROPIC_API_KEYwhen the base model is Anthropic. A gateway deployment typically carries its token inANTHROPIC_AUTH_TOKEN(whichclaudeChildEnvalready forwards), so it ends up setting both to the same value. Accepting either felt like a separate call about what the schema is asserting, so I left it.Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.