Skip to content

v2.5: Add OmniRoute self-hosted AI gateway support - #18

Open
agunggnn wants to merge 12 commits into
darkzOGx:masterfrom
agunggnn:master
Open

agunggnn wants to merge 12 commits into
darkzOGx:masterfrom
agunggnn:master

Conversation

@agunggnn

Copy link
Copy Markdown

Summary

  • Adds OmniRoute as a text-generation provider option alongside OpenAI, Gemini, OpenRouter, Kimi, MiMo, and GLM
  • OmniRoute is an OpenAI-compatible self-hosted gateway that owns upstream provider credentials, model routing, quotas, and failover — the app only needs a single client key and combo/model (e.g. auto/smart)
  • Wired through the credential setup wizard, guided walkthrough, .env.example, README, and a new docs/OMNIROUTE.md walkthrough doc

Test plan

  • node test.js — 16/16 passed
  • npm run lint — 0 errors (only pre-existing warnings)

🤖 Generated with Claude Code

Adds OmniRoute as a text-generation provider option alongside OpenAI,
Gemini, OpenRouter, Kimi, MiMo, and GLM — an OpenAI-compatible gateway
that owns upstream provider credentials, routing, and failover so this
app only needs a single client key and combo/model (e.g. auto/smart).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@00edd1e

00edd1e commented Oct 9, 2026

Copy link
Copy Markdown

Thanks @agunggnn for this. It covers a need we have too: sending text generation through a self-hosted gateway.

@darkzOGx, this is a question about scope, not a request to merge this PR as it stands. Would you take one generic "OpenAI-compatible endpoint" text provider, configured by base URL, key and model, instead of a preset per gateway? This PR and #16 (Atlas Cloud) would both fit it, and so would LiteLLM, vLLM, Ollama and similar servers.

What we hit on a self-hosted install pinned at e26792a:

  • Every preset in utils/ai-text-service.js hardcodes its baseURL, so reaching a local gateway means patching that file.
  • OPENAI_BASE_URL doesn't work around it. The text client passes baseURL explicitly, so it ignores the variable. The OpenAI clients in utils/ai-video-generator.js:25 and utils/credential-manager.js:591 are built without a baseURL, so they would follow it, and image generation and the key check would go to the gateway.

A shape that would cover it:

  • an openai-compatible entry in PROVIDERS, selected through aiProvider like the others
  • TEXT_OPENAI_BASE_URL, TEXT_OPENAI_API_KEY and TEXT_OPENAI_MODEL, all required, with no product defaults and no fallback to api.openai.com when one is missing
  • used only in _initOpenAICompatible, so image generation and the OpenAI key validation stay where they are
  • readiness checked with the configured model and enough token budget for a reasoning model to answer (Readiness text_provider fails on thinking Gemini models: the 16-token probe budget is spent on thoughts #63 has the details on why a 16-token probe can come back empty)

The existing max_completion_tokens -> max_tokens retry already handles gateways that accept only the older parameter.

If you'd rather build it yourself, that works for us. Either way, we're glad to test a branch against a local gateway and report back.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants