Answer the OpenAI-shaped calls from wherever the deployment says (#13)

* Answer the OpenAI-shaped calls from wherever the deployment says

OpenBot already runs on your own machine, in your own PostgreSQL, with a
model key you supply. The one thing it could not do was decide where that
key is spent: `agent-bot` constructed its client with a key and no base
URL, `agent-langgraph` did the same through `ChatOpenAI`, and compose
handed neither of them a way to be told otherwise. A deployment that had
put a gateway or a proxy in front of its models, or that runs the model
on hardware it controls, had to fork two files to use it.

`OPENAI_BASE_URL` is that decision, and the API server already honoured
it: `resolveModel` in the pinned runtime reads it for every `openai/*`
model, so the package built-in agents have always been movable. This
gives the two shipped Bots the same variable and hands it to both
containers, so one line moves the whole deployment rather than the half
of it that happens to run outside Docker.

It is a base URL rather than another `BOT_PROVIDER` branch because the
API is the contract. `anthropic` and `google` are different APIs, not
different URLs for this one, and they are untouched. Model names travel
verbatim in `BOT_MODEL` and in the tenant package's `default_model`,
because an endpoint names its own catalogue.

Verified against a live OpenAI-compatible gateway: both Bots stream
AG-UI and emit tool calls, which is what a Bot needs to drive its
computer.

* Point the other two providers somewhere else as well

The OpenAI half of this landed the right idea and stopped one provider short.
A deployment that fronts Anthropic or Google the same way, or runs either on
its own hardware, had the same silent split this change set out to remove:
the API server would follow and agent-langgraph would not.

Both use the names the API server already reads, so one line still moves the
built-in agents and the Bot together.

agent-bot is untouched. It speaks the OpenAI API directly by construction, so
there is no Anthropic or Google call in it to redirect.

---------

Co-authored-by: David McKay <davidmckayv@users.noreply.github.com>

Author: knowhycodata

.env.example

@@ -53,6 +53,29 @@ COPILOTKIT_LICENSE_TOKEN=
 # framework Bot unless you point it at another provider below.
 OPENAI_API_KEY=
 
+# Where that key is spent. Unset, it is OpenAI. Set, it is any endpoint speaking the same
+# `/v1/chat/completions` API: a gateway in front of several providers, a proxy, or a model on
+# hardware you control. The API server reads it for the built-in agents and both shipped Bots read
+# it too, so one line moves the whole deployment rather than one Bot.
+#
+# Model names travel verbatim, so use whatever the endpoint publishes. An endpoint that namespaces
+# its catalogue wants both halves of the name, in `BOT_MODEL` and in the tenant package's
+# `default_model` alike. LLMTR, a gateway that fronts several providers and hosts models in Turkey
+# for deployments that need the data to stay there, addressed that way:
+#
+#   OPENAI_BASE_URL=https://llmtr.com/v1
+#   OPENAI_API_KEY=llmtr-...
+#   BOT_MODEL=openai/gpt-4o          # or llmtr/gemma-4, or anthropic/claude-sonnet-4.5
+#
+# Its catalogue is public and needs no key: https://llmtr.com/v1/models
+#
+# OPENAI_BASE_URL=
+
+# The same for the other two providers, under the names the API server already reads. They are
+# different APIs rather than different URLs for this one, so each has its own.
+# ANTHROPIC_BASE_URL=
+# GOOGLE_GENERATIVE_AI_BASE_URL=
+
 # Framework Bot provider: openai, anthropic or google. It reads that provider's own
 # key and refuses to start without it, so a deployment on Anthropic never needs an OpenAI key for it.
 # The proof-of-concept Bot is OpenAI only by construction: it speaks that API directly.

README.md

@@ -175,6 +175,8 @@ Settings worth knowing:
 | Variable                             | Use                                                                       |
 | ------------------------------------ | ------------------------------------------------------------------------- |
 | `OPENBOT_DEV_NO_AUTH`                | Admits every request as one administrator. How OpenBot runs today.        |
+| `OPENAI_BASE_URL`                    | Answers the OpenAI-shaped calls from somewhere else: a gateway, a proxy.  |
+| `ANTHROPIC_BASE_URL`, `GOOGLE_GENERATIVE_AI_BASE_URL` | The same, for those two APIs.            |
 | `COMPUTER_TOKEN`                     | Secret every Bot computer request must present. `start.sh` sets one.      |
 | `SUPERVISOR_TOKEN`                   | Secret the supervisor requires. `start.sh` sets one.                      |
 | `COMPUTER_SUPERVISOR_URL`            | Gives each Bot a computer of its own instead of one shared computer.      |

agent-bot/src/index.ts

@@ -26,7 +26,21 @@ const PORT = Number.parseInt(process.env.PORT ?? "4200", 10);
  */
 const MODEL = process.env.BOT_MODEL ?? "gpt-5.5";
 
-const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
+/**
+ * Where that model is answered from.
+ *
+ * Unset, this is OpenAI. Set, it is any endpoint speaking the same `/v1/chat/completions` API: a
+ * gateway in front of several providers, a proxy, or a model on hardware you control. Which is the
+ * point of writing against that API by hand rather than against one company's URL.
+ *
+ * `BOT_MODEL` is sent verbatim, because an endpoint names its own catalogue.
+ */
+const BASE_URL = process.env.OPENAI_BASE_URL?.trim() || undefined;
+
+const openai = new OpenAI({
+  apiKey: process.env.OPENAI_API_KEY,
+  baseURL: BASE_URL,
+});
 
 /** Translate the conversation AG-UI carries into the shape the model provider expects. */
 function toProviderMessages(input: RunAgentInput) {

agent-langgraph/src/index.ts

@@ -58,6 +58,24 @@ const PROVIDER = (process.env.BOT_PROVIDER ?? "openai").toLowerCase();
 const MODEL = process.env.BOT_MODEL ?? defaultModelFor(PROVIDER);
 /** OpenAI only. Its newer models require the Responses API, which the integration handles. */
 const USE_RESPONSES_API = process.env.BOT_RESPONSES_API === "true";
+/**
+ * OpenAI only, and the same variable the API server reads for its built-in agents.
+ *
+ * Unset, `openai` means OpenAI. Set, it means any endpoint speaking that API: a gateway in front of
+ * several providers, a proxy, or a model on hardware you control. The integration owns the HTTP, so
+ * this is a base URL rather than another provider branch, and `BOT_MODEL` is sent verbatim because
+ * an endpoint names its own catalogue.
+ */
+const OPENAI_BASE_URL = process.env.OPENAI_BASE_URL?.trim() || undefined;
+/**
+ * The same idea for the other two providers, under the names the API server already reads.
+ *
+ * Sharing the variable names is the point: one line moves the built-in agents and this Bot
+ * together, and a deployment cannot end up with half of itself pointed somewhere else.
+ */
+const ANTHROPIC_BASE_URL = process.env.ANTHROPIC_BASE_URL?.trim() || undefined;
+const GOOGLE_BASE_URL =
+  process.env.GOOGLE_GENERATIVE_AI_BASE_URL?.trim() || undefined;
 
 function defaultModelFor(provider: string): string {
   if (provider === "anthropic") return "claude-sonnet-4-5";
@@ -177,19 +195,22 @@ function buildModel() {
       model: MODEL,
       apiKey: API_KEY,
       streaming: true,
+      ...(ANTHROPIC_BASE_URL ? { anthropicApiUrl: ANTHROPIC_BASE_URL } : {}),
     });
   }
   if (PROVIDER === "google") {
     return new ChatGoogleGenerativeAI({
       model: MODEL,
       apiKey: API_KEY,
       streaming: true,
+      ...(GOOGLE_BASE_URL ? { baseUrl: GOOGLE_BASE_URL } : {}),
     });
   }
   return new ChatOpenAI({
     model: MODEL,
     apiKey: API_KEY,
     streaming: true,
+    ...(OPENAI_BASE_URL ? { configuration: { baseURL: OPENAI_BASE_URL } } : {}),
     ...(USE_RESPONSES_API ? { useResponsesApi: true } : {}),
   });
 }

docker-compose.yml

@@ -163,6 +163,9 @@ services:
       - "${BOT_PORT:-4200}:4200"
     environment:
       OPENAI_API_KEY: ${OPENAI_API_KEY}
+      # Unset means OpenAI. Set, it is any endpoint speaking the same API, and BOT_MODEL is sent
+      # to it verbatim.
+      OPENAI_BASE_URL: ${OPENAI_BASE_URL:-}
       BOT_MODEL: ${BOT_MODEL:-gpt-5.5}
     healthcheck:
       test: ["CMD-SHELL", "bun -e \"await fetch('http://localhost:4200/health')\""]
@@ -182,8 +185,11 @@ services:
       # BOT_RESPONSES_API instead of changing the streaming loop here.
       BOT_PROVIDER: ${BOT_PROVIDER:-openai}
       OPENAI_API_KEY: ${OPENAI_API_KEY:-}
+      OPENAI_BASE_URL: ${OPENAI_BASE_URL:-}
       ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:-}
+      ANTHROPIC_BASE_URL: ${ANTHROPIC_BASE_URL:-}
       GOOGLE_API_KEY: ${GOOGLE_API_KEY:-}
+      GOOGLE_GENERATIVE_AI_BASE_URL: ${GOOGLE_GENERATIVE_AI_BASE_URL:-}
       BOT_MODEL: ${BOT_MODEL:-}
       BOT_RESPONSES_API: ${BOT_RESPONSES_API:-false}
     healthcheck:

docs/configuration.md

@@ -37,12 +37,50 @@ All four Intelligence values are required together. Missing any of them stops se
 | `TENANT_PACKAGE_DIR` | `../examples/fintech`              | Tenant package directory, resolved from `server/`.                  |
 | `DEPLOYMENT_ID`      | the tenant package's id            | Names this deployment inside a shared Intelligence project.          |
 | `OPENAI_API_KEY`     | unset                              | Default model key for built-in agents and both shipped Bots.        |
+| `OPENAI_BASE_URL`    | unset                              | OpenAI-compatible endpoint that key is spent against. See below.    |
 | `BOT_PROVIDER`       | `openai`                           | Provider for `agent-langgraph`: `openai`, `anthropic`, or `google`. |
 | `ANTHROPIC_API_KEY`  | unset                              | Anthropic key when `BOT_PROVIDER=anthropic`.                        |
+| `ANTHROPIC_BASE_URL` | unset                              | Anthropic-compatible endpoint that key is spent against.            |
 | `GOOGLE_API_KEY`     | unset                              | Google key when `BOT_PROVIDER=google`.                              |
+| `GOOGLE_GENERATIVE_AI_BASE_URL` | unset                   | Google-compatible endpoint that key is spent against.               |
 | `BOT_MODEL`          | provider default from Bot code/env | Model used by the shipped Bots.                                     |
 | `BOT_RESPONSES_API`  | `false`                            | Makes `agent-langgraph` use the OpenAI Responses API.               |
 
+## OpenAI-compatible endpoints
+
+`OPENAI_BASE_URL` decides where an OpenAI-shaped request is answered. Unset, that is OpenAI. Set, it is any endpoint speaking the same API: a gateway in front of several providers, a proxy, or a model on hardware you control.
+
+It moves the whole deployment rather than one Bot. The API server reads it for package built-in agents, `agent-bot` reads it for the client it constructs, and `agent-langgraph` reads it for `BOT_PROVIDER=openai`.
+
+The other two providers work the same way under their own names, because they are different APIs rather than different URLs for this one: `ANTHROPIC_BASE_URL` and `GOOGLE_GENERATIVE_AI_BASE_URL`. All three are the names the API server already reads, so one line moves the built-in agents and the Bots together and a deployment cannot end up with half of itself pointed somewhere else.
+
+Model names travel verbatim, so use whatever the endpoint publishes. An endpoint that namespaces its catalogue wants both halves of the name, in `BOT_MODEL` and in the tenant package's `default_model` alike.
+
+[LLMTR](https://llmtr.com) is one such gateway. It fronts OpenAI, Anthropic, Google and others behind one key, and hosts models in Turkey for deployments that need the data to stay there, which is the same reason a deployment runs OpenBot on its own infrastructure. Addressed the usual way:
+
+```sh
+OPENAI_BASE_URL=https://llmtr.com/v1
+OPENAI_API_KEY=llmtr-...
+BOT_MODEL=openai/gpt-4o          # or llmtr/gemma-4, or anthropic/claude-sonnet-4.5
+```
+
+and in the tenant package, where the name is namespaced the same way:
+
+```yaml
+model:
+  provider: openai
+  credential_secret_ref: openai-api-key
+  default_model: openai/gpt-4o
+```
+
+Its catalogue is public and needs no key, so the names above can be checked before anything is configured:
+
+```sh
+curl -s https://llmtr.com/v1/models
+```
+
+Two things are worth knowing before pointing a deployment at any gateway. Not every catalogue entry accepts tools, and a Bot without tool calling cannot drive its computer; the model list says which do. And `BOT_RESPONSES_API=true` needs an endpoint that implements the Responses API, not only chat completions.
+
 ## Authentication
 
 | Variable                     | Meaning                                                                                |
@@ -223,7 +261,7 @@ model:
   default_model: gpt-4.1
 ```
 
-`provider` must be `openai`. `credential_secret_ref` is a reference to a stored credential, not a credential value.
+`provider` must be `openai`. `credential_secret_ref` is a reference to a stored credential, not a credential value. `default_model` is passed through as written, so an OpenAI-compatible endpoint reached through `OPENAI_BASE_URL` takes the name that endpoint publishes.
 
 ### `knowledge.yaml`
 

tests/compose.test.ts

@@ -37,6 +37,31 @@ test("publishes every service on a settable port with the documented default", (
   }
 });
 
+/**
+ * Both Bots are reachable at whatever `OPENAI_BASE_URL` names.
+ *
+ * The API server reads that variable from `.env` directly, so it moves with the deployment. The
+ * Bots run in containers and see only what compose hands them, and a deployment that moved its
+ * models to a gateway and found half of itself still calling OpenAI would have no way to tell.
+ */
+test("gives both shipped Bots the OpenAI-compatible endpoint", () => {
+  const compose = readFileSync(
+    join(import.meta.dir, "..", "docker-compose.yml"),
+    "utf8",
+  );
+
+  // Both Bots speak OpenAI; only the framework Bot can be pointed at the other two.
+  expect(
+    compose.match(/OPENAI_BASE_URL: \$\{OPENAI_BASE_URL:-?\}/g),
+  ).toHaveLength(2);
+  for (const variable of [
+    "ANTHROPIC_BASE_URL",
+    "GOOGLE_GENERATIVE_AI_BASE_URL",
+  ]) {
+    expect(compose).toContain(`${variable}: \${${variable}:-}`);
+  }
+});
+
 test("enables pgvector before creating vector columns", () => {
   const migration = readFileSync(
     join(import.meta.dir, "..", "server", "drizzle", "0000_schema.sql"),