Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,9 @@
# -- Agent behavior -----------------------------------------------------------
# SKIP_PERMISSIONS=true # auto-approve all tool calls (dev only)
# MCP_CONFIG=/workspace/mcp.json # MCP server config file
# Exact, operator-controlled private hosts allowed during MCP OAuth discovery.
# Comma-separated; set on every console/node process in non-Compose deployments.
# TURNSTONE_MCP_OAUTH_TRUSTED_PRIVATE_HOSTS=gitlab.internal.example,auth.internal.example

# -- Channel gateway (Discord / Slack) ----------------------------------------
# TURNSTONE_DISCORD_TOKEN=
Expand Down
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,12 @@ the 1.7 line are not repeated here.

### Added

- **Trusted private hosts for MCP OAuth discovery.** Operators can allow exact,
controlled private-network hostnames or IP addresses through
`TURNSTONE_MCP_OAUTH_TRUSTED_PRIVATE_HOSTS` and the MCP Servers admin page.
Environment entries remain read-only and merge with database-managed entries;
the exception preserves HTTPS, same-origin, port, userinfo, and dangerous-address
protections.
- **Immutable memory-index snapshots (#902).** At the first real model call, a
workstream captures the complete memory metadata visible to its acting user
and project. That index stays cache-stable for the life of the workstream;
Expand Down
6 changes: 6 additions & 0 deletions compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,9 @@ services:
TURNSTONE_DB_BACKEND: *db-backend
TURNSTONE_DB_URL: *db-url
TURNSTONE_CONSOLE_URL: http://console:8090
# Exact private-network hosts trusted during MCP OAuth discovery. The
# console exposes these as read-only environment entries in the admin UI.
TURNSTONE_MCP_OAUTH_TRUSTED_PRIVATE_HOSTS: "${TURNSTONE_MCP_OAUTH_TRUSTED_PRIVATE_HOSTS:-}"
# Separate from TURNSTONE_CONSOLE_URL: this is the canonical responder
# base embedded in ACME directory/order URLs for cross-host enrollment.
TURNSTONE_ACME_EXTERNAL_URL: "${TURNSTONE_ACME_EXTERNAL_URL:-}"
Expand Down Expand Up @@ -308,6 +311,9 @@ services:
TURNSTONE_SEARXNG_URL: ${TURNSTONE_SEARXNG_URL:-http://searxng:8080}
MODEL: ${MODEL:-}
MCP_CONFIG: ${MCP_CONFIG:-}
# Must match the console value so OAuth discovery behaves consistently
# on every node that handles token refresh or connection work.
TURNSTONE_MCP_OAUTH_TRUSTED_PRIVATE_HOSTS: "${TURNSTONE_MCP_OAUTH_TRUSTED_PRIVATE_HOSTS:-}"
SKIP_PERMISSIONS: ${SKIP_PERMISSIONS:-}
TURNSTONE_NODE_ID: node-1
TURNSTONE_ADVERTISE_URL: http://node-1:8080
Expand Down
49 changes: 49 additions & 0 deletions docs/api-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -2505,11 +2505,60 @@ the `admin.settings` permission.
| DELETE | `/v1/api/admin/mcp-servers/{server_id}` | Delete an MCP server definition. |
| POST | `/v1/api/admin/mcp-servers/reload` | Tell all cluster nodes to re-read the `mcp_servers` DB table and reconcile (add new, remove stale, reconnect changed). |
| POST | `/v1/api/admin/mcp-servers/import` | Import servers from a pasted JSON config. Body: `{config: {mcpServers: {...}}}`. Skips existing names. |
| GET | `/v1/api/admin/mcp-servers/trusted-private-hosts` | Return the merged MCP OAuth private-host allow-list with source and read-only metadata. |
| PUT | `/v1/api/admin/mcp-servers/trusted-private-hosts` | Replace only the user-managed portion of the MCP OAuth private-host allow-list. Body: `{hosts: ["gitlab.internal.example"]}`. |

Permission: `admin.mcp`

Secrets (`env`, `headers` fields) are masked with `***` by default. Use `?reveal=true` on GET endpoints to see actual values.

#### Trusted private OAuth hosts

`GET /v1/api/admin/mcp-servers/trusted-private-hosts` returns both deployment
entries from `TURNSTONE_MCP_OAUTH_TRUSTED_PRIVATE_HOSTS` and entries managed by
users through the API/Web UI:

```json
{
"hosts": [
{
"host": "gitlab.internal.example",
"source": "environment",
"readonly": true
},
{
"host": "auth.internal.example",
"source": "manual",
"readonly": false
}
],
"environment_variable": "TURNSTONE_MCP_OAUTH_TRUSTED_PRIVATE_HOSTS",
"help": "Add only exact hostnames or IP addresses ..."
}
```

`PUT /v1/api/admin/mcp-servers/trusted-private-hosts` replaces the complete
**manual** list:

```json
{
"hosts": ["auth.internal.example", "192.168.5.120"]
}
```

The response has the same merged shape as GET. Environment entries are never
changed by PUT and take precedence over duplicate manual entries. Each value
must be an exact hostname or IP address; schemes, ports, paths, credentials,
and wildcards return `400`. The merged environment and manual list may contain
at most 100 unique hosts. A malformed deployment environment value makes GET
fail closed with `500`; correct the environment variable and restart the
affected process. Both methods require `admin.mcp`; missing configuration
storage returns `503`.

Trusting a host relaxes only private-address rejection during MCP OAuth
discovery. See [MCP OAuth: Private-network OAuth hosts](mcp-oauth.md#private-network-oauth-hosts)
for the remaining SSRF protections.

---

### MCP Registry
Expand Down
9 changes: 8 additions & 1 deletion docs/console.md
Original file line number Diff line number Diff line change
Expand Up @@ -477,7 +477,14 @@ in-flight requests keep their original definition snapshot; see
[Settings](settings.md#model-definition-reloads) for the full contract. The **Nodes** tab edits per-node
metadata, and the **TLS** tab manages CA and leaf certificates for the
internal mTLS fabric. The **Settings** tab edits ConfigStore values
live; edits apply without restart.
live; edits apply without restart. Under its **MCP** section, the
`oauth_trusted_private_hosts` row manages exact-host exceptions for
operator-controlled MCP/OAuth services on private networks. Entries supplied
through `TURNSTONE_MCP_OAUTH_TRUSTED_PRIVATE_HOSTS` are labeled **environment**
and cannot be changed in the UI; manually added entries are stored in the
database and can be removed. See
[MCP OAuth: Private-network OAuth hosts](mcp-oauth.md#private-network-oauth-hosts)
for the security boundary and deployment syntax.

**Users tab:**

Expand Down
1 change: 1 addition & 0 deletions docs/docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -281,6 +281,7 @@ interface, or anyone who can reach it can search through your instance.
| `TURNSTONE_WORKSPACE` | `/workspace` (image env) | Directory named as the user's workspace in the model's tool descriptions; informational only — see [Working directory](#working-directory) |
| `SKIP_PERMISSIONS` | — | Set to any value to auto-approve all tool calls (dev only) |
| `MCP_CONFIG` | — | Path to an MCP server config file |
| `TURNSTONE_MCP_OAUTH_TRUSTED_PRIVATE_HOSTS` | — | Comma- or newline-separated exact hostnames/IPs allowed to resolve to private addresses during MCP OAuth discovery. Set on every server and console container. Environment entries are read-only under **Settings → MCP** and merge with database-managed entries. See [MCP OAuth](mcp-oauth.md#private-network-oauth-hosts). |
| `TURNSTONE_IMAGE_TAG` | `latest` | ghcr.io image tag — production stack |

## Building
Expand Down
50 changes: 50 additions & 0 deletions docs/mcp-oauth.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,55 @@ Switching `auth_type` away from `oauth_user` / `oauth_obo` **deletes** that serv
| Scopes | No | Space-separated default scope set requested at the authorize endpoint. Per-tool step-up may union additional scopes from a server's `insufficient_scope` response. |
| Audience | No | RFC 8707 `resource=` parameter sent on every authorize and token request. Defaults to the MCP server URL when unset. Validate against the `aud` claim in returned JWT tokens. |

### Private-network OAuth hosts

OAuth discovery rejects endpoints that resolve to private addresses by default. This
prevents an untrusted MCP server or discovery document from turning Turnstone into an
SSRF proxy to internal services. If an MCP resource server or authorization server is
deliberately hosted on your private network, add its **exact hostname or IP address** to
the trusted private-host list.

For automated deployments, set the following environment variable on every
`turnstone-server` and `turnstone-console` process:

```bash
TURNSTONE_MCP_OAUTH_TRUSTED_PRIVATE_HOSTS=gitlab.internal.example,auth.internal.example
```

The value is a comma- or newline-separated list. Entries must contain only an exact
hostname or IP address. URLs, ports, paths, credentials, and wildcards are rejected:

```text
gitlab.internal.example # accepted
192.168.5.120 # accepted
https://gitlab.internal # rejected: URL
gitlab.internal:443 # rejected: port
*.internal.example # rejected: wildcard
```

Administrators with `admin.settings` and `admin.mcp` permissions can add
additional entries under **Admin → Settings → MCP** using the
`oauth_trusted_private_hosts` row. User-managed entries are stored in the database
and propagated to cluster nodes. Environment entries are merged first, shown with
an **environment** source label, and are read-only in the Web UI. If the same host
appears in both sources, the environment entry wins. The combined list is limited
to 100 hosts.

This is a narrow private-address exception, not a general SSRF bypass:

- matching is case-insensitive and exact; there is no suffix or wildcard matching;
- HTTPS is still required, apart from Turnstone's existing genuine-loopback development
exception;
- discovered endpoints must still satisfy issuer same-origin/trusted-host and port
checks;
- embedded credentials remain forbidden; and
- link-local, multicast, unspecified, reserved, and metadata-service addresses remain
refused even when their hostname is listed.

Only list hosts whose DNS and services are controlled by the deployment operator.
Removing a user-managed entry takes effect without a restart. Environment entries must
be changed in the process configuration and the affected processes restarted.

### Encryption key

```toml
Expand Down Expand Up @@ -168,6 +217,7 @@ Every transition that changes what a stored row *means* deletes the rows outrigh
| `mcp_consent_required` even after consenting | Token persistence failed, or refresh-token rejected by AS | Check audit log for `mcp_server.oauth.persist_failed` or `mcp_server.oauth.token_revoked`. Re-consent via settings modal. |
| `mcp_token_undecryptable_key_unknown` | Encryption key rotated without keeping the previous key in the keyring | Add the previous key back to `mcp_token_encryption_keys` until all rows have been re-encrypted, then drop. |
| `mcp_oauth_url_insecure` | MCP server URL is `http://` (not `https://`) on a non-loopback host | Use `https://`. Per-user bearers must not transit cleartext. |
| `PRM URL rejected: endpoint URL resolves to non-public address` | The MCP server or OAuth discovery endpoint deliberately resolves to a private address but is not trusted | Add its exact host under **Settings → MCP → oauth_trusted_private_hosts**, or set `TURNSTONE_MCP_OAUTH_TRUSTED_PRIVATE_HOSTS` on every server and console process. Do not add a wildcard or full URL. |
| Tools fail in scheduled / Discord / Slack runs | OAuth-MCP requires browser-based consent | Users must pre-consent via the web UI. Phase 9 dashboard badge surfaces deferred consents from these runs on next login. |
| Circuit breaker open repeatedly | Transport-level errors on the MCP server (DNS, TLS, 5xx) | Check the per-server error pill; auth errors do not trip the breaker. |
| **`oauth_obo`**: every tool call fails, log shows `obo_misconfigured` | Server row has no Audience, or `obo_grant_profile` is unset/unknown | Set the Audience on the server row; set `[oidc] obo_grant_profile` to `entra` or `rfc8693`. |
Expand Down
21 changes: 21 additions & 0 deletions docs/security.md
Original file line number Diff line number Diff line change
Expand Up @@ -297,6 +297,27 @@ API tokens are unaffected by this setting.

---

## MCP OAuth SSRF boundary

MCP protected-resource and authorization-server metadata is untrusted network
input. Turnstone therefore resolves and classifies every discovery URL before
fetching it, requires HTTPS outside the genuine-loopback development case,
checks discovered endpoint origin and port, rejects embedded credentials, and
refuses link-local/metadata, multicast, unspecified, and reserved addresses.

Private addresses are denied by default. Deployments with an intentionally
private MCP or authorization server can allow an **exact**, operator-controlled
hostname or IP through the MCP Servers admin page or
`TURNSTONE_MCP_OAUTH_TRUSTED_PRIVATE_HOSTS`. This opt-in relaxes only the
private-address classification for the listed host; it does not disable the
other checks above. Wildcards, URLs, and ports are not accepted. Environment
entries are read-only in the UI and merge with database-managed entries.

See [MCP OAuth: Private-network OAuth hosts](mcp-oauth.md#private-network-oauth-hosts)
for configuration, precedence, and examples.

---

## Token Detection Order

The auth middleware inspects the `Authorization: Bearer <token>` header
Expand Down
12 changes: 11 additions & 1 deletion docs/settings.md
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ initialization:
| `tools` | timeout, approval_timeout_seconds, truncation, agent_max_turns, skip_permissions, search, search_threshold, search_max_results |
| `server` | workstream_idle_timeout, max_workstreams |
| `cluster` | node_fan_out_limit, mcp_max_servers |
| `mcp` | config_path, registry_url |
| `mcp` | config_path, registry_url, oauth_trusted_private_hosts |
| `ratelimit` | enabled, requests_per_second, burst, trusted_proxies |
| `health` | backend_probe_interval, backend_probe_timeout, circuit_breaker_threshold, circuit_breaker_cooldown |
| `judge` | enabled, model, smart_approvals, confidence_threshold, max_context_ratio, timeout, parallel_evaluations, read_only_tools, output_guard, output_guard_budget_seconds, output_guard_llm, output_guard_model, output_guard_llm_timeout, redact_secrets, cancel_on_approval |
Expand All @@ -283,6 +283,16 @@ Settings are addressed by dotted key (e.g. `memory.relevance_k`). Each has a
declared type (`int`, `float`, `str`, `bool`), optional `min_value`/`max_value`
range, optional `choices` list, and an `is_secret` flag.

`mcp.oauth_trusted_private_hosts` stores only the user-managed portion of the
MCP OAuth private-host allow-list. Open **Admin → Settings**, expand **MCP**, and
use the `oauth_trusted_private_hosts` row. Its specialized editor displays the
merged source of each entry. Administrators without `admin.mcp` permission see
the standard Settings text editor for the database-managed value. Unlike ordinary
ConfigStore environment seeding,
`TURNSTONE_MCP_OAUTH_TRUSTED_PRIVATE_HOSTS` remains a live, read-only deployment
source and is never copied into the database. See
[MCP OAuth: Private-network OAuth hosts](mcp-oauth.md#private-network-oauth-hosts).

---

## Storage
Expand Down
41 changes: 40 additions & 1 deletion tests/test_app_js.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,46 @@
)
_CONSOLE_APP_JS = Path(__file__).resolve().parent.parent / "turnstone/console/static/app.js"
_CONSOLE_INDEX = Path(__file__).resolve().parent.parent / "turnstone/console/static/index.html"
_CONSOLE_ADMIN_JS = Path(__file__).resolve().parent.parent / "turnstone/console/static/admin.js"


def test_mcp_private_hosts_ui_lives_in_mcp_settings_and_uses_safe_dom_rendering() -> None:
html = _CONSOLE_INDEX.read_text(encoding="utf-8")
script = _CONSOLE_ADMIN_JS.read_text(encoding="utf-8")

assert 'id="mcp-private-host-form"' not in html
assert 'id="mcp-private-host-list"' not in html

settings_renderer_start = script.index("function _renderMcpTrustedPrivateHostsSetting(")
settings_renderer_end = script.index("function _renderSettingRow(", settings_renderer_start)
settings_renderer = script[settings_renderer_start:settings_renderer_end]
assert 'class="settings-row settings-row-mcp-private-hosts"' in settings_renderer
assert 'id="mcp-private-host-form"' in settings_renderer
assert 'id="mcp-private-host-list"' in settings_renderer
assert "TURNSTONE_MCP_OAUTH_TRUSTED_PRIVATE_HOSTS" in settings_renderer
assert "escapeHtml(item.help)" in settings_renderer

mcp_loader_start = script.index("function loadAdminMcp()")
mcp_loader_end = script.index("function loadMcpTrustedPrivateHosts()", mcp_loader_start)
assert "loadMcpTrustedPrivateHosts()" not in script[mcp_loader_start:mcp_loader_end]

settings_loader_start = script.index("function loadSettings()")
settings_loader_end = script.index("function _renderSettings(", settings_loader_start)
settings_loader = script[settings_loader_start:settings_loader_end]
assert '_consoleHasPermission("admin.mcp")' in settings_loader
assert "loadMcpTrustedPrivateHosts()" in settings_loader
assert 'items[j].key === "mcp.oauth_trusted_private_hosts" &&' in script, (
"settings-only admins must retain the standard Settings editor"
)
assert "/v1/api/admin/mcp-servers/trusted-private-hosts" in script
assert 'form.dataset.wired === "true"' in script
assert 'entry.source === "environment"' in script
assert "if (!entry.readonly)" in script
renderer_start = script.index("function _renderMcpTrustedPrivateHosts()")
renderer_end = script.index("function _setMcpPrivateHostStatus", renderer_start)
renderer = script[renderer_start:renderer_end]
assert ".textContent = entry.host" in renderer
assert "innerHTML" not in renderer


def _pane_method_offset(body: str, name: str) -> int:
Expand Down Expand Up @@ -752,7 +792,6 @@ def test_phase8_appendtooloutput_dispatches_mcp_error_before_renderer() -> None:
_COORD_JS = (
Path(__file__).resolve().parent.parent / "turnstone/console/static/coordinator/coordinator.js"
)
_CONSOLE_ADMIN_JS = Path(__file__).resolve().parent.parent / "turnstone/console/static/admin.js"
_CONSOLE_GOVERNANCE_JS = (
Path(__file__).resolve().parent.parent / "turnstone/console/static/governance.js"
)
Expand Down
Loading
Loading