Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,057 changes: 67 additions & 990 deletions Cargo.lock

Large diffs are not rendered by default.

4 changes: 0 additions & 4 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -85,10 +85,6 @@ futures-util = "0.3"

# OAuth 2.1 + PKCE authentication
oauth2 = { version = "5.0", features = ["reqwest", "rustls-tls"] }
# Credential storage: single cross-platform SQLite store (works headless, no dbus).
# `keyring-core` provides `Entry`; `db-keystore` is the SQLite-backed store.
keyring-core = "1.0"
db-keystore = "0.4"
sha2 = "0.11.0"
hex = "0.4"
serde_urlencoded = "0.7"
Expand Down
1 change: 0 additions & 1 deletion doc/dev/02-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -313,7 +313,6 @@ Key external crates:
- `clap` -- CLI argument parsing
- `serde` / `toml` -- serialization
- `oauth2` -- OAuth 2.1 + PKCE
- `keyring` -- secure token storage
- `hyper` -- HTTP server
- `reedline` -- interactive readline
- `syntect` -- syntax highlighting
2 changes: 1 addition & 1 deletion doc/dev/03-mcp-server-development.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Three built-in servers are shipped as `[[mcp.servers]]` entries in `config-templ
External:
| Server | Type | Purpose |
|--------|------|---------|
| `filesystem` (octofs) | stdio | `view`, `text_editor`, `batch_edit`, `extract_lines`, `shell`, `ast_grep`, `list_files`, `workdir` |
| `filesystem` (octofs) | stdio | `view`, `text_editor`, `batch_edit`, `extract_lines`, `shell`, `workdir` |

> `filesystem` is **not** declared in `default.toml`'s `[[mcp.servers]]`. It is supplied by a tap and referenced by name in roles' `server_refs` / `allowed_tools`. Only `core`, `runtime`, and `agent` are shipped as built-in server entries.

Expand Down
2 changes: 1 addition & 1 deletion doc/integration/03-daemon-and-hooks.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ Each running session exposes one per-name IPC endpoint that `octomind send` conn

| Platform | Endpoint | Extra |
|----------|----------|-------|
| Unix (macOS/Linux) | Unix socket at `~/.local/share/octomind/run/<name>.sock` | PID written to `<name>.pid` |
| Unix (macOS/Linux) | Unix socket at `$XDG_RUNTIME_DIR/octomind/<name>.sock` (fallback: `<system tmp>/octomind-<uid>/<name>.sock`) | PID written to `<name>.pid` |
| Windows | Named pipe `\\.\pipe\octomind-<name>` | — |

These files are created on session start and auto-cleaned when the session exits (a stale socket from a crash is removed on next bind). The injected message is trimmed; an empty message is rejected with `error: empty message`.
Expand Down
2 changes: 1 addition & 1 deletion doc/reference/01-cli-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,7 @@ Inject a message into a running named session.
Works against any running session that has started its inject listener (typically a session launched with
`--daemon`, but not exclusively). The message reaches the session over a per-OS transport:

- **Unix:** a Unix domain socket at `<run_dir>/<name>.sock` (run dir is `~/.local/share/octomind/run/`).
- **Unix:** a Unix domain socket at `<run_dir>/<name>.sock` (run dir is `$XDG_RUNTIME_DIR/octomind/`, or `<system tmp>/octomind-<uid>/` when that variable is unset).
- **Windows:** a named pipe `\\.\pipe\octomind-<name>`.

The session replies `ok` on successful delivery; any other reply is treated as an error and reported.
Expand Down
2 changes: 1 addition & 1 deletion doc/reference/03-config-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ MCP server definitions. Three types supported: `builtin`, `http`, `stdio`.
| `runtime` | `mcp`, `agent`, `skill`, `schedule`, `capability` | Harness reconfiguration and scheduling |
| `agent` | `agent_<name>` per `[[agents]]` entry | ACP sub-agent dispatch |

> **`filesystem` is not declared here.** Default roles reference a `filesystem` server in their `server_refs`, but it is **not** a builtin and is **not** defined in this config file's `[[mcp.servers]]`. It is an external `stdio` server backed by `octofs`, provided by the built-in tap. Its tools are `view`, `text_editor`, `batch_edit`, `extract_lines`, `shell`, `ast_grep`, `list_files`, and `workdir`. See [MCP Tools](../usage/07-mcp-tools.md) for the full surface.
> **`filesystem` is not declared here.** Default roles reference a `filesystem` server in their `server_refs`, but it is **not** a builtin and is **not** defined in this config file's `[[mcp.servers]]`. It is an external `stdio` server backed by `octofs`, provided by the built-in tap. Its tools are `view`, `text_editor`, `batch_edit`, `extract_lines`, `shell`, and `workdir`. See [MCP Tools](../usage/07-mcp-tools.md) for the full surface.

#### Common Fields

Expand Down
4 changes: 2 additions & 2 deletions doc/troubleshooting/01-common-issues.md
Original file line number Diff line number Diff line change
Expand Up @@ -268,8 +268,8 @@ max_session_spending_threshold = 0.0 # Disable the check (<= 0.0 disables)
### Cannot Send to a Running Session

`octomind send -n NAME "message"` talks to a session over a Unix domain socket
(`~/.local/share/octomind/run/<name>.sock`; a named pipe `\\.\pipe\octomind-<name>`
on Windows). If you get an error like:
(`$XDG_RUNTIME_DIR/octomind/<name>.sock`, or `<system tmp>/octomind-<uid>/<name>.sock`
when that variable is unset; a named pipe `\\.\pipe\octomind-<name>` on Windows). If you get an error like:

```
no running session named 'NAME' (socket not found ...)
Expand Down
2 changes: 1 addition & 1 deletion doc/usage/03-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Full directory structure:
sessions/ # Saved sessions
logs/ # Debug and error logs
cache/ # Cached data
run/ # Per-session Unix sockets and PID files (used by `send` / --daemon)
keystore.json # OAuth tokens for remote MCP servers (mode 0600)
learning/ # Cross-session adaptive learning (lessons), scoped by project/role
agents/ # Cached tap agent manifests (<category>/<variant>.toml)
```
Expand Down
2 changes: 1 addition & 1 deletion doc/usage/06-roles.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ allowed_tools = [
- `runtime` -- low-level harness control: `mcp` (register servers), `agent` (register dynamic agents), `skill` (load skills), `schedule`, `capability`. Most roles don't need this.
- `agent` -- dispatches to `[[agents]]`-defined ACP sub-agents (`agent_<name>` per entry).

> **`filesystem` is not a built-in declared server.** The filesystem tools (`view`, `text_editor`, `batch_edit`, `extract_lines`, `shell`, `ast_grep`, `list_files`, `workdir`) come from the **octofs** companion server supplied by the tap/capability layer, not from a `[[mcp.servers]]` entry. If you copy the examples below into a bare standalone config that has no tap/capability providing `filesystem`, the reference is silently dropped and no filesystem tools appear. To see exactly which tools a server exposes, run `/mcp list` (or `/mcp full`) in a session.
> **`filesystem` is not a built-in declared server.** The filesystem tools (`view`, `text_editor`, `batch_edit`, `extract_lines`, `shell`, `workdir`) come from the **octofs** companion server supplied by the tap/capability layer, not from a `[[mcp.servers]]` entry. If you copy the examples below into a bare standalone config that has no tap/capability providing `filesystem`, the reference is silently dropped and no filesystem tools appear. To see exactly which tools a server exposes, run `/mcp list` (or `/mcp full`) in a session.

**Pattern syntax:**
- `"server:*"` -- all tools from a server (e.g. `agent:*` grants every `agent_<name>` execution tool on the `agent` server)
Expand Down
32 changes: 1 addition & 31 deletions doc/usage/07-mcp-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Octomind ships **three builtin MCP servers** declared in the default config (`co
| `agent` | builtin | Delegates tasks to configured ACP sub-agents (each `[[agents]]` entry exposes an `agent_<name>` tool) |
| `local` | builtin | Project-local shebang-script tools auto-discovered from `<workdir>/.agents/tools/`. See [Local Tools](17-local-tools.md). |

The filesystem tools (`view`, `text_editor`, `shell`, `ast_grep`, …) are **not** a builtin server. They are served by a separate `octofs` MCP server (a stdio subprocess: command `octofs`, args `["mcp"]`) that is **not declared in the default config**. It is delivered through the built-in default tap [`muvon/tap`](../integration/04-tap-system.md)'s capabilities `filesystem-read` and `filesystem-write`, and roles reach it via `server_refs`/capabilities under the `filesystem` capability name — never a hardcoded `[[mcp.servers]]` block named `filesystem`. See [Filesystem Server Tools (octofs)](#filesystem-server-tools-octofs) below for the prerequisites.
The filesystem tools (`view`, `text_editor`, `shell`, …) are **not** a builtin server. They are served by a separate `octofs` MCP server (a stdio subprocess: command `octofs`, args `["mcp"]`) that is **not declared in the default config**. It is delivered through the built-in default tap [`muvon/tap`](../integration/04-tap-system.md)'s capabilities `filesystem-read` and `filesystem-write`, and roles reach it via `server_refs`/capabilities under the `filesystem` capability name — never a hardcoded `[[mcp.servers]]` block named `filesystem`. See [Filesystem Server Tools (octofs)](#filesystem-server-tools-octofs) below for the prerequisites.

`core` and `runtime` are the two split halves of what used to be a single `core` server. The split separates "what the agent uses to do work" (`core`) from "what reconfigures the harness mid-session" (`runtime`).

Expand Down Expand Up @@ -239,17 +239,6 @@ Read files, view directories, and search file content.
- `pattern` (string): Search pattern within file/directory
- `content` (string): Content search query

### `list_files` -- List Directory Entries

List files and directories at a path. Complements `view` (which reads content) when you only need the file listing.

**Parameters:**
- `path` (string): Directory to list (defaults to the current working directory)

```json
{"path": "src/"}
```

### `text_editor` -- File Editing

Comprehensive file manipulation with multiple commands.
Expand Down Expand Up @@ -342,25 +331,6 @@ Get or set the working directory for file and shell operations.

Thread-local: changes only affect the current session.

### `ast_grep` -- AST-Based Code Search

Search and refactor code using AST patterns with ast-grep (sg).

**Parameters:**
- `pattern` (string, required): AST pattern using ast-grep syntax
- `language` (string): `"rust"`, `"javascript"`, `"typescript"`, `"python"`, `"go"`, `"java"`, `"c"`, `"cpp"`, `"php"`
- `paths` (array): File paths or glob patterns (default: current directory)
- `rewrite` (string): Rewrite pattern for refactoring
- `json_output` (boolean, default: false): JSON format output
- `context` (integer, default: 0): Context lines around matches
- `update_all` (boolean, default: false): Apply rewrites to all matches

```json
{"pattern": "console.log($$$)", "language": "javascript"}
{"pattern": "oldFunc($ARGS)", "rewrite": "newFunc($ARGS)", "language": "javascript"}
{"pattern": "class $NAME", "language": "php", "paths": ["src/**/*.php"], "context": 2}
```

## Agent Server Tools

Each agent configured in `[[agents]]` becomes a separate tool: `agent_<name>`.
Expand Down
29 changes: 18 additions & 11 deletions doc/usage/11-structured-output.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Octomind can emit its session activity as machine-readable JSON instead of human-formatted terminal text. This is what you use for automation, CI/CD pipelines, and any program that needs to parse what the agent did.

> **Heads up:** Octomind does **not** currently let you enforce a JSON Schema on the assistant's *answer*. There is no `--schema` CLI flag, and no WebSocket or ACP protocol message accepts a schema. If you came here looking for "make the model reply with exactly this JSON shape," that is not a user-accessible feature today — see [Schema Enforcement](#schema-enforcement-not-user-accessible) below for the full picture. What *is* available is a structured **event stream** (`--format jsonl` and the WebSocket/ACP servers), described next.
> **Heads up:** "structured output" covers two different features here. To make the model's *answer* conform to a JSON Schema, pass `--schema <file>` to `octomind run` — see [Schema Enforcement](#schema-enforcement---schema) below. Independent of that, the session's *activity* is available as a structured **event stream** (`--format jsonl` and the WebSocket/ACP servers), described next. Note the WebSocket and ACP protocols do not accept a schema — `--schema` exists only on `octomind run`.

## The Automation Surface: `--format jsonl`

Expand Down Expand Up @@ -50,8 +50,8 @@ Example of a few lines from a `jsonl` run (one object per physical line):

```json
{"type":"status","message":"Session created: my-session","session_id":"my-session"}
{"type":"tool_use","tool":"list_files","tool_id":"call_abc","server":"filesystem","params":{"directory":"src"},"session_id":"my-session"}
{"type":"tool_result","tool":"list_files","tool_id":"call_abc","server":"filesystem","content":"src/main.rs\nsrc/lib.rs","success":true,"session_id":"my-session"}
{"type":"tool_use","tool":"view","tool_id":"call_abc","server":"filesystem","params":{"path":"src/"},"session_id":"my-session"}
{"type":"tool_result","tool":"view","tool_id":"call_abc","server":"filesystem","content":"src/main.rs\nsrc/lib.rs","success":true,"session_id":"my-session"}
{"type":"assistant","content":"Recent changes refactored the session loop...","session_id":"my-session"}
{"type":"cost","session_tokens":1234,"session_cost":0.0025,"input_tokens":1000,"output_tokens":200,"cache_read_tokens":30,"cache_write_tokens":4,"reasoning_tokens":0,"session_id":"my-session"}
```
Expand All @@ -74,7 +74,7 @@ Both stream the structured events listed above; neither accepts a schema on sess

## Provider Compatibility (Structured Output Capability)

Whether a provider *can* be asked for native structured output is exposed by each provider's `supports_structured_output(model)`. This capability is currently exercised internally (see [Schema Enforcement](#schema-enforcement-not-user-accessible)), not by a user-facing schema flag. For reference, against the active `octolib 0.21.6`:
Whether a provider *can* be asked for native structured output is exposed by each provider's `supports_structured_output(model)`. This capability gates the `--schema` flag on `octomind run` (see [Schema Enforcement](#schema-enforcement---schema)) and the internal compression decision call. For reference, against the active `octolib 0.21.6`:

| Provider | `supports_structured_output` |
|----------|------------------------------|
Expand All @@ -86,23 +86,30 @@ Whether a provider *can* be asked for native structured output is exposed by eac
| OpenRouter | Per model's reference capabilities, else Yes |
| Anthropic | Trait default — per model's reference capabilities, else No |

When code does request a schema from a provider that returns `false` for the given model, Octomind fails fast:
When a schema is requested from a provider that returns `false` for the given model, Octomind fails fast:

```
Provider 'anthropic' does not support structured output for model '<model-without-reference-capabilities>'. Remove the schema parameter or use a compatible provider.
```

## Schema Enforcement (Not User-Accessible)
## Schema Enforcement (`--schema`)

There **is** a schema mechanism in the codebase, but it is not wired to any user entry point:
Pass a JSON Schema **object** file to `octomind run` to constrain the assistant's replies:

- `ChatSession` has a `schema` field, but it is set to `None` at every construction site and the session-level `with_schema()` builder has no callers. In practice, per-session assistant output is **always** unconstrained — there is no CLI flag, WebSocket message, or ACP method that can populate it.
- The `ProviderResponse.structured_output` field exists in the provider response type, but because no session sets a schema it is always `None` for normal sessions, and there is no display logic that prefers it over `content`.
```bash
echo "List the top 3 TODOs" | octomind run developer:general --format jsonl --schema todos.schema.json
```

- The schema applies to **every assistant reply** for the session's lifetime — across multi-turn sessions, resumes, and daemon mode. Tool calls still flow normally underneath; only the final text is constrained.
- The resolved model must support structured output (see the provider table above), or the run fails fast with the error shown there.
- Like `--model`, the schema is a **runtime override** — it is not persisted with the session, so pass it again when resuming.
- The file must contain a JSON Schema object; it is loaded and validated before session init. A ready-to-use example ships at [`config-templates/todos.schema.json`](../../config-templates/todos.schema.json).
- `--schema` exists only on `octomind run` — the WebSocket and ACP session-init messages do not accept a schema.

The one place a schema is genuinely built and used today is **internal**: the conversation-compression decision call. When compression runs, it checks the *decision model's* provider via `supports_structured_output()`; if that returns true, it sends a generated compression schema (`build_compression_schema`) in strict mode to get a reliable decision/summary, otherwise it falls back to an XML-style prompt. This is invisible to your session output and uses the separate compression decision model, not your main model. (Default decision model: `openai:gpt-5-mini` — see [Context Compression](08-compression.md).)
The same mechanism is also used **internally** by the conversation-compression decision call: when compression runs, it checks the *decision model's* provider via `supports_structured_output()`; if true, it sends a generated compression schema in strict mode to get a reliable decision/summary, otherwise it falls back to an XML-style prompt. This is invisible to your session output and uses the separate compression decision model, not your main model. (Default decision model: `openai:gpt-5-mini` — see [Context Compression](08-compression.md).)

## Summary

- For machine-readable output, use `--format jsonl` on `octomind run` (or the WebSocket/ACP servers for live streaming).
- The JSONL/WebSocket/ACP streams emit typed `ServerMessage` events (`assistant`, `tool_use`, `tool_result`, `cost`, `status`, `error`, `mcp_notification`, `skill`, `injected`, `thinking`).
- There is no user-facing way to enforce a JSON Schema on the assistant's answer. The internal schema mechanism is used only by the compression decision call.
- To enforce a JSON Schema on the assistant's answer, pass `--schema <file>` to `octomind run` (structured-output-capable models only). The compression decision call uses the same mechanism internally.
2 changes: 1 addition & 1 deletion doc/use-cases/02-event-driven-agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ echo "Summarize what happened in the last hour" | octomind send --name ops-agent
```

`octomind send` connects to the running session over a per-session Unix socket
(`<data>/run/<name>.sock`) or, on Windows, a named pipe (`\\.\pipe\octomind-<name>`). It
(`<run_dir>/<name>.sock`) or, on Windows, a named pipe (`\\.\pipe\octomind-<name>`). It
only works **while a daemon/session with that name is live** -- if no such session is
running, it fails with `no running session named '<name>'`. The message must be non-empty,
and `send` reads back `ok` on success or an error string. (You can pass the message as an
Expand Down
2 changes: 1 addition & 1 deletion doc/use-cases/03-custom-development-workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ workflow · dev
╰ ✓ refine 1.4s · $0.0009 · 420 tok · ⚒0

╭ research
│ ▸ ast_grep · octofs
│ ▸ view · octofs
╰ ✓ research 6.2s · $0.0071 · 2980 tok · ⚒5

╭ execute
Expand Down
Loading
Loading