Agent Runtime

Chinese version: README.zh-CN.md

Agent Runtime is a self-hosted agent platform written in Rust and centered on the clawd daemon. It combines multi-channel chat access, task execution, tool and skill routing, memory, scheduling, a browser UI, and user_key-based identity in one deployable stack.

Overview

Agent Runtime is built for daily use and administration from messaging apps or a browser instead of a terminal-first workflow.

Current repository highlights:

multi-channel entry points: Telegram, WeChat, Feishu, Lark, WhatsApp Cloud, WhatsApp Web, and the browser UI through webd, with optional nginx/TLS
task runtime and HTTP API in clawd
shared skill dispatch with in-process builtins, external adapters, and runner subprocesses through skill-runner
built-in, external, and runner-based skills for system, files, web, image, audio, video, music, NNI, crypto, KB, and automation tasks
local browser UI in UI/, including Dashboard, Agent, Models, Tasks, Communication Setup, Account Binding, Tools/Skills, Skill Store, Memory, Logs, and Learning / Maintenance pages
Raspberry Pi / small-screen desktop app in pi_app/
shared Linux/macOS runtime contracts, with fail-closed Bubblewrap and Seatbelt process isolation selected through a machine-configured backend

Five ideas to know first

Agent: reads the current request and decides whether to answer, ask a question, or use a capability.
Task: the persisted unit of work. A task_id lets the UI return to its status and result later.
Capability: a stable description of what can be done, such as reading a file or searching the web.
Tool or skill: the executable implementation selected for a capability after permission and argument checks.
Checkpoint or artifact: saved progress or output that allows long work to continue without starting over.

From one request to a result

For example, “summarize this document” becomes a task. The agent inspects the attached file, selects a document capability, runs the verified tool or skill, checks the observation, and then writes the answer. Runtime policy controls identity, permissions, budgets, and side effects; it does not guess intent from hardcoded language phrases.

Complete a first task

Configure and save one text model on the Models page.
Open Agent, create a task, and send a small request such as “summarize the main points in three bullets.”
Keep the same task open for follow-up instructions. Check task details only when progress, permission, or recovery needs attention.

Messaging channels are optional. Configure them only when the browser workflow is already working and you need access from another app.

Product identity and safe renaming

configs/product_identity.toml is the only product-identity source. Its display name, release artifact ID, release repository, splash image, and terminal banner are projected into the Rust runtime, shell/release scripts, and UI build. Do not add brand defaults to business code or ordinary skills.

Changing this file changes presentation and release packaging only. It does not change the canonical agentctl, clawd, webd, skillctl, or clawcli entrypoints; the agent-runtime service/data namespace; X-Agent-Key; API routes; permissions; task and conversation records; or skill storage and receipts. A rename therefore does not require skill source changes or data migration.

Run the two-brand contract before publishing identity changes:

python3 scripts/check_product_identity_coupling.py --self-test
python3 scripts/check_product_identity_coupling.py
bash scripts/product_identity_tests.sh --with-ui

Agent Loop Architecture

Agent Runtime's main natural-language path uses a Codex / Claude style agent loop. Before the first planner call, the front door materializes text, audio transcripts, and attachments; binds task/session identity; and builds a machine-owned TurnBoundaryEnvelope containing explicit API fields, locators, permission/budget profiles, and safety context. Every ordinary ask then enters the agent loop, which decides whether to answer, clarify, or execute. A native model turn selects either call_capability for another observation/effect or respond for the terminal model-authored answer; providers using the structured plan protocol express equivalent verified steps. Recoverable failures return through RepairEnvelope machine fields, attempt history, and checkpoint state instead of user-language phrase matching.

Request and Agent Loop Flow

flowchart TD
    A[Channel / UI / API request] --> B[POST /v1/tasks]
    B --> BA[Authenticated task execution policy<br/>server-owned machine envelope]
    BA --> C[Persist task + queue]
    C --> D[Return task_id<br/>caller can poll]
    D --> E0[worker_once recovery tick<br/>stale running + due checkpoint]
    E0 --> E1[Claim next queued task]
    E1 --> E{Task kind}
    E -->|run_skill| RS[Direct run_skill path<br/>explicit skill_name only]
    E -->|ask| F[Materialize text/audio/attachments]
    F --> G[TurnBoundaryEnvelope<br/>identity + explicit machine facts + safety/budget profile]
    G --> H[Ask context bundle<br/>memory provenance + recent execution + goal/journal refs]
    H --> J[Agent loop<br/>ordinary semantic authority]
    J --> L{Loop round}
    L --> N[Planner LLM<br/>native call_capability / respond<br/>validated structured plan when required]
    N -.-> PS[Recognized respond/free_text bytes<br/>public-output policy + presentation events]
    N --> P[PlanVerifier<br/>permission_decision + risk + effect + contract]
    P --> Q{Verified step}
    Q -->|respond| R[Structured terminal response<br/>free_text, exact list, authored object,<br/>or observed object projection]
    Q -->|synthesize_answer| S[Grounded synthesis]
    Q -->|call_tool / call_skill| QP[Pre-tool hooks + adapter preflight<br/>policy_decision + contract args]
    QP --> MG{Non-idempotent mutation?}
    RS --> RSG[No planner / resolver choice<br/>no verifier semantic selection]
    RSG --> MG
    MG -->|yes| MI[Persist intent + deterministic idempotency key<br/>persist attempt before invocation]
    MG -->|no| EX{Execution adapter<br/>sandbox backend diagnostics when process-backed}
    MI --> EX
    EX -->|long-tail async_start| AS[Async media/job adapter<br/>pending_async_job + poll/cancel contract + checkpoint]
    AS --> ASP[Progress machine reply<br/>checkpoint_id + poll_ref + next_check_after + can_poll/can_cancel]
    EX -->|call_tool| T[Tool execution]
    EX -->|call_skill / direct run_skill| U[Shared skill dispatch]
    T --> MR{Mutation receipt state}
    U --> MR
    MR -->|not a mutation| V[Observed result]
    MR -->|receipt returned| MC[Persist receipt + verification + commit<br/>under exact worker claim]
    MR -->|timeout / crash ambiguity| MU[Reconciliation checkpoint<br/>never infer from prose]
    MC --> V
    MU --> ASP
    S --> V
    V --> W[Evidence coverage + answer-shape check]
    W -->|repair / missing evidence| WR[RepairEnvelope<br/>issue codes + attempt ledger]
    WR --> J
    W -->|round observed| BD{BudgetDecision<br/>progress + deadline + policy + hard ceilings}
    BD -->|continue| J
    BD -->|finish| X[Observed-output finalizer]
    BD -->|checkpoint_requeue / waiting / needs_user| BC[Persist TaskBudgetSlice + task_checkpoint<br/>release exact claim]
    BC --> ASP
    BD -->|terminal| X
    R --> Y[User-visible message assembly]
    X --> Y
    Y --> Z[Output-contract guard + task result]
    Z --> AA[Channel delivery]
    Z --> AB[Journal + session update]
    AB --> AD[Task event stream<br/>goal + context + transition + checkpoint + tool/coding/team events]
    PS --> AD
    ASP --> AA
    ASP --> AB
    ASP --> AD
    AD --> AE[CLI / UI watch + report]
    AD --> AF[Teaching mode timeline<br/>per-turn trace selection + raw JSON details]
    Z -. optional .-> AC[Background memory refresh]

POST /v1/tasks: channel daemons, the browser UI, and HTTP callers converge on the same persisted task queue.
Authenticated task execution policy: clawcli stays on the configured approval/sandbox policy unless an enabled admin key explicitly requests global --yolo. Other communication adapters authenticated with an enabled admin key default to YOLO. The server removes caller-supplied policy envelopes, reissues the machine contract after authentication, and revalidates the current admin key before each use. YOLO means approval_policy=never plus sandbox_mode=danger_full; it does not bypass registry allow/deny, schemas, path validation, external-publish controls, cancellation, budgets, redaction, or audit evidence.
task_id polling: API/channel request timeouts only affect how long the caller waits. The background task remains queryable through GET /v1/tasks/{task_id} unless worker lifecycle logic marks it terminal.
worker_once recovery tick: before claiming new queued work, the worker checks stale running tasks, protected paused checkpoints, due resume work, async poll results, and result projections.
Task kind: kind=ask enters the planner-owned natural-language path; kind=run_skill bypasses the planner loop, capability selection, and plan verifier, then calls the explicitly requested skill through the same shared skill dispatcher/protocol used by planner skill calls. Both task kinds persist results under the original task_id, so callers can still inspect final state through task query APIs.

Ask and Run Skill Boundary

This boundary is intentionally explicit because run_skill is an API-level task kind, not a natural-language routing shortcut.

Quick facts for direct skill tasks:

kind=run_skill does not run the planner / agent loop. The caller already supplied payload.skill_name and args.
kind=run_skill still uses the shared skill dispatcher and skill protocol after the explicit skill name is accepted.
kind=run_skill still creates and updates a normal task row, so the final state and result remain queryable by task_id.

Question	`kind=ask`	`kind=run_skill`
Is there a pre-planner semantic LLM/router?	No. The front door only builds `TurnBoundaryEnvelope` and context refs.	No. The caller already supplied the target skill.
Does it enter the planner / agent loop?	Yes for every ordinary natural-language task.	No. It does not ask the planner to choose a skill or action.
Does it use `CapabilityResolver` / `PlanVerifier` as semantic selectors?	No. The planner owns ordinary semantic choice; resolver/verifier resolve and validate planned steps before execution.	No. Direct skill tasks bypass semantic selection; the explicit skill call still uses dispatch/protocol validation.
Does it use the shared skill dispatcher?	Yes when the planner chooses `call_skill` or a capability resolved to a skill.	Yes. It dispatches `payload.skill_name` through the same builtin / external / runner skill protocol.
Is the result queryable by `task_id`?	Yes.	Yes. The direct skill result is saved under the original task row and can be read through `GET /v1/tasks/{task_id}` or `clawcli get`.

Operationally: use kind=ask when the user gave a natural-language request and Agent Runtime should decide whether to answer, ask, plan, or execute. Use kind=run_skill when an API caller already knows the exact skill and args and only wants Agent Runtime to run that explicit skill under the task queue, auth, lifecycle, and result projection machinery.

Planner-owned front door: materializes text/audio/attachments and builds TurnBoundaryEnvelope from task identity, explicit API fields, structured locator facts, and safety/budget profiles. It performs no semantic LLM call and contains no ordinary respond/clarify/execute branch.
Agent-loop semantic authority: every ordinary natural-language task enters the loop. Native turns choose call_capability or structured respond; structured-plan turns may express equivalent tool, skill, synthesis, clarification, repair, checkpoint, or stop steps.
CapabilityResolver / PlanVerifier: resolves call_capability into the current tool or skill implementation, then checks visibility, required arguments, allowed action, risk/effect, confirmation, and output contract before execution.
permission_decision: verifier and preflight blockers expose machine fields such as allowed, needs_confirmation, denied_by_policy, dry_run_required, external_provider_blocked, risk_level, action_effect, and registry dedup/idempotency metadata. UI, API clients, finalizers, and i18n should render these fields instead of parsing runtime prose.
Side-effect outbox: non-idempotent mutations from both planner-owned execution and explicit kind=run_skill persist intent_recorded and attempt_started before invocation. A deterministic key derived from task plus canonical action fingerprint enters runner context, external HTTP Idempotency-Key, or supported local-adapter environment. Receipt, verification, reconciliation, and commit transitions are fenced by the exact worker claim. Receipt-bearing states suppress original-action replay; ambiguous timeout/crash state checkpoints as mutation_reconciliation and accepts only a fingerprint-bound structured applied|not_applied|still_unknown resume constraint.
Async job start: long-tail tool work can publish a machine reply with checkpoint_id, poll_ref, next_check_after, can_poll, and can_cancel while the task remains recoverable through checkpoint polling. Media skills expose this shape through registry capabilities such as image.generate / image.poll / image.cancel, audio.synthesize / audio.poll / audio.cancel, video.generate / video.poll / video.cancel, and music.generate / music.poll / music.cancel.
Capability result observation: every successful CapabilityResultEnvelope is projected into one bounded, redacted generic machine observation for the next planner turn. Optional domain projections may compact common evidence, but unknown or newly installed capabilities retain structured provider, artifact, async-job, effect, and verification fields without requiring a new runtime branch.
Evidence coverage: tool, skill, and synthesis outputs become loop observations. Missing evidence or recoverable failures go back into the loop with compact attempted-method history.
TaskBudgetSlice / BudgetDecision: interactive work is governed by resumable soft wall-time slices and structured progress, not ordinary max_rounds or max_tool_calls completion thresholds. After each observed model/tool result, runtime chooses continue, finish, checkpoint_requeue, waiting, needs_user, or terminal from verifier-approved plan facts, evidence/artifact progress, continuation state, policy, cancellation, deadlines, and administrator hard ceilings. Profile timeout classes cap planner provider calls and agent-loop tool/MCP calls before the slice boundary; a timed-out mutation enters reconciliation instead of blind replay. The model can request continuation but cannot raise cost, permission, time, or resource ceilings.
RepairEnvelope: repair is bounded loop recovery. Runtime supplies machine fields such as repair_source, issue_codes, missing_evidence, permission_decision, provider_status, attempt_fingerprint, side_effect_fingerprint, checkpoint_id, and next_recovery_kind; planner/finalizer can use those fields to replan, clarify, wait in background, or fail structurally without parsing localized prose.
Observed-output finalizer: publishes grounded results only after the answer shape and evidence contract are satisfied.
Output-contract guard: normalizes final text, message arrays, file tokens, scalar/strict shapes, and channel delivery consistency before the result is saved.
Journal + session update: task state, observed facts, and active-session anchors are persisted after finalization; background memory work is optional and non-blocking.
Task event stream: journal trace events expose machine-readable progress such as task_goal, context_budget, context_compaction, budget_decision, task_transition, checkpoint_created, assistant_output_started|delta|completed|aborted|replaced, tool/coding/provider/hook/subagent events, agent_team_started, subagent_finished, agent_team_aggregated, and task_final. Raw provider deltas remain private; only recognized respond/free_text bytes that pass public-output policy become presentation events. Provider/context projections retain machine metrics such as prompt_truncation_count and prompt_bytes_before_max. CLI and UI render these fields directly, including budget, continuation, goal/checkpoint, verification, child-graph, team, and safe assistant-output progress, instead of reading raw logs or localized text. Coding events are immutable snapshots: a resumed task appends a higher projection revision, and consumers select that latest projection while retaining earlier red-test evidence as history.

Planner, LLM, and Capability Flow

Detailed flow: Agent loop and planning.

TurnBoundaryEnvelope: is built deterministically from authenticated task/session state, attachments, explicit API fields, locators, and policy profiles. It is context for the planner, not a semantic route result.
Planner prompt: is the first semantic LLM call for an ordinary ask. Resume and async-poll executors may restore a previously admitted machine checkpoint, but they cannot introduce a new pre-planner semantic decision path.
call_capability: is the preferred planner action because it keeps skill/tool choice behind registry metadata and resolver policy.
CapabilityResultEnvelope: returns to the next planner turn through a bounded, redacted generic projection. Skill-specific projections are optional optimizations; they are not required for a new capability's structured data to remain visible to the loop.
respond: is the native terminal formatting action, not a capability simulator. Ordinary answers carry model-authored free_text; strict lists carry an item array and exact count. Model-authored named-field/JSON replies use object, where each value_json is one complete serialized JSON value and malformed values receive bounded structured repair rather than silent coercion. When requested values already exist in successful CapabilityResultEnvelope observations, observed_object carries only output names, exact capability tokens, and language-neutral dotted paths; runtime copies the original JSON values without asking the model to re-serialize nested machine data. Missing, failed, or invalid references are rejected. Provider omissions for payloads unused by the selected shape canonicalize only to empty/zero, and redundant authored object content is accepted only when its parsed JSON exactly equals the named fields. Runtime-owned provider/config/permission, domain parse/normalize/validate/preview, dry-run, artifact/job, checkpoint, diff, verification, repair, and rewind fields require a prior matching capability observation. A lower-level environment fact may support that call but cannot replace the disclosed domain capability that owns the result. Runtime materializes terminal machine payloads without parsing multilingual user text or adding fixed prose. A single scalar, identifier, title, token, or path stays free_text.
Generated INTERFACE prompts: come from crates/skills/*/INTERFACE.md, optional_skills/*/INTERFACE.md, external_skills/*/INTERFACE.md, and prompts/layers/generated/skills/*; new skills should improve these contracts instead of adding clawd main-flow branches.
Exact machine output: the planner requests response_shape=strict plus a validated structured_field_selector such as command_output; the runtime projects only that field from CapabilityResultEnvelope. Free-form and one-sentence contracts remain model-synthesized.
PlanVerifier: blocks unavailable capabilities, missing required fields, unsafe mutations, and disallowed output/evidence shapes before any executor runs. Denials should carry stable machine fields rather than user-facing fixed reply text.
Pre-tool hooks + adapter preflight: loop execution and bounded recovery retries pass through the same hook, contract-argument, command-policy, and structured error checks before any effectful adapter runs.
Task journal event: executor observations are projected into stable task_goal, context_budget, context_compaction, tool_started, tool_step, tool_finished, optional coding_checkpoint, optional coding_task_contract, optional coding_evidence, and optional team lifecycle events with refs, counts, status tokens, verification tokens, timing, and failure attribution for CLI/UI progress views.
agent.subagent / agent.subagent_batch / agent.subagent_persistent: planner-authorized child work enters through the same native call_capability -> registry resolver -> verifier path as other runtime actions. All three capabilities now materialize durable child-thread IDs and use the same child DAG scheduler. Single/batch children stay read-only; persistent writer roles remain explicitly high-risk and run in isolated worktrees. The per-session open-thread limit is configured by max_concurrent_threads_per_session and excludes the main Agent. Legal overflow remains queued_capacity instead of being skipped. join_wait_ms only bounds the parent's current wait and never cancels a child; ordinary children have no default whole-operation deadline. The planner cannot create one: an API caller must explicitly set subagent_execution.runtime_deadline_ms on the original parent task payload, and the source remains auditable. A parent owns one durable child graph, so wake-up, reconnect, and repeated planner calls reuse its child IDs; new constraints use steering or typed retry controls. Trusted role definitions and isolation/permission policy come from registry plus agent_guard.toml; planner-supplied policy fields are discarded. The queue claims only ready nodes, serializes overlapping writer ownership, and permits disjoint isolated worktree writers to run concurrently. The parent reviews persisted ownership, conflicts, dirty-parent state, and precondition hashes before admitting or rejecting a patch. Children never publish externally, and only an explicitly admitted local-current-workspace role may write outside an isolated task worktree.
Skill dispatcher: uses the same dispatch layer for direct run_skill and planner skill calls. Direct run_skill does not ask the planner to choose a skill; it only dispatches the explicit payload.skill_name. Builtins run in-process, external skills use adapters, and runner skills launch skill-runner plus the concrete binary.
Runner credential bridge: runner subprocesses never inherit the parent environment wholesale. When the verified action declares credential access, clawd derives the selected structured provider connection and issues separate one-use secret tokens for the canonical vendor variable and any required protocol alias, such as MINIMAX_API_KEY plus OPENAI_API_KEY for an OpenAI-compatible MiniMax adapter. Offline preview actions receive neither token.
Skill process protocol: runner skills receive one request JSON line and finish with one final response JSON line. Skills that explicitly declare run.progress_frames=true may emit bounded, versioned machine progress records first; stable decision fields still belong in final extra.
synthesize_answer: is scheduled inside the loop when evidence needs natural-language synthesis; it is not a fixed final LLM call after every task.
RepairEnvelope: verifier, executor, permission, provider, and checkpoint recovery paths expose structured repair context to the next loop round; user-visible fallback prose should come from i18n, finalizer, UI, or the model, not runtime templates.
Output-contract finalization: is a thin protocol boundary. It preserves exact validated machine fields and artifact transport, otherwise it publishes the model's evidence-grounded synthesis; it does not select skills or render domain-specific prose.

Permission Plane and Command Policy

The permission plane is a structured execution boundary. Registry metadata from configs/skills_registry.toml, bundled evidence policy for non-capability output shapes, and verifier state are projected into permission_decision so UI/API/finalizer layers can explain what happened without hardcoded runtime prose. Ordinary registry capability families are selected by planner call_capability plus resolver metadata.

risk_level, requires_confirmation, once_per_task, idempotent, and dedup_scope come from registry and planner capability metadata where available.
action_effect is derived from structured skill/action args and contract metadata, not from user-language phrase matching.
run_cmd decisions include a nested command_policy for machine fields such as policy_authority, literal_command_token, command_arg_present, unresolved_runtime_template_present, and command effect flags.
Explicit user command preservation is represented by _clawd_literal_command; otherwise run_cmd is treated as planner-structured command args and remains subject to contract and media-artifact blockers.
Recovery paths such as non-interactive sudo retry are still adapter calls: they must reuse the same contract, hook, policy, and audit machinery as the original planner step.
Risky local coding or file-mutation capabilities should declare an isolation profile in registry metadata. local_temp_workspace is for disposable previews, dry runs, and generated artifacts that can be cleaned through artifact refs; local_worktree is for deliberate workspace edits that must be visible through task evidence, changed-file refs, and verification commands. UI and CLI surfaces read permission_decision.steps[].sandbox, workspace_scope, and registry_policy instead of interpreting localized text.
Confirmation decisions use the closed machine protocol approve_once|always_for_scope|deny. always_for_scope is exposed only for registry-declared local workspace mutations with an exact capability/effect/resource scope; it excludes run_cmd, network access, external publish, credential access, package installation, and privilege escalation. Grants are HMAC-signed, bound to the authenticated actor plus channel/chat session, expire after at most one hour, and are stored, matched, listed, and revoked by clawd. CLI/UI state never grants permission by itself.

Sandbox and Cross-Platform Execution

sandbox_mode defines the permission scope; [tools].sandbox_backend selects the platform mechanism that enforces that scope. The two settings never substitute for each other. The default backend is auto: it resolves to Bubblewrap on Linux and Seatbelt (/usr/bin/sandbox-exec) on macOS. Selecting a backend does not grant access; the verifier still applies capability effect, risk, confirmation, workspace, network, credential, and privilege policy before an adapter starts. When a restricted mode cannot find its backend, the platform does not match, or the remote executor is unconfigured, execution is rejected with a structured result instead of silently falling back to an unsandboxed process.

Detailed flow: Security and execution.

Backend	Host	Selection	Current contract
Bubblewrap	Linux	`auto` or `bubblewrap`	Filesystem write scope, PID/IPC/UTS namespaces, optional network namespace, parent-bound or durable lifetime. Missing `bwrap` fails closed.
Seatbelt	macOS	`auto` or `macos_seatbelt`	Read-all/write-scoped profile, optional network access, process policy, parent-bound or durable lifetime. Missing `sandbox-exec` fails closed.
Remote container	Any	explicit `remote_container`	Reserved executor contract only. It returns `sandbox_remote_backend_not_configured` until a remote executor is configured and is never an automatic fallback.
Direct	Any	explicit `sandbox_mode = "danger_full"` or authenticated per-task YOLO	No sandbox claim. This is an operator-selected bypass, not an availability fallback.

clawcli --yolo <task-producing command> requests direct execution for that task and requires a currently enabled admin key. It is intentionally not the CLI default. Communication adapters other than clawcli default to the same mode when their request is authenticated as admin, so channel admin keys must be treated as full execution credentials. The backend remains the only policy authority; request payloads, browser state, planner output, and user wording cannot grant this mode.

Diagnostics report the requested/resolved backend, platform, availability, fail-closed state, reason code, and filesystem/network/process/credential/ resource/environment control levels. Service discovery is also platform-owned: Linux may use systemd/SysV, while macOS uses Homebrew services, launchd, or process observation. An incompatible explicit manager returns a structured unsupported_platform result without launching a Linux command. Development and release scripts use scripts/shell_compat.sh instead of GNU-only file/date commands or Bash 4-only collection syntax. See docs/cross_platform_contract.md.

Trusted lifecycle hooks are configured in configs/agent_guard.toml and stay disabled by default. Administrators can inspect a redacted status projection through GET /v1/admin/hooks/status or the browser Models page: it reports setup state, enabled/valid/invalid counts, trust and hash readiness, and every supported stage, without returning handler arguments, endpoint URLs, environment references, or credentials. Enabling or trusting a hook still requires reviewed repository configuration; the UI cannot promote an unreviewed script into an execution boundary.

Natural Language Contract Boundary

Agent Runtime keeps natural-language understanding on the LLM side and deterministic execution on the runtime side. The planner may read user wording, examples, skill docs, and multilingual prompt guidance, but it must turn that understanding into structured actions before runtime code acts on it. The pre-planner front door may only expose authenticated machine facts through TurnBoundaryEnvelope; it cannot infer ordinary intent.

Runtime code should consume stable contracts such as:

evidence-policy answer-shape fields, for example final_answer_shape = "summary_with_evidence" and final_answer_shape_class = "grounded_summary"
planner-owned capability refs, for example capability_ref = "package.detect_manager" or call_capability("package.detect_manager")
action names, for example read_field, validate_config, or transform_data
registry metadata and planner_capabilities
EvidencePolicyContext / OutputContract fields, target locators, and explicit field_path values
JSON/TOML/YAML field paths, file extensions, structured tool output, exit codes, error kinds, and risk/effect metadata
permission_decision and command_policy machine fields

Runtime code should not add per-language phrase tables or prompt.contains(...) branches to make a single natural-language case pass. If a new user wording needs better handling, update registry capability metadata, INTERFACE.md, generated skill prompts, planner schema, or a necessary vendor prompt patch so the LLM emits the same structured action in any language. Ordinary skills such as weather, web, image, photo, publishing, package manager, Docker, RSS, and market quote flow through registry capability metadata. python3 scripts/check_no_nl_hardmatch.py is the local guard for this boundary.

Memory System

Agent Runtime memory is split into short-term conversation records, structured user preferences, long-term fact cards, and retrieval indexes. The design goal is to make memory useful without letting old assistant output become a hidden instruction for a new task.

Core Boundaries

Memory is scoped to the authenticated identity first, then to the current conversation. Channel IDs from Telegram, WeChat, Feishu, browser UI, and other adapters are normalized into the same task identity model, so a bound user_key can keep memory consistent across channels while still preserving user_id / chat_id level conversation state. Recent conversation state stores active-task anchors, alias bindings, and follow-up context separately from durable facts; it is allowed to help resolve “that file” or “the previous result”, but it is not treated as a new user instruction.

The memory layer has three hard boundaries:

current user input always wins over recalled memory
memory text is background context unless current task/session state explicitly binds it to the turn
runtime code consumes memory through structured fields, source kinds, scores, safety flags, and use-policy decisions rather than per-language phrase branches

This keeps old assistant output, task logs, and knowledge snippets from silently steering execution. If recalled context conflicts with the current request, the planner prompt and memory-use policy require the current request to win.

Storage Model

The main persisted memory stores are:

memories: short-term conversation records and task-visible snippets. Rows keep role, memory type, salience, safety state, timestamps, success state, and source metadata.
conversation_states: active per-chat state such as alias bindings, active task anchors, and follow-up state. This is session state, not durable knowledge.
user_preferences: structured user preferences such as response language, response style, response format, and agent display name.
memory_facts: durable fact cards with fact_key, fact_value, fact_text, source refs, confidence, status, expiry, and conflict-group metadata.
long_term_memories: supplemental summary rows used only where the current memory policy allows summary recall.
memory_retrieval_index: hybrid retrieval index over short-term records, preferences, fact cards, and knowledge snapshots.

configs/memory.toml controls budgets, retention, long-term refresh intervals, write filters, preference extraction, retrieval limits, and embedding/index behavior. Defaults are conservative: short acknowledgement messages can be filtered, assistant replies are marked, and LLM-written preferences must pass confidence and runtime validation before they are stored.

Write Path

After an ask task finalizes, Agent Runtime can persist:

short-term records in memories, scoped by user_key, user_id, chat_id, role, memory type, salience, and safety flag
user preferences in user_preferences, such as response_language, response_style, response_format, and agent_display_name
long-term fact cards in memory_facts, with source, confidence, scope, status, conflict group, expiry, and supersede metadata

Preference and fact writes go through a structured memory intent contract. The model is asked to emit memory_actions such as upsert, delete, expire, or noop; runtime code then validates action enum, kind, scope, confidence, source evidence, TTL, and safety fields before anything is stored. The runtime does not decide durable preference writes by matching a single natural-language phrase.

Long-term summary refresh provides supplemental recall, while durable knowledge is stored as fact cards. A fact card keeps fact_key, fact_value, human-readable fact_text, source_ref, source_memory_ids_json, reason, confidence, expires_at_ts, conflict_group, and status. New active facts in the same conflict group supersede older facts, and expired or deleted facts are removed from retrieval.

Memory writes are intentionally after-answer work. The user-visible response is saved first; then background memory refresh can run when configured. This prevents memory extraction latency from blocking normal replies and makes memory write failures non-fatal to the already completed task.

Recall and Use Policy

Memory recall is built as structured context and then filtered by the current consumer's memory use policy:

planner: can use unfinished goals, preferences, relevant facts, and knowledge docs, but excludes supplemental summaries, assistant results, similar triggers, and raw recent snippets
chat: uses stable preferences and facts; bounded recent context is allowed only when current session state makes it relevant
skill: _memory is cropped by the skill registry memory_policy; skills without a policy get a safe default scoped profile

The photo_organize skill, for example, declares a memory policy that allows preferences, relevant facts, and knowledge docs while excluding long-term summaries, recent events, assistant results, similar triggers, unfinished goals, and raw recent snippets.

Each use-policy decision records what it included and why. Prompt builders receive already-filtered structured context rather than raw database rows. The common policy is:

new standalone tasks get stable facts and preferences, not old assistant results
follow-up turns can use recent observations and active aliases only when session state says the user is continuing the same task
planner prompts can see enough memory to avoid repeating work, but memory remains background and cannot override the current request
skill _memory payloads are cropped per skill registry policy so specialized skills only receive the memory sources they are expected to use

Retrieval Index

Hybrid recall uses memory_retrieval_index plus optional FTS. Each indexed row records source_kind, source_ref, memory kind, metadata, salience, success state, and embedding metadata:

embedding_model
embedding_dims
embedding_version

The default provider is local-hash-v1, which runs offline. Unsupported or unavailable embedding providers fall back to local hash so the runtime keeps working. Retrieval only uses cosine scoring when the stored embedding metadata matches the current provider spec; mismatched rows fall back to lexical, salience, recency, and success-state scoring. Set reindex_on_startup = true in configs/memory.toml, or start with an empty index, to rebuild the runtime retrieval index from short-term records, preferences, and fact cards. KB document rows remain in the separate KB-owned database and are queried through its own pool.

Retrieval combines several signals instead of trusting a single score: exact / lexical matches, vector similarity when compatible, salience, recency, source kind, success state, safety filter, and the current memory use policy. This makes the index useful for multilingual recall while keeping execution grounded in planner actions and output/evidence contracts.

Knowledge Base Design Flow

The kb skill is the user-managed document knowledge-base path. It is selected like other ordinary capabilities: ask tasks let the agent loop plan call_capability("kb.*"), while direct API callers can use kind=run_skill with skill_name=kb. Runtime code does not special-case knowledge-base wording before the planner; it resolves and verifies registry capability metadata, then dispatches the same skill protocol as other runner skills.

Detailed flows: Task state and context and Skill-owned storage.

Key boundaries:

kb.ingest is a local mutation capability. Registry policy marks it medium risk, once per task, and async-preferred through the local process adapter.
kb.search, kb.list_namespaces, and kb.stats are observe-mode capabilities. They return structured machine fields such as namespace, hits, names, document_count, and chunk_count.
Namespace snapshots and kb_doc retrieval rows live only in the KB-owned database under [database].skill_data_root (normally data/skills/kb/state.db). The runtime queries that pool during recall; it does not copy KB rows into the main runtime database.
KB rows are scoped by user_key and workspace files, not by one chat thread. They can be recalled later only through the same memory use-policy filters as other knowledge docs, and current user input remains the authority.

User Control

The browser console includes a Memory page. It shows counts, preferences, fact cards, and recent records for the current identity. Users can:

delete a preference, fact, or recent memory item
mark a fact card as expired
clear recent records, preferences, facts, or all memory for the current identity
enable or disable long-term memory through configs/memory.toml

The HTTP API behind the page is:

GET    /v1/memory
GET    /v1/memory/recent
GET    /v1/memory/preferences
GET    /v1/memory/facts
DELETE /v1/memory/:id
POST   /v1/memory/:id/expire
POST   /v1/memory/clear
POST   /v1/memory/settings

Recent records with safety flags are hidden by default in the UI. Fact-card details such as reason, source, and conflict group are available in a secondary details view instead of being shown as raw JSON first.

Trace and Troubleshooting

Task journal summaries and traces include memory_trace. This records the stage, use policy, recalled source refs, inclusion reason, and character budget without copying raw memory text. It is intended for debugging why a task used memory while reducing the chance of leaking sensitive stored content.

The browser teaching-mode trace, clawcli llm-trace, and /v1/debug/tasks/{task_id} also show a compact flow_summary above numbered LLM calls, with stage/module/retry/verifier/finalizer/provider-error machine counts, structured memory/KB policy, model_catalog_trace, model_catalog_trace.readiness, and resume_trace next to raw request/response details. Each browser chat turn keeps only a lightweight task/trace index; raw provider requests and responses are reloaded from the current server log or its retained seven-day dated archives and are never copied into browser storage. The trace response reports whether full detail is available, metadata-only, still pending, or unavailable because it was not recorded or has expired.

When teaching mode is selected, clicking either the user's question or the assistant's reply selects that turn and shows the corresponding task id, status, LLM call count, stage count, verifier/finalizer counts, goal/context/team/coding/checkpoint event timeline, model/provider capability decision, selected-model readiness decision, resume/checkpoint decision, and numbered raw LLM request/response details. When teaching mode is not selected, message clicks do not change the teaching trace.

Execution boundaries are exposed as machine fields instead of prose-only notes. Teaching mode, subagent review, clawcli report, and replay tooling should consume fields such as workspace_root, current_process_cwd, current_workspace_scope, write_enabled, external_publish_enabled, allowed_roles, runtime_config.max_concurrent_threads_per_session, the legacy-compatible runtime_config.max_parallel_readonly, thread_state, execution_state, queue_reason, join_wait_ms, runtime_deadline_ms, hook_stages, hook_decisions, permission_decision, policy_decision, checkpoint_id, poll_ref, and provider_blocker. Consumers do not infer lifecycle from finalizer or child prose.

When debugging memory behavior, check these questions in order:

Was the request a new task or a follow-up bound to an active session?
Which stage built the memory context: route, planner, chat, schedule, image, or skill?
Did memory_trace include the expected source_kind / source_ref?
Did the use policy exclude recent assistant output or long-term summaries by design?
Was the index stale because embedding metadata changed or reindex_on_startup was false?
Did a fact conflict group supersede the older fact?
Was the item hidden because it was expired, deleted, low confidence, or safety-risk flagged?

Useful code and config entry points:

configs/memory.toml
crates/clawd/src/memory/intent.rs
crates/clawd/src/memory/apply.rs
crates/clawd/src/memory/facts.rs
crates/clawd/src/memory/use_policy.rs
crates/clawd/src/memory/retrieval.rs
crates/clawd/src/memory/indexing.rs
crates/clawd/src/memory/api.rs

Background, Resume, and Memory Flow

Detailed flow: Task state and context.

Important lifecycle details:

Foreground HTTP/channel waits are short by design. A caller that stops waiting should keep polling the same task_id; it should not create a duplicate task or treat the background task as failed.
task_lifecycle is machine-readable. Query APIs expose state, db_status, can_poll, can_cancel, checkpoint_id, resume_due, resume_wait_seconds, and heartbeat fields for UI rendering.
Source of truth: crates/clawd/src/task_lifecycle.rs owns lifecycle projection, and repo::get_task_query_record() attaches that projection to GET /v1/tasks/{task_id}. UI, CLI, and channels should render these structured fields instead of deriving status from text or error_text.
clawcli get, clawcli watch, and clawcli wait <task_id> --until terminal|completed|background|needs-user render or wait on lifecycle machine fields; clawcli cancel-task <task_id> cancels by task ID, while clawcli cancel-index cancels the selected entry from the active-task list.
clawcli resume-task <task_id> marks an existing checkpoint due for recovery; clawcli continue <task_id> [message] is a shorter structured resume entrypoint; clawcli pause-task <task_id> --pause-seconds N delays an existing waiting/background checkpoint. These commands do not restart tasks without checkpoint state.
clawcli submit --detach returns a task_id quickly; clawcli submit --wait polls until terminal state; --json keeps submit/watch output script-friendly.
clawcli --yolo submit|exec|code|chat|run-skill ... requests approval_policy=never and sandbox_mode=danger_full for newly submitted tasks. The backend accepts it only for a currently enabled admin key. This is a high-risk mode: it removes local approval prompts and process sandbox isolation, while registry, schema, external-publish, cancellation, budget, redaction, and audit controls remain active.
clawcli exec is the CI/script-oriented runner: it submits or resumes an ask task, waits by default, returns stable exit classes/codes, supports --profile quick|coding|release-gate|long-tail, can stop on background checkpoints, prints budget/coding/resume evidence as exec_compact_* machine lines when present, and can write summary.json, task.json, events.jsonl, verification.json, diff_summary.json, llm_summary.json, resume.json, and index.json artifacts. clawcli code is the concise coding-agent shortcut for exec --profile coding. See docs/clawcli_exec_replay.md.
clawcli goal start/status/pause/resume/edit/clear manages structured long-task goal contracts with objective, done_conditions, verification_commands, constraints, checkpoint resume fields, and redacted control responses.
clawcli active prints a compact task table by default and supports --json; clawcli events <task_id> prints filtered task event streams with optional --jsonl and machine filters such as --event-type, --checkpoint-id, --policy-decision, --subagent-id, and --async-job-id.
clawcli tui --user-id <id> --chat-id <id> is a terminal task console over the same task APIs; add --once for a single snapshot and --task-id <task_id> for selected task details. Selected-task snapshots include raw task data plus selected_progress and selected_summary machine fields for checkpoint/resume, goal/outcome state, LLM budget/calls, coding verification, side effects, and artifacts. Interactive key tokens are stable machine commands: r refresh, w watch, p pause, c cancel, u resume, n continue, e export, 1 report, 2 review, 3 subagents, 4 permission, and q quit.
clawcli session list/show/resume/archive/delete/fork keeps a local session navigation store for session_id, task_ids, active_goal_id, workspace_root, checkpoint, event sequence, archive status, and fork source. This store is operator metadata under APP_CLAWCLI_SESSION_STORE, $XDG_STATE_HOME/agent-runtime/, or ~/.local/state/agent-runtime/; it is not used as a natural-language route source.
In interactive chat, /continue resumes the current background/checkpoint task from persisted thread state without copying a task id, /approve approves the exact pending action once, /approve-scope approves only the backend-provided exact capability/resource scope for the current session, and /deny closes the pending request. clawcli permission grants lists server-side scope grants and clawcli permission revoke <grant_id> revokes one immediately. The browser Tasks page exposes the same structured choices and revocation API.

clawcli session list --user-id 1 --chat-id 1 --json
clawcli session show task-123 --json
clawcli session resume task-123 "continue from the checkpoint" --json
clawcli session archive task-123 --json
clawcli session fork task-123 task-123.fork --json
clawcli session delete task-123 --json

clawcli goal start "make the focused change" --objective "ship the fix" --done tests_pass --verify "cargo test -p clawcli" --json
clawcli goal status task-123 --json
clawcli goal pause task-123 --pause-seconds 3600
clawcli goal resume task-123 --checkpoint-id ckpt-123 --message "continue from checkpoint"
clawcli goal edit task-123 --objective "updated goal" --done tests_pass --goal-status background
clawcli goal clear task-123

clawcli llm-trace <task_id> [--raw] [--limit N] reads the task debug endpoint and prints numbered LLM calls with llm_call_ref=LLM#1..N, flow/code attribution, provider/model/status tokens, usage tokens, and optional raw request/response fields.
Task event streams include goal, context budget/compaction, task-budget decisions, transitions, checkpoints, tool lifecycle, coding evidence, provider, hook, subagent/team, and final events. A bounded hot suffix serves SSE while an append-only redacted archive preserves the full task sequence, hash chain, and periodic source-range snapshots. archive_replay means the durable archive recovered an old hot cursor; cursor_expired now means the archive itself has a real gap. clawcli events/watch, reports, replay export, and browser task details consume these versioned events; raw event JSON stays in secondary details. See docs/task_event_archive_contract.md.
clawcli run-skill <skill_name> --args-json '{...}' submits explicit kind=run_skill work without natural-language routing; add --wait to poll the same task_id.
clawcli skills reads registry-backed skill metadata; clawcli capabilities and clawcli permission capability read flattened capability/policy metadata. Add --json when another script should consume the response.
clawcli replay export/run/diff supports redacted recorded_only replay bundles for debugging and CI comparison without live model or tool calls; replay run --coverage exposes recorded coverage, replay run --view llm|tools|checkpoints|summary filters recorded evidence, and replay diff includes taxonomy tokens such as route_changed, plan_changed, permission_changed, and final_status_changed. See docs/clawcli_exec_replay.md.
Stale ordinary running tasks become timeout; paused checkpoints in waiting or background stay running so recovery can claim them by checkpoint id.
Async long-tail tools should start an external job, write pending_async_job, checkpoint, and publish an accepted machine reply with checkpoint_id, poll_ref, and next_check_after. Poll and cancel actions should be exposed as structured capabilities when the provider or dry-run adapter can support them. Worker recovery later polls through poll_async_job.
Terminal async poll projection preserves an existing visible ask reply. If the ask task has only machine executor output, projection adds a machine JSON reply with checkpoint_id, poll_ref, task_id, and final_result_json.
Seeded resume restores the persisted TaskBudgetSlice, cumulative model/tool/token/cost/elapsed counters, continuation index, observations, artifact refs, repair state, and completed side-effect fingerprints before re-entering the agent loop. A healthy task continues while it produces structured progress; repeat/stagnation, cancellation, policy, or administrator hard ceilings stop it.
Runtime recovery and projection code moves only machine fields such as status_code, message_key, executor_state, resume_directive, job_id, and artifact refs. User-facing prose is rendered later by finalizer, i18n, UI, or the model.
Lease/heartbeat model: see docs/task_lifecycle_lease_model.md; every foreground and resume-executor write is fenced by the exact task-row (lease_owner, claim_attempt). Heartbeat only renews that claim, checkpoint recovery advances the generation, and stale workers cannot publish claimed process events or terminal results.

CLI lifecycle and its persisted teaching evidence are documented in Task state and context and Coding and observability.

Detailed Architecture Guide

GitHub README pages do not support true pagination. Detailed diagrams are maintained as an ordered guide so that each page stays focused. The Learning / Maintenance UI renders these same Markdown sources instead of maintaining a second copy:

Use the architecture index for language selection and previous/next navigation. The full documentation index links every engineering document in English and Simplified Chinese.

Main Components

crates/clawd: core runtime, HTTP API, routing, memory, scheduling, auth, task queue
crates/skill-runner: verifies an installed receipt, resolves a language-neutral SkillLaunchSpec, and supervises the skill's JSONL process or typed HTTPS adapter
crates/skill-sdk: manifest validation, isolated Cargo/Python/Node/Go/prebuilt builders, protocol checks, receipts, rollback, and developer CLI/templates
crates/clawcli: terminal CLI for talking to clawd
crates/webd: browser UI host, login/session boundary, and authenticated proxy to the internal core API
crates/telegramd, crates/wechatd, crates/feishud, crates/larkd, crates/whatsappd, crates/whatsapp_webd: channel daemons
services/wa-web-bridge: local Node bridge used by the WhatsApp Web channel
crates/skills/*: fixed/core built-in skill implementations and INTERFACE.md specs
optional_skills/*: bundled Skill Store skills compiled and installed on demand
external_skills/*: externally submitted skills and their required INTERFACE.md specs
UI/: Vite + React local console
pi_app/: small-screen desktop monitor and launcher scripts

Quick Start

The complete installation, configuration, build, systemd, cloud, Raspberry Pi, update, and troubleshooting runbook is maintained in USAGE.md. This README keeps the product and architecture overview instead of duplicating operational instructions.

Most users should download the GitHub Release package matching their platform. Build from source only for development or unsupported platforms.

# Install local command entrypoints without nginx
bash install-agent-cmd.sh --user --no-deploy-ui

# Smallest startup path
agentctl start -q

# Status, health, and logs
agentctl -status
agentctl -health
agentctl -logs clawd 200 --follow

Operational rules:

Local deployments open the UI through webd and do not need nginx.
Cloud deployments opt in to nginx only when a domain or TLS reverse proxy is needed.
Linux systemd units are generated for the detected user and workspace by scripts/install-systemd-service.sh; the repository does not keep a host-specific unit.
Raspberry Pi users should prefer the prebuilt aarch64 Release package to avoid repeated full builds on low-memory hardware.
Keep credentials in an environment file outside the repository and never commit them.

Identity and Access

Agent Runtime uses user_key as the main identity across the UI and messaging channels.

permissions are resolved by user_key
conversations are resolved by channel + external_chat_id
the browser UI sends X-Agent-Key
when the auth table is empty, clawd can bootstrap the first admin key

Key management:

agentctl -key list
agentctl -key generate user
agentctl -key generate admin
agentctl -key add rk-xxxx admin
agentctl -key disable rk-xxxx

Telegram Transport Boundary

Telegram is limited to message transport, identity binding, and task control. It no longer owns skill-specific configuration or natural-language routing:

Text, images, audio, video, and files are submitted uniformly as kind=ask; the agent loop decides whether to answer, clarify, or call a capability.
The command surface is limited to /help (including /start), /key, /cancel, and /voicemode.
/status, /run, and unknown slash text are ordinary ask input for bound users; the transport does not inspect them or query host/runtime internals.
/voicemode controls only text/voice delivery for the current Telegram chat and does not select capabilities.
Installation, enablement, and configuration of on-demand Skill Store skills, including crypto, belong to the browser management surface or controlled core APIs. Telegram does not accept skill credentials or expose skill configuration commands.
On startup, telegramd refreshes the Telegram command menu through setMyCommands; restart the channel daemon after removing old commands from configuration.

UI, API, and `webd`

The main API is provided by the loopback-only clawd, while all browser traffic enters through webd:

flowchart LR
    B[Browser]
    N[nginx<br/>optional TLS + static UI]
    W[webd :8788<br/>UI + login + session + proxy]
    C[clawd 127.0.0.1:8787<br/>internal /v1 API]
    U[UI/dist]

    B -->|local| W
    B -->|domain / TLS| N
    N -->|static files| U
    N -->|/v1 and /webd| W
    W -->|static files without nginx| U
    W -->|authenticated /v1| C

local workstation: open webd directly; it serves UI/dist and proxies authenticated /v1 requests
cloud/server: nginx may serve UI/dist and proxy /v1 and /webd to webd; nginx never proxies to clawd
webd is the browser security boundary and provides password login, session persistence, credential injection, request limits, and API proxying
clawd does not serve browser assets and cannot bind a non-loopback address; local channel daemons and clawcli may use its internal API
The dashboard keeps two separate entry controls: webd public port switches between direct device-IP access (0.0.0.0:<port>) and loopback-only access (127.0.0.1:<port>), while Web server entry configuration reports nginx installation, process, site, and UI deployment status. Keep direct webd access open when running locally without nginx. Closing it does not interrupt a configured native nginx deployment, because nginx keeps proxying over loopback.
when the UI is opened through a domain, login defaults use the current origin without appending :8787 or :8788; direct local ports are inferred only for local access
Browser voice input uses hold-to-talk: press and hold to record, then release to send the voice turn automatically. Browsers expose the microphone only to secure contexts: remote access through a LAN IP must use a trusted HTTPS endpoint; plain http://<pi-ip> cannot be granted microphone access by UI code. http://localhost remains available when the browser runs on the Agent Runtime host itself.
The Learning / Maintenance page reads the bundled README and architecture guides. It provides beginner, operator, and developer routes, full-text search, per-page navigation, saved reading progress, and Mermaid zoom/pan/full-screen controls in both UI languages.
The Agent page keeps server-backed conversation history. Each task has a directly available rename control, and the saved name remains available after refresh or restart.
On desktop, clicking anywhere in the main work area collapses the navigation sidebar; the sidebar toggle restores it. The mobile navigation menu closes after page selection or an outside click.
Dashboard task counts and the Active Tasks page share one identity scope: admins see the system scope, while normal keys see their own tasks across conversations. Dashboard running counts and oldest-running age include only tasks with a live worker lease; user-waiting, paused, and resumable checkpoints remain visible through task lifecycle surfaces without triggering long-running warnings.
The dashboard system-dependency check covers Agent Runtime runtime requirements, source/UI build tools, and native dependencies used by built-in tools and skills. It reports detected versions and capability ownership. Administrators can start allowlisted installs through a Linux package manager or macOS Homebrew when the service already has non-interactive package-manager permission; installs run asynchronously and remain observable after a page refresh. The browser cannot submit arbitrary package names, system commands, or operating-system passwords.

clawd has a fixed internal endpoint at 127.0.0.1:8787; it is not a user-facing listen setting. webd uses configs/channels/webd.toml and can listen on either 0.0.0.0:8788 for direct device-IP access or 127.0.0.1:8788 for nginx-only/local access. The dashboard preserves the configured port when switching scope and atomically updates only the listener address. Docker publishes 8788, not 8787; container networking must be evaluated before changing the listener to loopback.

Useful endpoints (send X-Agent-Key for the current UI/user key):

GET /v1/health
GET /v1/system/host-summary: returns a versioned, authenticated, secret-free host summary for the dashboard, including OS/version, architecture, memory, Agent Runtime data-volume storage, uptime, and machine-readable unavailable fields
GET /v1/system/dependencies: returns Linux/macOS dependency state, installed versions, consuming tools/skills, and controlled-install availability
POST /v1/admin/system-dependencies/install: starts an allowlisted asynchronous install by fixed dependency_id; arbitrary commands and package names are rejected
POST /v1/tasks
GET /v1/tasks/{task_id}
GET /v1/tasks/{task_id}/artifacts: returns the authenticated task artifact manifest
GET/HEAD /v1/tasks/{task_id}/artifacts/{artifact_id}/content: previews, downloads, or reads metadata for one controlled artifact
GET /v1/tasks/conversation-history
PUT /v1/tasks/conversations/{conversation_id}/title
POST /v1/tasks/active
POST /v1/tasks/cancel
POST /v1/tasks/cancel-by-task-id
POST /v1/tasks/cancel-one: cancels by active-list index
POST /v1/services/{service}/{action}: browser-console service start/stop/restart; failures return machine fields such as error_code, status_code, message_key, service, and action
GET /v1/admin/nginx: returns admin-only nginx installation, process, site, and deployed-UI status
GET /v1/admin/webd-exposure: returns the configured webd listener, port, process state, and direct-access state
POST /v1/admin/webd-exposure: atomically switches webd between direct and loopback-only access, then schedules a platform-appropriate restart
POST /v1/admin/workspace-update/nginx-enable: checks, installs, or updates nginx on Linux/macOS, repairs and starts the entry, then deploys existing UI assets
POST /v1/admin/workspace-update/nginx-disable: stops and disables nginx, then removes the Agent Runtime site and dedicated UI deployment; cloud servers lose this web entry immediately
GET /v1/auth/me
POST /v1/auth/channel/bind
GET/POST /v1/auth/crypto-credentials: reads or overwrites exchange credentials scoped to the current X-Agent-Key
GET /v1/models/catalog: returns the secret-free model/provider capability catalog used by the UI Models page and teaching-mode model_catalog_trace
GET/POST /v1/llm/config: reads or updates the main text-model selection; administrators may store a custom-provider credential in the machine-local private credential broker
POST /v1/llm/test: tests a draft text-model configuration without persisting a supplied credential
GET/POST /v1/admin/model-config: reads or updates the independently selected provider, model, endpoint, and managed credential for each multimodal module
GET/POST /v1/skills/config: reads or updates skill enablement; each multimodal skill can be switched independently without discarding its model settings
GET /v1/skills/store: returns the registry-driven catalog for optional bundled and imported skills, including separate installed and enabled states
GET /v1/skills/store/{skill_name}/dependencies: reports the manifest-declared dependencies and their observed installed, missing, or not-applicable state
POST /v1/skills/store/install: installs and enables an optional catalog skill, then reloads runtime skill views
POST /v1/skills/store/remove: removes an optional skill from runtime and planner visibility while retaining its bundled or imported package for reinstallation; always-on core and tool skills reject this action

Machine-local API example (8787 must not be exposed or port-forwarded):

curl http://127.0.0.1:8787/v1/health \
  -H "X-Agent-Key: rk-xxxx"

curl -X POST http://127.0.0.1:8787/v1/tasks \
  -H "Content-Type: application/json" \
  -H "X-Agent-Key: rk-xxxx" \
  -d '{"user_id":1,"chat_id":1,"user_key":"rk-xxxx","channel":"ui","external_user_id":"local-ui","external_chat_id":"local-ui","kind":"ask","payload":{"text":"hello"}}'

Model Capability Catalog and Chinese Provider Validation

What the catalog answers

The model capability catalog is configuration-derived machine truth, not a runtime guess based on model names. It combines the LLM provider tables in configs/config.toml with the independently selected modules in configs/image.toml, configs/audio.toml, configs/video.toml, and configs/music.toml. Secret-free entries expose provider/model identity, configured choices, input/output modalities, generation and understanding capabilities, async and dry-run requirements, timeout, context window, active text-provider state, and configuration sources. credential_state is one of configured_inline, configured_env, not_required_local, or missing; it never contains the credential value.

clawcli models catalog emits model_catalog_summary and model_catalog_entry machine lines. clawcli models readiness, task teaching traces, and clawcli llm-trace project the same selected entry through model_readiness_summary or model_catalog_trace.readiness, so callers do not need to parse prose or inspect provider logs.

See Skills, media, and models and Release validation for the full catalog, readiness, and provider-validation flow.

Hosted relay preset

The model settings page exposes a managed OpenAI-compatible relay preset in addition to direct provider configuration. Selecting the preset configures the existing custom provider with public model alias minimax; it does not add a second provider adapter and never silently changes an existing installation. The relay maintains its own Slot 0 public-key allowlist. On first use, an allowlisted physical or simulated signing device signs one short-lived challenge; the relay then returns a device-specific access key once. clawd stores it in the machine-local private credential broker rather than tracked TOML or browser storage. Normal model calls use the access key and do not sign each request.

flowchart LR
    UI[Model settings] -->|select managed preset| CL[clawd]
    CL -->|first use: Slot 0 signed challenge| TLS[llm.matrixai.one TLS endpoint]
    TLS -->|one-time relay key response| CL
    CL -->|later calls: bearer relay key| TLS
    TLS --> RELAY[standalone LLM relay]
    RELAY --> QUOTA[per-device authentication and UTC quota]
    RELAY -->|server-held provider credential| UPSTREAM[configured upstream model]

The initial relay policy permits 100 upstream model attempts per allowlisted Slot 0 device per UTC day. Local validation failures, model-list reads, and quota reads do not consume that allowance. Once an authenticated request is dispatched upstream, successes and failures both count. The quota database and service credential remain on the relay host; prompts, responses, tool arguments, and raw keys are not exposed by the quota API.

How providers are validated

scripts/check_chinese_model_catalog.py guards MiniMax M3/M2.7, MiMo, Qwen, and DeepSeek metadata. Its self-test covers missing or unreadable TOML/env files, invalid UTF-8, syntax errors, and other structured findings before the release gate trusts configuration-derived metadata. Run scripts/nl_tests/run_chinese_provider_smoke_matrix.sh --dry-run to validate cases and credential state without contacting providers. A live run must use a clawd process started with the matching provider configuration; use --live-providers <machine-token-csv> to declare the account scope explicitly.

How the release gate proves the result

The agent parity gate records the catalog, provider-smoke preflight, structured evidence, permission tokens, registry policy, long-tail async contracts, and hard-match/fixed-reply checks as portable artifacts. Paths remain repository-relative or artifact-relative, and the reports never record credential values, env-file locations, or machine-specific absolute paths. This makes provider readiness and the absence of new runtime hard replies independently auditable.

NL Regression Shortcuts

Choose the smallest useful scope

Use the smallest affected NL set while code is still moving, then widen coverage only at phase or release gates:

Static compact coverage: python3 scripts/nl_tests/check_compact_coverage.py --report verifies that the compact source-controlled case files cover basic skills, route/lifecycle classes, and media dry-run cases without calling a provider. The compact gate also requires Codex-style agent parity tags for coding, continuous development, shell/git/config/DB/web/KB, async, permission, subagent, memory, multilingual behavior, and failure recovery.
Focused affected suite: 10-30 hand-picked cases for the code path being changed.
Typical aggregate: compressed representative coverage after a phase batch.
Canary: 500 client-like cases for high-risk runtime boundary changes.
Safe aggregate: compact equivalent coverage first, then full 2100+ coverage for release hardening when the affected surface justifies it.

Live NL runs should use bash scripts/nl_tests/run_all_nl_with_server.sh. It creates a random loopback listener, isolated task/audit databases, and a non-delivering ui channel by default, then removes that temporary state after the run. Reusing a development server is opt-in with --reuse-server. Use --suite <name> or --category <name> for a focused scope; numbered raw LLM#1..N request/return fields stay enabled unless explicitly disabled.

Current configs/agent_guard.toml keeps verifier and registry guards enabled, including answer_verifier_enforce_required_scope = "all" and registry_idempotency_guard_scope = "all". When a runtime boundary changes, run the boundary guards and update replay and README flow descriptions together.

Understand the release evidence

The agent parity gate writes portable release evidence rather than trusting a single successful command. agent_loop_static_contracts.txt contains the self-tested planner-authority, NL hard-match, hardcoded-language, and front door boundary guards. In particular, scripts/check_frontdoor_boundary_dispatch.py --self-test and its main check must produce FRONTDOOR_BOUNDARY_DISPATCH_CHECK findings=0, proving that the ask front door only prepares the turn boundary and does not decide whether an ordinary request should answer, clarify, or execute.

The same gate writes planner_runtime_boundary_contracts.txt and requires PLANNER_RUNTIME_BOUNDARY_CHECK findings=0, CONTRACT_REPAIR_LOOP_OBSERVATION_BOUNDARY findings=0, ROUTE_REASON_MARKER_FACADE_SELF_TEST ok, and FINALIZER_ARCHITECTURE_SELF_TEST ok. These checks keep semantic decisions in the planner-owned loop, repair on structured loop observations, route reasons as machine markers, and final delivery model-led rather than domain-template driven. The related artifacts are runtime_hard_reply_baseline.txt, policy_boundary_hard_reply.txt, repair_no_user_text_fields.txt, policy_decision_tokens.txt, agent_loop_guard_final_scope.txt, registry_policy_contracts.txt, skill_registry_aliases.txt, long_tail_skill_contracts.txt, no_agent_mode_payload.txt, and evidence_extractor_contracts.txt. The evidence extractor is itself verified with check_evidence_extractor_contracts.py --self-test.

Task continuity and operator surfaces are also release-gated: task_lifecycle_contracts.txt covers background execution, checkpoint/resume, async polling, and cancellation, while task_event_context_team_contracts.txt covers goals, context compaction, coding evidence, and child-task lifecycle events. Wrapped runs validate suite_artifact_contract.json, suite_artifact_contract_self_test.txt, chinese_model_catalog_self_test.txt, runner_path_ref_contract.json, and llm_raw_trace_runner_contract.txt. The nested report must record agent_parity_gate_contract.checked=true; report generation uses --validate-contract-report-content and --require-contract-report-content-checked, and records contract_report_content_checked=true.

Gate summaries use portable out_dir_ref, run_dir_ref, and run_log_ref values. live_metrics=0|1 distinguishes whether a live run directory was supplied: metrics=1 means the metrics gate is enabled, while live_metrics=1 proves that live rollout metrics were generated and validated. NL runners retain the task id and model-I/O log offset so llm_raw_trace_runner_contract.txt can verify numbered LLM#1..N raw request and response fields without exposing secrets.

The agent parity gate writes clawcli_exec_replay_contracts.txt and records clawcli_exec_replay_contracts=1. The artifact must contain CLAWCLI_EXEC_REPLAY_CONTRACT_SELF_TEST ok and CLAWCLI_EXEC_REPLAY_CONTRACT_CHECK findings=0, proving that clawcli exec artifacts and recorded_only replay coverage/view/diff behavior remain machine-field contracts in the release gate.

The same gate writes clawcli_session_tui_contracts.txt and records clawcli_session_tui_contracts=1. The artifact is produced by scripts/check_clawcli_session_tui_contracts.py --self-test plus its main check and must contain CLAWCLI_SESSION_TUI_CONTRACT_SELF_TEST ok and CLAWCLI_SESSION_TUI_CONTRACT_CHECK findings=0. It release-gates clawcli session list/show/resume/archive/delete/fork, local session-store metadata, the clawcli tui selected-task snapshot, selected_progress, selected_summary, operator key tokens, and TUI report/review/subagents/ permission projections as machine-field contracts.

The same gate writes clawcli_goal_contracts.txt and records clawcli_goal_contracts=1. The artifact comes from scripts/check_clawcli_goal_contracts.py --self-test plus its main check and must contain CLAWCLI_GOAL_CONTRACT_SELF_TEST ok and CLAWCLI_GOAL_CONTRACT_CHECK findings=0. It keeps clawcli goal start/status/pause/resume/edit/clear, done conditions, verification commands, constraints, checkpoint/resume control summaries, and sensitive-field redaction on structured machine contracts.

The same gate writes clawcli_llm_trace_contracts.txt and records clawcli_llm_trace_contracts=1. The artifact comes from scripts/check_clawcli_llm_trace_contracts.py --self-test plus its main check and must contain CLAWCLI_LLM_TRACE_CONTRACT_SELF_TEST ok and CLAWCLI_LLM_TRACE_CONTRACT_CHECK findings=0. It release-gates clawcli llm-trace, llm_call_ref=LLM#1..N, flow/code attribution, provider/model/status/usage tokens, raw request/response fields, llm_trace_model_readiness, and the UI teaching-trace helpers.

The same gate writes clawcli_models_catalog_contracts.txt and records clawcli_models_catalog_contracts=1. The artifact comes from scripts/check_clawcli_models_catalog_contracts.py --self-test plus its main check and must contain CLAWCLI_MODELS_CATALOG_CONTRACT_SELF_TEST ok and CLAWCLI_MODELS_CATALOG_CONTRACT_CHECK findings=0. It keeps clawcli models catalog, model_catalog_summary, model_catalog_entry, provider filtering, credential_state, modalities, capability flags, async/dry-run metadata, and the UI model catalog on secret-free machine fields.

The same gate writes clawcli_models_readiness_contracts.txt and records clawcli_models_readiness_contracts=1. The artifact comes from scripts/check_clawcli_models_readiness_contracts.py --self-test plus its main check and must contain CLAWCLI_MODELS_READINESS_CONTRACT_SELF_TEST ok and CLAWCLI_MODELS_READINESS_CONTRACT_CHECK findings=0. It keeps clawcli models readiness, clawcli llm-trace, model_readiness_summary, model_catalog_trace.readiness, selected provider/model matching, selected_entry_status, readiness/capability flags, and missing-selection behavior on secret-free machine contracts.

Current release acceptance combines compact live NL for the affected class, release-gate equivalent coverage (scripts/nl_tests/build_release_gate_subset.py --check selects the maintained representative set), loop-boundary/replay review with no unexplained mismatch, and planner/runtime/repair/static guards. The subset generator treats shared compact contracts, release behavior tags, machine capability/action-family tags, and suite breadth as mandatory. Planner and output-contract work runs python3 scripts/check_planner_runtime_boundary.py, python3 scripts/check_route_reason_marker_facade.py, and python3 scripts/check_finalizer_architecture.py; repair work runs python3 scripts/check_repair_boundary_inventory_coverage.py and python3 scripts/check_repair_no_user_text_fields.py.

Focused long-tail closed-loop entries:

bash scripts/nl_tests/run_suite.sh ops_closed_loop
bash scripts/nl_tests/run_suite.sh long_tail_flows
bash scripts/nl_tests/run_suite.sh ops_http_repair
bash scripts/clawcli_smoke.sh: compact CLI operator smoke for health, skills, submit, get, events, and watch. It uses APP_CLI_SMOKE_KEY / APP_ADMIN_KEY when provided, otherwise clawcli falls back to the local enabled admin key; optional env vars enable active/cancel/pause/resume/run-skill coverage. Set APP_CLI_SMOKE_REQUIRE_CAPABILITIES=1 when the smoke must fail if /v1/capabilities is unavailable. New CLI-only logic is also covered by cargo test -p clawcli.

ops_http_repair is the focused bilingual retry suite for ops_http_repair_then_validate_{zh,en} and writes logs under scripts/nl_suite_logs/ops_http_repair/<timestamp>/.

Validate UI delivery

UI notes:

source lives in UI/
built assets live in UI/dist
build-ui-nginx.sh builds UI/dist by default; pass --deploy only for an explicit nginx deployment
build-ui-nginx.sh --deploy-if-configured updates nginx only when an Agent Runtime nginx site already exists, so local updates never touch system configuration
the full build-all.sh flow uses that conditional deployment mode: without nginx it only refreshes UI/dist; with nginx already deployed it syncs the latest UI to the site's existing root
deploy-ui-nginx.sh is the "deploy existing UI/dist" path, with optional --build
install-agent-cmd.sh defaults to a local no-nginx install; pass --deploy-ui-nginx for a cloud/server deployment
the dashboard checks source and compatible GitHub Release versions when an admin opens it; it shows the running package version and latest platform-specific Release tag
packaged Release installations check and show only Release updates: they do not run Git commands or show source-build controls. An admin can explicitly choose Switch to source mode, which clones and validates the complete repository, migrates persistent runtime state, and enables Git pull/build controls only after a successful restart
the dashboard system-information section shows OS/version, architecture, memory, system storage, deployment type, and uptime without exposing host paths or environment values; missing Linux/macOS facts remain partial data rather than breaking the page
build progress is only shown for a real build/deploy session, and full build/deploy, UI-only, and clawd-only modes refresh the page once after successful completion
the login page and navigation show the UI build version, making it easy to compare an nginx-hosted page with the page served directly by webd on port 8788
the Models page keeps text-model selection separate from seven multimodal modules: image editing, image generation, image understanding, speech synthesis, speech transcription, video generation, and music generation. Each module has its own provider/model settings and its own enable switch; disabling a module preserves those settings
工具/技能 / Tools/Skills manages switches for installed skills; the adjacent Skill Store page owns optional-skill install, remove, reinstall, configuration retention, and third-party import flows
Skill Store install details show every declared dependency as a checked or missing item from backend-observed state before the user installs or repairs a skill
service-control notices are rendered from backend machine codes (error_code / message_key) instead of parsing backend English strings
webd is the only browser-facing Agent Runtime service; nginx is an optional outer TLS/static layer, and clawd remains loopback-only

Skills

Admission, hot plug, update, and removal

Repository-maintained core and bundled skills come from the read-only base registry. Runtime imports go through one SkillAdmissionService; UI, CLI, and the extension manager do not edit tracked registry/config/prompt files. Admission validates the typed manifest, builds through its declared adapter, runs the JSONL or typed-HTTP protocol smoke, verifies an immutable package receipt, applies a host-owned policy grant, and atomically publishes a data-root overlay generation.

flowchart LR
    A[Package + capability request] --> B[Validate + isolated adapter build]
    B --> C[Protocol smoke + immutable receipt]
    C --> D[Host policy grant]
    D --> E[Atomic overlay generation]
    E --> F[Planner catalog + resolver + verifier]
    F --> G[Pinned version/receipt/policy execution]
    G --> H[Structured result + evidence]
    E --> I[Update publishes next generation]
    I --> J[Old version lease drains]
    J --> K[Version GC]
    E --> L[Disable/revoke/tombstone]
    L --> M[Block new calls]
    M --> J

External packages begin disabled or awaiting approval. Enabling requires an explicit host grant; disabling or revoking a grant blocks new calls immediately, while an in-flight call finishes under its pinned generation and version lease. Uninstall first tombstones the skill, drains leases, and removes only skill-owned packages. Private skill data is retained by default, and shared Cargo/Python/Node/Go toolchains and caches are never removed.

Agent Runtime currently ships a broad skill set. Representative groups:

system and ops: system_basic, process_basic, service_control, health_check, log_analyze, task_control
runtime and workspace primitives: run_cmd, read_file, write_file, list_dir, make_dir, remove_file, workspace_patch, task_plan, subagent
files, config, and developer tools: fs_basic, code_index, config_basic, config_edit, config_guard, archive_basic, fs_search, git_basic, package_manager, install_module, docker_basic, db_basic
network and content: http_basic, rss_fetch, browser_web, browser_session, media_download, web_search_extract
documents and office work: doc_parse, office_workspace, transform
multimodal and media generation: image_generate (image.preview_generate / image.generate / image.poll / image.cancel), image_edit, image_vision, audio_transcribe (audio.preview_transcribe / audio.transcribe), audio_synthesize (audio.preview_synthesize / audio.synthesize / audio.poll / audio.cancel), video_generate (video.preview_generate / video.generate / video.poll / video.cancel), music_generate (music.preview_generate / music.generate / music.poll / music.cancel)
workflow and publishing: schedule, extension_manager, photo_organize, invest_copy, x
domain and knowledge skills: crypto, stock, weather, chinese_almanac, map_merchant, kb

Use browser_web to extract bounded evidence from exact public URLs. Use the separate task-scoped browser_session only when the task must navigate, click, type, select, download, or verify a page transition; its element references are tied to the current snapshot generation and mutating interactions remain policy/confirmation gated.

media_download returns original media by default. Douyin and Xiaohongshu image-article posts also include verified platform text; up to nine images are delivered individually and larger sets use one source-ordered ZIP. OCR or speech transcription runs only when the user explicitly asks to convert the media to text. Image recognition prefers the configured image_vision model and may use local Tesseract as an explicit fallback; audio/video uses speech transcription.

Skill installation and enablement are separate states. skill_switches=false keeps an installed skill available in the normal inventory but disables it. uninstalled_skills removes an optional skill from runtime, planner visibility, and the normal Tools/Skills inventory while keeping it discoverable in Skill Store. Bundled entries marked install_mode="on_demand" are excluded from the normal build-all.sh release build. The current on-demand set is chinese_almanac, crypto, invest_copy, map_merchant, media_download, photo_organize, stock, weather, and x; clicking Install reads that skill's skill.toml, runs only its declared adapter, performs a protocol smoke test, writes a verified receipt, and only then enables/reloads it. Normal source, cross-target, Docker, and release-package flows use the registry's supported_os declarations to build all core/runtime tools plus only the fixed runner packages supported by the target platform. An on-demand install is rejected before installation when the current platform is unsupported; package manifests come from the registry rather than script-local skill maps. Removing one of these skills deletes its versioned runtime/receipts and asks whether its registry-declared dedicated configuration files should be preserved. Reinstalling never overwrites configuration files that still exist. Core skills and registry entries whose planner_kind=tool are always available and cannot be removed through Skill Store. Third-party import requires skill.toml plus INTERFACE.md, installs through the same adapter/receipt boundary, clears stale uninstall state, and only then exposes the verified package in Skill Store and Tools/Skills.

The implementation flow is language-neutral:

skill.toml -> build adapter -> install receipt -> SkillLaunchSpec -> JSONL capability result

Packages may request run.execution_profile="stateless_readonly" to let the host reuse only the outer runner process. The host still verifies read-only effects and effective permissions for every call; storage, credentials, network, writes, subprocesses, policy changes, and low-memory conditions keep the normal per-request lifecycle.

See Polyglot Skill SDK contract for Rust, Python, Node, Go, prebuilt, lifecycle, security, and publishing guidance.

If you need to answer “how is this skill configured / bound / enabled, and what prerequisite is missing”, start with prompts/references/skill_setup_guide.md.

Skill discovery and runtime behavior are driven by:

configs/skills_registry.toml
[skills] in configs/config.toml
crates/skills/*/INTERFACE.md
optional_skills/*/INTERFACE.md
external_skills/*/INTERFACE.md
prompts/layers/generated/skills/*.md

Planner skill selection is registry-, capability-, and interface-driven. After a skill is registered, enabled, documented in INTERFACE.md, synced with python3 scripts/sync_skill_docs.py, and, when planner-facing, given planner_capabilities in configs/skills_registry.toml, the planner should learn when to use it from registry metadata plus the generated skill prompt. Do not add per-skill selection branches to clawd just to make new natural-language examples pass. If selection accuracy is weak, improve the registry capability metadata, skill interface, generated prompt, or model-specific vendor patch; keep Rust code for protocol validation, resolver/verifier boundaries, permission/safety checks, runner dispatch, output-contract enforcement, and deterministic execution compatibility.

Skill integration entry points:

built-in and standard runner skills: skill_develop/README.md
external skill example: external_skills/example/README.md
skill setup and prerequisite reference: prompts/references/skill_setup_guide.md

Speech-To-Text Backends

The default audio_transcribe path uses local whisper.cpp. When its binary and multilingual model are present, Agent Runtime starts the loopback model server with the rest of the runtime and stops it during normal shutdown. Qwen qwen3-asr-flash is also supported through /chat/completions input_audio; select it in configs/audio.toml when a working Qwen account is preferred.

Local whisper.cpp

audio_transcribe can use a local whisper.cpp server through the custom OpenAI-compatible provider. Use a dedicated local port such as 8178 so it does not collide with clawd or UI ports.

Download a multilingual model into the gitignored local model directory. The script picks tiny / base / small / medium from detected device memory, and large-v3 is available only when explicitly requested with --model large-v3.

bash scripts/download-whisper-model.sh
component_start/start-whisper-server.sh --check

The check command validates the configured binary and model without starting the server. Normal start-all-bin.sh and component_start/start-clawd.sh startup perform the actual launch.

Use a multilingual Whisper model for Chinese, for example ggml-small.bin, ggml-medium.bin, or ggml-large-v3.bin; avoid English-only .en models for Chinese audio.

[audio_transcribe]
default_vendor = "custom"
adapter_mode = "compat"
allow_compat_adapters = true
default_model = "local-whisper"
custom_models = ["local-whisper", "whisper-1"]

[audio_transcribe.providers.custom]
base_url = "http://127.0.0.1:8178/v1"
api_key = ""
model = "local-whisper"
timeout_seconds = 120

The empty api_key is accepted only for loopback custom providers (localhost, 127.0.0.1, ::1). Remote custom providers still require a real key.

Directory Guide

configs/: runtime, channel, model, memory, and skill configuration
crates/: Rust services, daemons, CLI, and skills
optional_skills/: bundled Skill Store packages installed on demand
external_skills/: externally submitted skills and example scaffolds
prompts/: prompt layers and generated skill prompt files
scripts/: setup, regression, maintenance, and skill-call helpers
services/: non-Rust helper services such as the WhatsApp Web bridge
UI/: browser UI project
pi_app/: desktop small-screen app
docker/: docker-oriented configs and entrypoint files
systemd/: service templates

Pi App

The small-screen desktop app lives in pi_app/.

cd pi_app && ./run-small-screen.sh
cd pi_app && ./install-desktop.sh
cd pi_app && ./enable-autostart.sh
cd pi_app && ./open-small-screen.sh

It reads health status from clawd, so start the backend first.

Developer Notes

build-all.sh is the most accurate repo-level build entry if you are building from source
Native Linux builds use the active Rust toolchain's bundled LLD. ARM and memory-constrained hosts use one Cargo job; an x86 host with 12-16 GiB total memory, at least 8 GiB currently available, and at least four CPUs uses two jobs. Larger hosts keep Cargo's own default. Repeated local builds keep incremental compilation enabled. Set CARGO_BUILD_JOBS / CARGO_INCREMENTAL explicitly to override these defaults, or APP_DISABLE_BUNDLED_LLD=1 to diagnose linker compatibility.
install-agent-cmd.sh is the most convenient operator-facing entry because it can handle both launcher installation and optional UI/nginx deployment
the installer verifies Python 3.11+ with tomllib; on macOS it installs the current Homebrew python formula when missing, and runtime/build entrypoints select that interpreter without replacing the chosen Rust toolchain
if you only want to rebuild the local UI, use build-ui-nginx.sh; use deploy-ui-nginx.sh only for an nginx-hosted server
if you are integrating skills, run python3 scripts/sync_skill_docs.py explicitly; startup scripts no longer sync skill docs for you
many helper and regression scripts live in scripts/
for the local ops_closed_loop regression stack, run bash scripts/regression_ops_closed_loop.sh

License

This project uses a non-commercial source-available license.

English legal text: LICENSE
Chinese reference translation: LICENSE.zh-CN.md

Name		Name	Last commit message	Last commit date
.cargo		.cargo
.github/workflows		.github/workflows
UI		UI
component_start		component_start
configs		configs
crates		crates
docker		docker
docs		docs
external_skills		external_skills
image		image
llm-relay-server		llm-relay-server
migrations		migrations
optional_skills		optional_skills
patches/open-lark		patches/open-lark
pi_app		pi_app
plan/archived_completed_20260807		plan/archived_completed_20260807
prompts		prompts
scripts		scripts
services/wa-web-bridge		services/wa-web-bridge
skill_develop		skill_develop
.codex		.codex
.dockerignore		.dockerignore
.gitignore		.gitignore
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
LICENSE		LICENSE
LICENSE.zh-CN.md		LICENSE.zh-CN.md
README.md		README.md
README.zh-CN.md		README.zh-CN.md
README_cn.md		README_cn.md
USAGE.md		USAGE.md
agentctl		agentctl
app-illustration.png		app-illustration.png
app-splash-480x320.png		app-splash-480x320.png
build-all.sh		build-all.sh
build-ui-nginx.sh		build-ui-nginx.sh
check-logs.sh		check-logs.sh
deploy-github-release.sh		deploy-github-release.sh
deploy-pi-nginx.sh		deploy-pi-nginx.sh
deploy-ui-nginx.sh		deploy-ui-nginx.sh
docker-compose.yml		docker-compose.yml
failed-routing-cases.md		failed-routing-cases.md
git-add-exclude.sh		git-add-exclude.sh
install-agent-cmd.sh		install-agent-cmd.sh
longxia.png		longxia.png
pack-to-parent.sh		pack-to-parent.sh
package-lock.json		package-lock.json
package-release.sh		package-release.sh
package.json		package.json
push-sanitized-configs.sh		push-sanitized-configs.sh
release-latest.sh		release-latest.sh
setup-config.sh		setup-config.sh
simulate-telegramd.sh		simulate-telegramd.sh
start-all-bin.sh		start-all-bin.sh
start-all.sh		start-all.sh
stop-agent.sh		stop-agent.sh
system_report.sh		system_report.sh

Latest commit

History

Folders and files

Repository files navigation