Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ All notable changes to NanoClaw will be documented in this file.

## [Unreleased]

- **Claude tool allowlist reconciled with the pinned CLI, plus a tool-surface drift guard.** `TOOL_ALLOWLIST` in the agent runner named five tools that don't exist on claude-code 2.1.197 (`Task` — renamed `Agent` upstream — plus `TodoWrite`, `TeamCreate`, `TeamDelete`, `ToolSearch`); the phantom entries are removed and the comment corrected: `allowedTools` is a permission auto-approve list, **not an availability filter** — interleaved wire captures show no allowlist effect on the stable tool surface, and `bypassPermissions` moots its permission role. (The surface itself shows per-query variance in conditional tools on this CLI pin — the drift fixture records the flicker set separately so regeneration cannot flake on it — so neither `allowedTools` nor `disallowedTools` alone should be relied on to shape what the model sees — the runner's PreToolUse hook is the deterministic block.) No wire-visible behavior change. A new wire-captured fixture (`container/agent-runner/src/providers/sdk-tools-baseline.json`) and `claude.tools.test.ts` now fail on any future claude-code CLI **or SDK** pin bump until the fixture is regenerated with `dump-sdk-tools.ts` and the lists re-verified.
- [BREAKING] **`whatsapp-formatting` and `slack-formatting` container skills moved from trunk to the `channels` branch.** They now install with their channel — `/add-whatsapp` / `/add-slack` and the setup installers copy them in — so installs without those channels stop carrying channel-specific formatting instructions in every agent's context. **Migration — only if this install has the channel wired** (check `src/channels/whatsapp.ts` / `src/channels/slack.ts`): updating removes both skills from the working tree — WhatsApp agents lose the formatting fragment from their composed CLAUDE.md on next spawn, Slack agents lose the mrkdwn skill from `~/.claude/skills`. Re-run the matching skill, `/add-whatsapp` or `/add-slack` (idempotent), to restore. Installs without the channel need nothing — do NOT run the add-skill just in case; it installs the full channel adapter.
- **Pre-task script failures back their series off instead of spinning.** A `--script` that errors lands the occurrence as a failed run (`script-skip:error` ack → `failed` status); recurrence reads the series' trailing failed streak and re-arms at `max(cron next, now + 2·2^(n−1) min, cap 60)`; after 8 consecutive failures the series is auto-paused with a host-written note in its run log (`ncl tasks resume` revives it). A deliberate `wakeAgent:false` gate is a normal run and never backs off. Also fixed: an explicitly-addressed `<message to>` in a task fire's final text now delivers as a deliberate send (previously suppressed as a turn-reply echo → zero delivery when the agent skipped the MCP tool); identical echoes of an MCP send are dropped in the runner, where the duplication originates.
- [BREAKING] **Scheduled tasks moved from MCP tools to `ncl tasks`.** The six scheduling MCP tools are no longer exposed to agent containers; agents and operators manage tasks with `ncl tasks list/get/create/update/cancel/pause/resume/delete`. New tasks run from a per-agent-group system session rather than waking the chat session that created them, and task writes are not approval-gated inside the owning group. **Migration:** [docs/ncl-tasks-migration.md](docs/ncl-tasks-migration.md).
Expand Down
2 changes: 1 addition & 1 deletion container/agent-runner/bun.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion container/agent-runner/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
"test": "bun test"
},
"dependencies": {
"@anthropic-ai/claude-agent-sdk": "^0.3.197",
"@anthropic-ai/claude-agent-sdk": "0.3.197",
"@anthropic-ai/sdk": "^0.108.0",
"@modelcontextprotocol/sdk": "^1.29.0",
"cron-parser": "^5.0.0",
Expand Down
82 changes: 82 additions & 0 deletions container/agent-runner/src/providers/claude.tools.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
/**
* Drift guard for the harness tool surface. sdk-tools-baseline.json is a wire
* capture of every tool the pinned CLI can offer under our configuration
* (regenerate with dump-sdk-tools.ts — instructions in its header). These
* tests catch upstream renames/removals when the claude-code pin moves:
* bumping container/cli-tools.json fails the version assertion until the
* fixture is regenerated and the lists below are re-verified.
*/
import fs from 'fs';

import { describe, expect, it } from 'bun:test';

import cliTools from '../../../cli-tools.json';
import { SDK_DISALLOWED_TOOLS, TOOL_ALLOWLIST } from './claude.js';
import baseline from './sdk-tools-baseline.json';

/**
* Disallow entries that do NOT exist on the pinned CLI in headless SDK mode
* (wire-verified: never offered, in both string and streaming input modes).
* Kept in SDK_DISALLOWED_TOOLS as drift insurance — if an upstream version
* starts offering one, the fixture regeneration surfaces it here and the
* entry moves out of this set.
*/
const KNOWN_ABSENT_DISALLOWED = ['AskUserQuestion', 'EnterPlanMode', 'ExitPlanMode'];

const installedSdkVersion = (
JSON.parse(
fs.readFileSync(new URL('../../node_modules/@anthropic-ai/claude-agent-sdk/package.json', import.meta.url), 'utf8'),
) as { version: string }
).version;

// Membership checks run against stable ∪ variant: a tool that flickers on
// this pin (see dump-sdk-tools.ts header) is still a real tool.
const baselineTools = new Set<string>([...baseline.tools, ...baseline.variantTools]);

describe('sdk tool-surface drift guard', () => {
it('fixture matches the pinned claude-code CLI version', () => {
const pin = cliTools.find((t) => t.name === '@anthropic-ai/claude-code')?.version;
expect(baseline.cliVersion).toBe(pin);
});

it('fixture matches the installed Agent SDK version', () => {
// The SDK is a caret-free pinned dep; a bump must be captured deliberately.
expect(baseline.sdkVersion).toBe(installedSdkVersion);
});

it('allowedTools has no surface effect: the stable cores match across interleaved rounds', () => {
// `tools`/`toolsBare` are STABLE cores over N interleaved captures per
// mode; per-query flicker (see the dump-sdk-tools.ts header) lands in
// `variantTools` instead of here. Passing TOOL_ALLOWLIST has never been
// observed to filter or promote a stable tool under bypassPermissions —
// if a future CLI makes allowedTools an availability filter, the cores
// diverge in every round and this fails deterministically.
expect([...baseline.tools].sort()).toEqual([...baseline.toolsBare].sort());
});

it('every allowlist entry names a real tool on this surface', () => {
for (const name of TOOL_ALLOWLIST) {
expect(baselineTools.has(name), `allowlist entry '${name}' not in captured surface`).toBe(true);
}
});

it('every disallow entry is either a real tool or documented drift insurance', () => {
for (const name of SDK_DISALLOWED_TOOLS) {
const real = baselineTools.has(name);
const insurance = KNOWN_ABSENT_DISALLOWED.includes(name);
expect(
real || insurance,
`disallow entry '${name}' is neither on the surface nor in KNOWN_ABSENT_DISALLOWED`,
).toBe(true);
}
});

it('drift-insurance entries are still absent from the surface', () => {
for (const name of KNOWN_ABSENT_DISALLOWED) {
expect(
baselineTools.has(name),
`'${name}' now exists on the surface — move it out of KNOWN_ABSENT_DISALLOWED and re-verify its disposition`,
).toBe(false);
}
});
});
41 changes: 22 additions & 19 deletions container/agent-runner/src/providers/claude.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ function log(msg: string): void {
// the question and blocks on the real reply.
// - EnterPlanMode / ExitPlanMode / EnterWorktree / ExitWorktree: Claude
// Code UI affordances; in a headless container they'd appear stuck.
const SDK_DISALLOWED_TOOLS = [
export const SDK_DISALLOWED_TOOLS = [
'CronCreate',
'CronDelete',
'CronList',
Expand All @@ -36,30 +36,33 @@ const SDK_DISALLOWED_TOOLS = [
'ExitWorktree',
];

// Tool allowlist for NanoClaw agent containers. MCP-tool entries are derived
// at the call site from the registered `mcpServers` map so that any server
// added via `add_mcp_server` (or wired in container.json directly) is
// reachable to the agent — without this, the SDK's allowedTools filter
// silently drops every MCP namespace not listed here.
const TOOL_ALLOWLIST = [
// Pre-approved tool set for NanoClaw agent containers. `allowedTools` is a
// permission auto-approve list, NOT an availability filter: interleaved wire
// captures show no allowlist effect on the stable tool surface (the fixture's
// stable cores are equal; claude.tools.test.ts asserts it), and its permission
// role is moot under this runner's `bypassPermissions`. The surface itself has
// per-query variance in conditional tools (see dump-sdk-tools.ts header), so
// never rely on this list — or on `disallowedTools` alone — to shape what the
// model sees. Kept as the accurate pre-approval set for a
// hypothetical non-bypass mode, and because the per-server `mcpAllowPattern`
// entries derived at the call site are retained (MCP invocation-gating under
// non-bypass modes is unverified). Exported for the fixture regenerator and
// tests.
export const TOOL_ALLOWLIST = [
'Agent',
'Bash',
'Read',
'Write',
'Edit',
'Glob',
'Grep',
'WebSearch',
'WebFetch',
'Task',
'TaskOutput',
'TaskStop',
'TeamCreate',
'TeamDelete',
'NotebookEdit',
'Read',
'SendMessage',
'TodoWrite',
'ToolSearch',
'Skill',
'NotebookEdit',
'TaskOutput',
'TaskStop',
'WebFetch',
'WebSearch',
'Write',
];

// MCP server names are sanitized by the SDK when forming tool prefixes:
Expand Down
182 changes: 182 additions & 0 deletions container/agent-runner/src/providers/dump-sdk-tools.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,182 @@
/**
* Regenerates sdk-tools-baseline.json — the bare SDK tool-surface fixture
* asserted by claude.tools.test.ts.
*
* Must run INSIDE the agent container image (the pinned CLI binary only
* exists there). From the repo root:
*
* docker run --rm --network none \
* -v "$PWD/container/agent-runner/src":/app/src:ro \
* --entrypoint bun <nanoclaw-agent image> /app/src/providers/dump-sdk-tools.ts \
* > container/agent-runner/src/providers/sdk-tools-baseline.json
*
* MEASURED NONDETERMINISM on the pinned CLI (2.1.197): across query
* invocations with byte-identical options — even inside one process — (a)
* conditional tools (Glob, Grep) flicker in and out of the surface, and (b)
* `disallowedTools` sometimes strips flag-gated tools (Workflow, DesignSync,
* EnterWorktree) and sometimes ignores them entirely; each single query is
* internally coherent. Consequences: schema-stripping via disallowedTools is
* BEST-EFFORT — the deterministic enforcement is the runner's PreToolUse
* hook, which blocks the invocation regardless of whether the schema shipped.
*
* Because the variance is per-query, a single capture pair cannot isolate the
* allowlist's effect. So: ROUNDS interleaved captures per mode (allowlist /
* bare), then split each mode into its STABLE core (tools present in every
* capture of that mode) and the VARIANT set (present in some but not all
* captures across both modes):
* - `tools` : stable core WITH the production TOOL_ALLOWLIST.
* - `toolsBare` : stable core with no allowedTools.
* - `variantTools` : the flicker set, recorded for membership checks.
* - `toolsDisallowProbe` : one capture with disallowedTools=[DISALLOW_PROBE]
* — a diagnostic of which disallow behavior this regen observed (no test
* asserts stripping; a fixed assertion on a nondeterministic mechanism
* would be a coin-flip).
* The drift test asserts tools === toolsBare on the STABLE cores: no allowlist
* effect has ever been observed there, so the list is permission-layer only
* (moot under bypassPermissions). If a CLI/SDK bump makes the allowlist shape
* the surface, the stable cores diverge across every round by construction
* and the assertion fails deterministically instead of flaking.
*
* Agent-teams is enabled via a temp settings.json (wire-verified: settings
* env strictly beats SDK options env).
*
* Zero API traffic: ANTHROPIC_BASE_URL points at an in-process stub answering
* 401; the full tools array rides on the first /v1/messages request, captured
* before the run dies on the auth error. The fixture records WIRE tool names
* (the SDK init message reports legacy aliases, e.g. `Task` for wire `Agent` —
* do not swap this to an init capture).
*/
import { execFileSync } from 'child_process';
import fs from 'fs';

import { query } from '@anthropic-ai/claude-agent-sdk';

import { TOOL_ALLOWLIST } from './claude.js';

let requests: string[] = [];
let captured: (() => void) | null = null;

const server = Bun.serve({
hostname: '127.0.0.1',
port: 0,
async fetch(req) {
const url = new URL(req.url);
const body = await req.text();
if (url.pathname.includes('/messages')) {
requests.push(body);
captured?.();
}
return new Response(
JSON.stringify({ type: 'error', error: { type: 'authentication_error', message: 'fixture-capture-stub' } }),
{ status: 401, headers: { 'content-type': 'application/json' } },
);
},
});

const HOME = '/tmp/dump-sdk-tools-home';
const CWD = '/tmp/dump-sdk-tools-ws';
fs.mkdirSync(`${HOME}/.claude`, { recursive: true });
fs.mkdirSync(CWD, { recursive: true });
fs.writeFileSync(
`${HOME}/.claude/settings.json`,
JSON.stringify({ env: { CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS: '1' } }, null, 2),
);

/**
* Diagnostic probe for disallowedTools: a flag-gated tool whose stripping is
* nondeterministic on the current pin (see header). The fixture records what
* this regen run observed; future pins can be compared against it.
*/
export const DISALLOW_PROBE = 'Workflow';

/** Run one capture and return the sorted wire tool names. */
async function capture(opts?: { allowedTools?: string[]; disallowedTools?: string[] }): Promise<string[]> {
requests = [];
const firstRequest = new Promise<void>((resolve) => {
captured = resolve;
});
const q = query({
prompt: 'fixture capture: reply with one word',
options: {
cwd: CWD,
pathToClaudeCodeExecutable: '/pnpm/claude',
systemPrompt: { type: 'preset' as const, preset: 'claude_code' as const },
env: {
...process.env,
HOME,
ANTHROPIC_BASE_URL: `http://127.0.0.1:${server.port}`,
ANTHROPIC_API_KEY: 'fixture-dummy-key',
ANTHROPIC_AUTH_TOKEN: undefined,
},
permissionMode: 'bypassPermissions',
allowDangerouslySkipPermissions: true,
settingSources: ['user'],
...(opts?.allowedTools ? { allowedTools: opts.allowedTools } : {}),
...(opts?.disallowedTools ? { disallowedTools: opts.disallowedTools } : {}),
},
});
void (async () => {
try {
for await (const _m of q) {
/* drain until the auth error kills the run */
}
} catch {
/* expected: 401 from the stub */
}
})();
await Promise.race([firstRequest, Bun.sleep(75_000)]);
await Bun.sleep(1_500); // let retries land so we can pick the largest body
if (requests.length === 0) {
console.error('[dump-sdk-tools] no /v1/messages request captured');
process.exit(1);
}
const biggest = requests.reduce((a, b) => (b.length > a.length ? b : a));
const parsed = JSON.parse(biggest) as { tools?: Array<{ name: string }> };
return [...new Set((parsed.tools ?? []).map((t) => t.name))].sort();
}

// Interleaved so a drifting gate state biases both modes equally (see header).
const ROUNDS = 3;
const withRuns: string[][] = [];
const bareRuns: string[][] = [];
for (let i = 0; i < ROUNDS; i++) {
withRuns.push(await capture({ allowedTools: TOOL_ALLOWLIST }));
bareRuns.push(await capture());
}
const toolsDisallowProbe = await capture({ disallowedTools: [DISALLOW_PROBE] });

const stable = (runs: string[][]): string[] => runs[0].filter((t) => runs.every((r) => r.includes(t))).sort();
const union = (runs: string[][]): Set<string> => new Set(runs.flat());

const tools = stable(withRuns);
const toolsBare = stable(bareRuns);
const stableSet = new Set([...tools, ...toolsBare]);
const variantTools = [...union([...withRuns, ...bareRuns])].filter((t) => !stableSet.has(t)).sort();

const cliVersionRaw = execFileSync('/pnpm/claude', ['--version'], { encoding: 'utf8' }).trim();
const cliVersion = cliVersionRaw.split(/\s+/)[0];
const sdkVersion = (
JSON.parse(fs.readFileSync('/app/node_modules/@anthropic-ai/claude-agent-sdk/package.json', 'utf8')) as {
version: string;
}
).version;

console.log(
JSON.stringify(
{
cliVersion,
sdkVersion,
capturedAt: new Date().toISOString(),
capture: `wire names; stable cores over ${ROUNDS} interleaved rounds per mode (tools=production allowlist, toolsBare=no allowedTools); variantTools=flicker set; toolsDisallowProbe=single capture with disallowedTools:[probe]; teams on`,
rounds: ROUNDS,
disallowProbe: DISALLOW_PROBE,
tools,
toolsBare,
variantTools,
toolsDisallowProbe,
},
null,
2,
),
);
process.exit(0);
Loading
Loading