before_agent_start: forced system prompt keeps tools that setActiveTools removed in the same chain

[isaacgarza]

What do you want to change?

When a before_agent_start handler returns systemPrompt (or sets forceSystemPrompt), the forced text is captured before the active tool set is reconciled. A tool that another before_agent_start handler removed via setActiveTools() therefore stays advertised in the prompt the provider receives, while being absent from the tool array.

The model reads the prompt, calls the tool, and gets Tool <name> not found.

Reproduction

One extension, no dependencies. ghost_tool is registered, then gated off in before_agent_start; a second handler appends to the prompt.

import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
import { Type } from "typebox";

export default function repro(pi: ExtensionAPI) {
  pi.registerTool({
    name: "ghost_tool",
    label: "Ghost",
    description: "A tool that should NOT be offered to the model.",
    promptSnippet: "This tool should never be advertised",
    parameters: Type.Object({}),
    async execute() {
      return { content: [{ type: "text", text: "ghost ran" }], details: {} };
    },
  });

  // Gate it off every turn.
  pi.on("before_agent_start", () => {
    const active = pi.getActiveTools?.() ?? [];
    pi.setActiveTools?.(active.filter((n) => n !== "ghost_tool"));
    return undefined;
  });

  // The trigger: any handler that returns a replacement prompt.
  pi.on("before_agent_start", (event) => ({
    systemPrompt: `${event.systemPrompt}\n\nAppended by another extension.`,
  }));
}

pi --print -ne -nc -ns -e ./repro.ts --no-session \
  "Look at your system prompt under 'Available tools:'. Is there a line starting with '- ghost_tool:' ? Reply with exactly YES or NO."

	0.84.4	0.87.1
as above	YES	YES
with the prompt-returning handler removed	NO	NO
second handler mutates systemPromptOptions.appendSystemPrompt instead	NO	NO

ghost_tool is correctly absent from the tool array in every case — only the prompt text disagrees. Removing the prompt-returning handler is necessary and sufficient, so the two handlers interacting is the trigger, not the gating alone.

Mechanism

In 0.87.1, emitBeforeAgentStart stores a returned prompt as currentOptions.forceSystemPrompt (packages/coding-agent/src/core/extensions/runner.ts, emitBeforeAgentStart). After the chain, agent-session.ts reconciles tools before building messages:

if (!handlerEditedTools)
    result.systemPromptOptions.selectedTools = this.getActiveToolNames();

That refresh is real — it is what keeps the tool array correct. But buildSystemPromptState (packages/coding-agent/src/core/system-prompt.ts) short-circuits on the forced text:

export function buildSystemPromptState(input) {
    if (input.forceSystemPrompt !== undefined)
        return { content: input.forceSystemPrompt };
    return { content: "", sections: buildSystemPromptSections(input) };
}

So selectedTools never reaches the rendered prompt when a prompt was forced. The refreshed value is used for the tool array and discarded for the text, which is exactly how the two drift apart.

0.84.4 reaches the same end state by a different route: the prompt is snapshotted into emitBeforeAgentStart before handlers run, and a handler's returned prompt becomes _systemPromptOverride, pinning that pre-gating snapshot. The 0.87.x restructure fixed the tool array but the forced-text path kept the symptom.

Why

Handler order cannot avoid it. The forced text is whatever the prompt looked like when that handler ran, so a gating handler that runs earlier is still discarded, and one that runs later cannot affect text already captured. Load order is readdir order, so neither extension author can see the coupling: each works correctly alone.

The failure is silent and lands on the model. In our case a router extension gated a consult tool off when its mode was inactive, and an unrelated extension appended an untrusted-data warning on every turn. The model was offered consult, called it, got Tool consult not found, and substituted a nested subagent that wedged until the parent timed out — a 45-minute no-op run from two individually correct extensions.

The docs recommend mutating systemPromptOptions over returning systemPrompt (packages/coding-agent/docs/extensions.md), and that path is unaffected — but returning systemPrompt is still documented and supported, and forceSystemPrompt is the documented way to own the leading system prompt.

Possible fixes

• Re-render the forced prompt after the tool reconciliation, so forceSystemPrompt carries the settled tool list.
• Or have handlers receive a prompt built from the live tool set at call time, rather than a value captured before reconciliation.
• Or, at minimum, warn when selectedTools changes during before_agent_start while forceSystemPrompt is set — today the mismatch is completely silent until a model calls the missing tool.

Version

Reproduced on 0.84.4 and 0.87.1 (latest at time of writing), macOS, node 26.9.0.

Line references above are from the published 0.87.1 dist; source paths given where known.

[github-actions]

This issue was auto-closed. All issues from new contributors are auto-closed by default.

Maintainers review auto-closed issues daily and reopen worthwhile ones. Issues that do not meet the quality bar in CONTRIBUTING.md will not be reopened or receive a reply.

If a maintainer replies lgtmi on one of your issues, your future issues will stay open. If a maintainer replies lgtm, your future issues and PRs will stay open. The command must be at the start of the reply (optionally after one or more @username mentions) or at the end.

See CONTRIBUTING.md.

[davidbrai]

want to better understand this

[holny]

root cause: emitBeforeAgentStart passes currentSystemPrompt (captured before tool reconciliation) to each handler, so when a handler calls setActiveTools and another handler returns systemPrompt, the returned prompt text was built from the pre-reconciliation tool list.

fix approach: after before_agent_start handlers complete, re-render the returned systemPrompt using the post-reconciliation selectedTools from systemPromptOptions, or pass the live tool state to getSystemPrompt() inside handlers so returned prompts carry the correct tool list.

self-nomination: I will implement the fix.

AI-assisted content (per CONTRIBUTING.md): drafted by my local agent, reviewed and posted by me.

[IgorWarzocha]

Before_agent_start is my nemesis and I would love something to be rewired.

I would love for it to run in some capacity before anything that happens in Pi. Like there is no real reason (I can think of) that purple-box system messages or extension-created events do not run it.

I am doing a lot of hacking around Pi's extensions api so might be very niche, but three most recent examples:

a. an extension that kicks off a herdr pane and delivers a prompt as a developer role (hacked into Pi. Non responses models just see the normal Pi system message). But if this triggers the agent run, before agent start never runs, so it needs to be non-turn-triggering, and followed by "continue" sent as user message
b. OAI's ctx management system, when agent requests a new ctx window, since the new ctx window would be continued via an extension triggered event, it also needs to be non turn triggering and instead followed by a "continue" user message.
c. a subagents /review extension via pi rpc subprocess - I had some system messages around that at some point, and after the subagent review, the clanka would just start investigating etc. Took me a while to realise that if /review is the first thing I run in a session, with no other previous user messages, none of before_agent_start would run, basically leaving the session in a semi-broken state since sysprompt/tools altered by extensions have never been accounted for properly.

I'm sure there is some sort of a high-level valid reason for this, but as it stands, it heavily limits using Pi+extensions for hands off automation work etc.

Only here to do a +1