Repository navigation
fix: say a refused tool is not available and list only offered tools - #1293
Merged
Merged
Conversation
A session's tools can change mid-conversation: an MCP server drops a tool or goes away, or another user sends on a shared workstream and the catalog follows the sender. The model can still call such a tool, from its history or from a replayed tool search, and dispatch refused it as "Unknown tool" with a request to use a listed name exactly, as if the model had made the name up. The list named every built-in preparer and the sender's MCP tools, not what the request offered: coordinator tools in a regular session, tools a persona hides or an operator revoked, and deferred tools left to tool search. The refusal now says the tool is not available now and gives the possible causes: removed since it was offered; available only to another user, once the workstream is shared; or misspelled, unless the name was offered exactly. It lists the tools the request offers outright, without revoked ones, and points to tool search when more can be found. A task agent's refusal lists the agent's own tools. The operator's error and the tool header say the same. _get_deferred_names, which the refusal and every request read, now reads the tool search manager once, as _get_active_tools does. A catalog refresh on the MCP thread could drop it between the check and the read, raising AttributeError, on a free-threaded build or for a caller passing no caps. Validation: live through real ChatSession turns, with a fake MCP server that drops a tool between turns. Qwen on vLLM (tool search off and client-side) and claude-opus-5-5 called the gone tool, got the refusal and explained it accurately, and the requests after it returned 200. gpt-6-astra never called it (4 of 4): its hosted tool search found nothing and it declined.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1287.
Problem
A session's tools can change mid-conversation: an MCP server drops a tool or goes away
(
_on_mcp_tools_changed), or another user sends on a shared workstream and the catalog follows thesender (
bind_acting_user). When the model then called such a tool,_prepare_tool_itemrefusedit with:
and the operator saw "Model called unknown tool". Both read as if the model had made the name up.
The list was every built-in preparer plus the sender's MCP tools, not what the request offered:
a regular session was told about the coordinator tools, a persona session about tools its persona
hides, a coordinator about tools an operator revoked, and a tool-search session about every
deferred tool.
Change
_unavailable_tool_errorbuilds the refusal:offered; available only to another user, once the workstream is shared (
_shared_workstream);misspelled, unless the name was offered exactly.
deferred and revoked ones, or a task agent's own tools, which
_prepare_tool_for_principal(offered=...)carries for the agent loop. With tool search on, itsays more may be found by searching.
header "X: not available".
_get_deferred_names, which the refusal and every request read, reads the tool search manageronce, as
_get_active_toolsdoes. A catalog refresh on the MCP thread could drop it between thecheck and the read and raise
AttributeError: on a free-threaded build, or on the standard buildfor a caller that passes no
caps, since_get_capabilities()then runs in between. Bothcurrent callers pass
caps, so the shipped image could not reach it.Deferred tools are left to tool search rather than listed. A live probe found that listing them
would be safe (neither native model called an unloaded deferred tool directly: claude-opus-5-5
declined, gpt-6-astra searched first), but it would save nothing, and tool search exists to keep a large catalog out
of the context.
Correction to the issue
The issue says that on OpenAI the replayed
tool_search_outputmakes the API treat a gone tool asloaded. Live, it does not: gpt-6-astra never called the gone tool in 6 of 6 runs, even when asked
for it by exact name. Its hosted tool search found nothing and it declined, so on that lane the
request's own tools already tell the model, and this refusal is not reached.
Validation
Live smoke through real
ChatSessionturns with a fake MCP server that drops a weather toolbetween turns (temporary SQLite; tools that touch the machine refused in the harness):
Qwen and gpt-6-astra were rerun on the final code. claude-opus-5-5 ran on an earlier revision,
whose refusal also named another user; the request it sends has not changed since. In 1 of
the 6 gpt-6-astra runs, once its search came back empty, the model called its own earlier,
real result invalid. That comes from the OpenAI replay of fix: replay OpenAI hosted search items with their tool namespaces #1285, not from this change; the
findings are on OpenAI Responses replay omits hosted tool items, so the model loses its own searches and re-runs tool search #1281.
An earlier wording named "another user" in every refusal, and a live Qwen reply passed that
guess on to a one-person chat; it is now named only on shared workstreams, and a live recheck
shows the reply without it.
tests/test_unavailable_tool_refusal.py: the offer list, revoked tools, personas, native andclient-side tool search, shared and single-user causes, the empty offer, a task agent through
_run_agent, that an agent's offer does not outlive its call, and deferred names surviving themanager being dropped mid-read (fails against the old two-read body). Three tests that pinned
the old wording are updated.
Mutation controls: each of 14 breaks of the new behavior fails a test.
Full suite passes; ruff and mypy clean.
Not changed
reads as the request path, so a catalog swap landing between them can skew one refusal's list,
and the next one reads the new catalog. Locking would put MCP manager calls under the session
lock, which the MCP code avoids by design.
mode").