design: Orchestrator route-first redesign (route-to-existing as primary, create as fallback)#1741
Open
wqymi wants to merge 6 commits into
Open
design: Orchestrator route-first redesign (route-to-existing as primary, create as fallback)#1741wqymi wants to merge 6 commits into
wqymi wants to merge 6 commits into
Conversation
…ways halt
Add unconditional empty-step guard in processor.ts BEFORE the doom-loop
check. Genuine empty tool calls (undefined/null/empty object, even with
harmless extra fields like {_meta}) are always counted regardless of
error/summary/structured/finish state. When emptyStepCount hits
EMPTY_STEP_THRESHOLD=3, halt with error.
- Add emptyStepCount to ProcessorContext (persists across process calls)
- Add isEmptyInput helper that filters out _meta-prefixed keys
- Reset counter on non-empty input, increment on empty
- Regression test: single LLM response with 3 empty tool calls → stop
… empty tool calls Replace the threshold-based empty-step guard (tolerate N, halt on N+1) with an immediate retry mechanism that mirrors autoRetryTextToolCall: - A SINGLE empty tool call now triggers an immediate retry: the bad assistant turn is discarded (error-tagged with EmptyToolCallError) and a synthetic user turn is appended to re-drive generation. - Bounded by EMPTY_TOOL_CALL_RETRY_LIMIT (default 2, mirrors TEXT_TOOL_CALL_RETRY_LIMIT). On exhaustion the error stays terminal. - The empty turn is discarded from context (toModelMessages skips error-tagged messages) so it cannot poison later context. Changes: - message-v2.ts: add EmptyToolCallError to error discriminated union - flag.ts: add MIMOCODE_EMPTY_TOOL_CALL_RETRY_LIMIT (default 2), remove MIMOCODE_EMPTY_STEP_MAX_RECOVERY - prompt.ts: replace handleEmptyStep (threshold-halt) with autoRetryEmptyToolCall (immediate retry), wire into all 3 classify sites (existing-assistant, fork, main) - processor.ts: remove processor-level threshold-halt guard (EMPTY_STEP_THRESHOLD, emptyStepCount, isEmptyInput), retry now handled at prompt loop level - empty-step-detection.ts: keep isEmptyStep + isEmptyInput (with _meta filtering), remove unused threshold constants and nudge messages - Tests: update to assert retry behavior (single empty call → retry + error-tagged discard, exhaustion → halt, recovery → valid answer)
… on no-arg tools
isEmptyStep case (a) flagged every tool call with {} as empty, including
legitimate no-arg tools (plan_exit, plan_enter, future z.object({}) tools).
These tools accept no parameters - {} is the only valid call - but the
guard treated it as empty, triggering a retry loop that could never succeed.
Two fixes:
1. isEmptyStep now accepts an opts.toolHasArgs callback that checks whether
a tools parameter schema has required fields. When toolHasArgs returns
false (schema is z.object({}) or all-optional), {} input is NOT flagged.
2. Substantive text or reasoning alongside a tool call now prevents the
empty classification - fixing the early-return bug where case (a)
returned before the text/reasoning check at :60.
prompt.ts builds the lookup from the tool registry (z.ZodObject shape
introspection) and passes it to isEmptyStep via autoRetryEmptyToolCall.
Regression tests:
- No-arg tool called with {} - NOT flagged (with toolHasArgs)
- Args-accepting tool called with {} - still flagged
- Tool call + substantive text - NOT flagged (text overrides)
- Legacy (no opts) - backwards-compatible, all {} inputs flagged
…ry, create as fallback) Add design document for the Orchestrator route-first redesign. Core thesis: the Orchestrator is a router, not a creator — its default action should be 'route to an existing session' (session send), with create as a fallback when no existing session fits. Key design points: - New session route operation as first-class routing primitive - Harness injects live active-sessions context into orchestrator prompt - create demoted to fallback (only when no existing session matches) - orchestrator.txt decision guidance rewritten from decompose→dispatch to route→(create only if none fits) - 4-phase implementation roadmap: context injection → prompt rewrite → route primitive → deprecated path cleanup
… injected context Revision addressing user feedback: the Orchestrator is itself an AI that understands semantics — routing decisions should be made by the AI, not by tool-level matching algorithms. Key changes: - Removed session route operation and findBestMatch pseudocode entirely - New Core Principle: 'AI Routes, Tools Provide + Execute' - Added 'Why Tool-Level Matching Cannot Work' section - R1 is now <active-sessions> context injection (harness provides the list) - R2 is orchestrator.txt rewrite guiding AI to route-first - No new tool verb needed — AI uses existing session send/create - Phase 3 changed from 'hardcoded route primitive' to 'optional prompt strengthening if Phase 2 guidance is insufficient' - Phase 1 + Phase 2 are the main body; no tool-layer matching ever
New section addressing the context bloat problem: injecting full session details every turn wastes tokens on non-routing turns and scales poorly. R1.1 analyzes 5 injection strategies (on-demand pull, full detail, compact summary, conditional, incremental) and recommends compact summary + on-demand detail: - Default: inject compact one-liner per session (id|title|mode|status) only for non-terminal sessions (~30 tokens/session) - On-demand: AI uses session ask/status for details when needed - No dir or recent-activity in the injected block (confirmation-level info, not decision-level) Also updated R1 example to match compact format, R2 prompt example to reference compact format, and Code Impact section accordingly.
wqymi
force-pushed
the
docs/orchestrator-route-first-redesign
branch
from
477465b to
f826382
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
session send), withcreateas a fallback when no existing session fitsKey Design Points
session routeoperation — first-class routing primitive that gets the active session list, matches the task to an existing session, and routes viasession send(or recommends create if no match)<active-sessions>block injected into the orchestrator's system prompt on every turn, so it always sees its fleet without needing explicitsession listcallscreatedemoted to fallback — only used when no existing session matches the task;--topicmechanism preserved but deprecatedorchestrator.txtrewrite — decision guidance changes fromdecompose → dispatch (create)toroute → (create only if none fits)Problem This Solves
The current create-first default causes session explosion: the Orchestrator creates a new child session for every task, even when an existing session could handle same-theme work. Topic-based reuse (
--topic) was a workaround but is fundamentally unreliable because it couples routing to string matching on LLM-generated labels.Architecture Impact
packages/opencode/src/tool/session.tsrouteverb;createtopic logic removed;listsummary formatpackages/opencode/src/session/llm.ts<active-sessions>context injection inbuildSystemArraypackages/opencode/src/session/prompt/orchestrator.txtdocs/harness/MiMo Orchestrator Mode.mdFull details in:
docs/compose/specs/2026-07-14-orchestrator-route-first-redesign.md