Conversation
jsell-rh
left a comment
There was a problem hiding this comment.
Thorough design — this addresses real, compounding problems in the current oversight story. The fact that tmuxApprove sent Enter while request_decision pasted text and hoped for C-m to submit it is exactly the kind of brittleness a unified gate model removes. Commenting on the six open questions and a few implementation notes.
Open Questions
1. Should gate resolution nudge the agent?
Agree with the doc's recommendation: no. Nudging injects tmux-paste back into a flow that was specifically designed to escape it. Polling check_gate is correct. If we want an agent to react faster, the ignition prompt should tell it to poll on a reasonable cadence (10–30s) rather than wait for a push.
2. Multiple approvers?
Phase 1: no. The audit trail the Gate model gives us is sufficient. If multi-approver becomes a requirement, the data model already supports adding an approvals []GateApproval relation without breaking the existing resolved_by field. Leave it open rather than designing it now.
3. Tool approval deny: Escape vs. record-only?
Send Escape. Record-only silently leaves Claude Code in a blocked state waiting for the tool call to be answered — that's worse than the retry risk. The denial reason captured in gate.resolution is returned by check_gate, so the agent can see it next time it checks in. Worth noting: Escape behavior may vary across Claude Code versions, so this path should be exercised in CI the same way tmuxApprove is tested.
4. Ignition prompt format?
Brief summary as shown in §12.2 is right. Agents don't need the full rule table at boot — just awareness that gates exist and the two tool names (request_gate, check_gate). Full policy is accessible via boss://protocol or a new boss://hitl-policy/{space} MCP resource. Consider auto-injecting the summary only when HITLPolicy is non-empty to avoid noisy ignition prompts for unconfigured spaces.
5. Denied lifecycle gates retryable?
Yes — new request, new gate. The old gate stays in the audit trail. This is also more consistent with how decision gates work: an agent that receives denied for a spawn can make an architectural adjustment and ask again. The alternative (one-strike denial) creates confusing dead ends.
6. Approvals tab vs. InterruptTracker.vue?
Run both in parallel through Phase 1. The interrupts table isn't going away immediately, and existing operators are familiar with the tracker. Phase 2 unification is the right call. One practical note: when gates supersede an interrupt record, mark the interrupt resolved server-side so InterruptTracker.vue doesn't show it as a duplicate pending action.
Implementation Notes
Gate ID namespace collision. The spec shows "gate_<timestamp>". Millisecond timestamps will collide under concurrent requests. Suggest "gate_" + ulid() or "gate_" + uuid() — the same approach used for agent IDs. Very low cost to fix before any code is written.
check_gate without polling interval guidance. The doc says "10–30 second poll interval is reasonable" but this is hidden in §8.4. The MCP tool response for request_gate should include a poll_interval_sec hint (e.g., 15) so agents don't have to read the spec to know the right cadence.
gateTimeoutLoop and liveness loop coupling. §11.1 shows the timeout loop stopping on s.stopLiveness. This is correct for the cleanup path, but name the channel more generically (s.stopBackground) or give it its own done channel — liveness and gate timeouts are separate concerns and shouldn't share stop signals.
Cascade on space delete. When a space is deleted, its pending gates should either be cancelled or cascade-deleted. The DB schema should include ON DELETE CASCADE on space_name (referencing the spaces table), or the space delete handler needs to explicitly cancel pending gates before teardown.
Fleet YAML dry-run output. The example output in §10.3 is exactly the right level of detail. Make sure the diff logic treats "no HITL policy" → "HITL policy set" as a change and "HITL policy set" → "no HITL policy" as a removal — same pattern as how description diffs work in fleet.go.
What's well-done
- The backward-compat guarantees in §12 are solid. Zero breaking changes on unconfigured spaces is the right default.
- The phased rollout (§13) is realistic. Phase 1b (MCP tools + decision gates) is the highest-value, lowest-risk starting point.
- Colocating
HITLPolicyin existingagents.configJSON avoids a schema migration for the common case. - The sequence diagrams in §8 are the clearest part of the doc — should stay through implementation.
Approving as a design spec. Ready to move to Phase 1a once the gate ID format is clarified. Great contribution.
|
Operator feedback (additional review pass): §4.1 — Decision gates: polling is unreliable
The doc recommends agents poll at a 10–30s interval. In practice, agents don't always reliably poll — they get busy with other work and miss the window, or restart and lose the polling loop entirely. The nudge-on-message pattern already exists and works: when a message is delivered to an agent's inbox, the coordinator sends a tmux nudge prompting the agent to check in. Recommendation: On gate resolution, fire the existing nudge mechanism to the requesting agent in addition to setting §4.2 — Task transition gates: no notification on successful executionWhen the operator approves a task gate, the coordinator moves the task server-side — but there is no notification to the agent that the move happened. The agent either has to poll Recommendation: Pair the server-side task move with a message to the requesting agent: §4.3 — Decision gates: rely on message + nudge instead of pollingSame as §4.1 — the polling model is fragile. The message + nudge system is the established, reliable pattern. Recommendation: When a decision gate is resolved, deliver the resolution as a message to the requesting agent's inbox and nudge the agent. The agent reads it in its next check-in cycle — same as any other message. §4.4 — Tool approval deny: message instead of Escape
Sending Escape stops the tool call but leaves the agent without context — it just sees the call rejected with no explanation. The agent may retry blindly or get confused about why the action was blocked. Alternative recommendation: On deny, don't send Escape. Instead:
This keeps the agent operational rather than stopping it dead in the water. If the operator wants to cancel the specific tool call they can note that in the rationale. The trade-off is that the tmux tool prompt remains pending — the agent will need to handle or re-navigate it. Worth calling out explicitly in the design whether that's acceptable. §12.2 — Ignition prompt: same polling concernThe ignition prompt as drafted tells agents to "poll |
|
RE: 4.4 Further, perhaps the deny path has an optional checkbox that will stop the agent execution. |
Operator Feedback — Structured ReviewGood design overall. Main cross-cutting concern: polling is unreliable — we've seen agents not poll consistently, especially when busy. The spec should lean on the existing nudge-on-message system for gate resolution rather than requiring agents to poll. §4.2 Task Transition GatesMissing: notification to agent on gate resolution. When the gate fires, the task stays blocked and the agent gets a Suggestion: On gate resolution (approve or deny), the server should send a message to the agent (same path as
This means agents don't need to poll at all — they just handle the resolution like any other inbound message. §4.3 Decision Gates — Polling ConcernThe spec relies on agents polling The current Suggestion: When the operator resolves a decision gate, the backend should:
Agents can optionally call §4.4 Tool Approval Gates — Deny BehaviorSending Escape on deny may stop the agent dead in the water. The current deny path sends Escape to cancel the tool call. This works for tmux, but it interrupts the agent's flow without explanation. The agent has no idea why it was denied or what to do next. Suggestion: On a deny:
Example message: "Tool The agent can then adapt rather than being stopped cold. If Escape must be sent (e.g. Claude Code won't proceed without it), consider sending it after the message is delivered. Open question inherited from §14.3: For ambient backends, Escape isn't applicable — the message-based path is the only option anyway. Worth clarifying this in the spec. §12.2 Ignition Prompt — Polling InstructionThe current draft tells agents to ## HITL Policy (Active)
When an action requires approval, the MCP tool returns a `gate_id` with
status `pending_approval`. **Do not poll for the resolution.**
Continue working on other tasks. When the gate resolves, you will
receive a message with the outcome — treat it like any other inbound
message and act accordingly.Summary of Requested Changes
|
Key changes based on reviewer feedback: - Resolution delivery is now push (message + nudge), not pull (polling). check_gate remains as a fallback for restart recovery and explicit state queries. This aligns with the established nudge-on-message pattern that already works reliably across the system. - Tool approval deny delivers rationale as a message rather than sending Escape by default. Optional "interrupt agent" checkbox sends Escape when explicitly requested. Ambient backend note added. - Gate IDs use ULID format to avoid collision under concurrent requests. - request_gate response includes poll_interval_sec hint. - Gate timeout loop uses dedicated stop channel (not s.stopLiveness). - Space deletion cascades to pending gates. - Interrupt dedup: when a gate supersedes an interrupt, mark the interrupt resolved server-side. - Ignition prompt updated to "await a message" pattern. - Open questions Q1 and Q3 resolved per reviewer decisions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Introduces a design document for a Human-in-the-Loop (HITL) gate system that provides configurable safety guardrails for agent actions. The spec covers lifecycle gates, task transition gates, structured decision gates, and tool approval upgrades. Design doc only — no code changes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Key changes based on reviewer feedback: - Resolution delivery is now push (message + nudge), not pull (polling). check_gate remains as a fallback for restart recovery and explicit state queries. This aligns with the established nudge-on-message pattern that already works reliably across the system. - Tool approval deny delivers rationale as a message rather than sending Escape by default. Optional "interrupt agent" checkbox sends Escape when explicitly requested. Ambient backend note added. - Gate IDs use ULID format to avoid collision under concurrent requests. - request_gate response includes poll_interval_sec hint. - Gate timeout loop uses dedicated stop channel (not s.stopLiveness). - Space deletion cascades to pending gates. - Interrupt dedup: when a gate supersedes an interrupt, mark the interrupt resolved server-side. - Ignition prompt updated to "await a message" pattern. - Open questions Q1 and Q3 resolved per reviewer decisions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
f243140 to
e12690c
Compare
Summary
Design document proposing a unified Human-in-the-Loop (HITL) gate system for OpenDispatch. This is a spec for review — no code changes.
What it proposes
A Gate is a checkpoint that blocks an action until a human resolves it. Four gate types:
request_gate/check_gateMCP tools (replaces the fragilerequest_decision→ tmux paste loop)Key design decisions
check_gatehitl:section on space and agent definitionsOpen questions for reviewers
Files
docs/design-docs/hitl-gates.md— the full design spec🤖 Generated with Claude Code