Skip to content

fix(infinity-daemon): settle tool calls orphaned by a daemon restart - #164

Closed
akainth015 wants to merge 1 commit into
pr/interrupt-history-kindfrom
pr/orphaned-tool-calls
Closed

akainth015 wants to merge 1 commit into
pr/interrupt-history-kindfrom
pr/orphaned-tool-calls

Conversation

@akainth015

@akainth015 akainth015 commented Oct 19, 2026 •

Copy link
Copy Markdown
Member

After a restart, just viewing a session in the web UI could flip it to green "Running" permanently. Two causes:

  1. Orphaned calls. If a thread's history ends in a tool call that was still running when the old daemon died, the thread waits forever: viewing it spawns a driver, awaiting_tool_result() is true, and the driver never exits. Fix: PersistentConversationStore::settle_orphaned_tool_calls runs at cold start, before the agent system starts. It appends the same synthetic interrupted result an interrupting message would, with text that names the restart. A RawValue tail probe deserializes only each thread's last message, so threads that don't end in such a call are never loaded.
  2. Sleeping threads held drivers. sleep_until_event_or_input is never answered by design, but only close_thread was exempt from awaiting_tool_result. So any woken sleeping thread stayed resident and showed Running, even without a restart. Fix: a shared tool_call_expects_result exempts both tools. Sleeping drivers now exit and respawn on the next event or input.

Tests:

Each test fails without its half of the fix. Stacked on #161.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Deploying infinity with  Cloudflare Pages  Cloudflare Pages

Latest commit: 01e889e
Status: ✅  Deploy successful!
Preview URL: https://af2bd09e.infinity-dc7.pages.dev
Branch Preview URL: https://pr-orphaned-tool-calls.infinity-dc7.pages.dev

View logs

@akainth015
akainth015 force-pushed the pr/session-status-reconcile branch from cafeeee to d46d069 Compare October 20, 2026 00:05
@akainth015
akainth015 force-pushed the pr/orphaned-tool-calls branch from 3e802b3 to 6e2bfb5 Compare October 20, 2026 00:14
@akainth015
akainth015 marked this pull request as draft October 20, 2026 02:52
@akainth015
akainth015 force-pushed the pr/session-status-reconcile branch from d46d069 to a97b661 Compare October 20, 2026 17:42
@akainth015
akainth015 force-pushed the pr/orphaned-tool-calls branch from 6e2bfb5 to 086cb4b Compare October 20, 2026 17:42
…ing can't wake zombie sessions

After a restart, viewing a stale session in the web UI flipped it
permanently to Running (even once #161 reports it Idle): attaching a client
wakes the thread's driver (`Live` clears the idle flag), and the driver
parks forever because `awaiting_tool_result()` is true for its trailing
unanswered call. Two causes:

* **Orphaned calls**: the RAP server, input queue or timer that would
  deliver the result died with the old process.
* **`sleep_until_event_or_input` held its driver**: its no-op call is never
  answered by design, but only `close_thread` was exempt from
  `awaiting_tool_result`, so any woken sleeping thread showed Running,
  restart or not.

## Fix

Builds on #170 (`InfinityMessage::Interrupt`).

* **core**: exported `tool_call_expects_result(name)` exempts `close_thread`
  and `sleep_until_event_or_input`; sleeping drivers now exit and are
  respawned by the router on the next event or input.
* **daemon**: `PersistentConversationStore::settle_orphaned_tool_calls` runs
  at cold start, before the agent system starts, and settles every trailing
  call that expects a result with an `InfinityMessage::Interrupt` entry, as
  an interrupting message would (the model sees the usual "interrupted"
  result; the entry carries no cause, so the restart is recorded only in its
  `-orphaned-by-restart` message ID). History then ends on the interrupt:
  the thread is stopped, not waiting. A tail probe (`RawValue`, so only the
  last message is deserialized) keeps threads that don't end in such a call
  unloaded.

## Tests

* `driver_idles_after_sleep_until_event_or_input_call` (core residency).
* `settle_orphaned_tool_calls_answers_only_orphaned_calls` (per-tool
  selection, persistence, idempotence).
* `startup_settles_orphaned_tool_calls` (the sweep runs at startup).

Co-authored-by: Infinity 🤖 <infinity@hydro.run>
@akainth015
akainth015 changed the base branch from pr/session-status-reconcile to pr/interrupt-history-kind October 22, 2026 01:18
@akainth015
akainth015 force-pushed the pr/orphaned-tool-calls branch from 086cb4b to 01e889e Compare October 22, 2026 01:18
@akainth015 akainth015 closed this Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant