Skip to content

fix(backend-pi): finish a duplicate-resume loser's teardown before its busy refusal (#3488) - #3501

Merged
schlessera merged 1 commit into
mainfrom
fix/3488-pi-duplicate-resume-teardown
Nov 3, 2026
Merged

schlessera merged 1 commit into
mainfrom
fix/3488-pi-duplicate-resume-teardown

Conversation

@schlessera

Copy link
Copy Markdown
Owner

Closes #3488

What changed

When two concurrent resumes of one non-resident session both open a pi worker, the one that returns second still refuses with BackendBusyError. That refusal now settles only after the losing worker's teardown has finished: the worker has exited, its turn temp has been closed exactly once, and its stdio released() has been awaited exactly once. Until now the loser only got a synchronous dispose(), which requests SIGKILL and returns. The turn-temp close ran detached on exit, its refusal was dropped, and nothing waited for the stdio release. While the refusal settled, the ledger still charged both workers' reservations. Now it charges only the winner's. If the loser's cleanup is refused, the caller still gets BackendBusyError. The refusal is logged through backend.log as a warning, the reservation stays charged, and the worker's shared abort() can retry the close.

Why this shape

This follows the issue's scope and the #2863 lifecycle ruling ("Turn completion includes worker teardown"). The losing resumed acquisition decides busy or reuse, and claims a reused entry, synchronously, before any await. That way a third acquisition can never see an unclaimed idle entry while the loser waits. The pool then awaits the losing worker's shared inactive abort(), the same owner #2880's cancelAcquiredSession uses, disposes the worker and re-raises the original outcome. Cleanup refusal is reported rather than substituted, following the backend's other preserved-cleanup warnings. The pool gets the existing BackendLogFn seam as an optional third argument, and createPiBackend passes options.log. The unreachable new-session duplicate branch, public dispose(), worker protocols, the busy and result shapes, and the five-second release bound are unchanged, as the issue's out-of-scope list requires. The decision record now notes #3488 under the #2880 paragraph. The investigation report's citations were re-anchored to the moved lines, and it now says the repair landed.

Proof

Final head 2d80ed9409c5ee3d8f7ed3448cfbf6a60ed3b19f on base a9330064e56ffc82804e3025ce7c77d6127057a8 (origin/main). Linux host with bwrap, keyless, private TMPDIR.

bun install --frozen-lockfile          # no changes
NODE_OPTIONS=--max-old-space-size=8192 bun run typecheck   # exit 0
bun run build                          # exit 0
bun run lint                           # lint: all gates clean
bun scripts/proof-system.ts classify --base origin/main --head HEAD   # not a proof-system change
bun run test packages/ui-backend-pi/tests/duplicate-acquisition.test.ts packages/ui-backend-pi/tests/early-cancel-lifecycle.test.ts   # 11 pass, 0 fail
bun run check:pr --base origin/main    # exit 0, 831 pass, 0 fail

Failing first (new duplicate-acquisition.test.ts against unmodified main a272a790b; the source files this PR touches are the same on the current base):

error: PI_DUPLICATE_RESUME_LOSER_EXITED_BEFORE_REFUSAL
Expected: true
Received: false
(fail) … refuses only the loser after its confirmed teardown
error: PI_DUPLICATE_RESUME_LOSER_EXITED_BEFORE_REFUSAL
Expected: true
Received: false
(fail) … refuses only the loser after its refused teardown
 0 pass
 2 fail

The test now runs twice: once with a confirmed loser cleanup and once with a refused one. At the moment the loser's BackendBusyError settles, it asserts the loser has exited, with 1 turn-temp close, 1 released() call and 1 completed released(). With confirmed cleanup, the ledger charges only the winner's half and no warning is logged. With refused cleanup, both reservations stay charged and exactly one warning names the refusal; then a retry through session.abort() closes again (2 closes) and frees the loser's half. In both runs the winner is not signalled and is still running, it ends with success, and every reservation is back at baseline. All of the test's earlier assertions are unchanged.

Mutation receipts (on the final head; tree restored after each, git status clean):

Mutation Result
Loser path synchronous again (disposeSession(opened.session) instead of the awaited release) both runs fail PI_DUPLICATE_RESUME_LOSER_EXITED_BEFORE_REFUSAL (expected true, received false)
Busy re-raise dropped fails PI_DUPLICATE_RESUME_LOSER_REFUSES_BUSY (received undefined). The non-throwing loser then prompts, and the run's teardown waits until the timeout.
Cleanup refusal not logged refused run fails PI_DUPLICATE_RESUME_REFUSED_LOSER_CLEANUP_REPORTED (expected length 1, received 0)
Cleanup refusal replaces the busy outcome refused run fails PI_DUPLICATE_RESUME_LOSER_REFUSES_BUSY (received the cleanup refusal)

Workspace tests that cannot run on this host. bun run test packages/ui-backend-pi gives 478 pass and 24 fail. The same 24 tests fail on unmodified main, checked by diffing the (fail) lists from a throwaway origin/main worktree. They need unprivileged user namespaces, which this host refuses (unshare: write failed /proc/self/uid_map: Operation not permitted). They are in host-relay, native-host-auth-qualification, provider-history, provider-history-apify, session-prompt, stable-roster-runtime, startup-runtime and transcript-prefix, and are left to hosted CI. That covers the provider-history guard. This change adds nothing provider-facing: it only changes when a losing worker's refusal settles.

{"version":1,"kind":"brain-check-pr-receipt","head":"2d80ed9409c5ee3d8f7ed3448cfbf6a60ed3b19f","base":"origin/main","mode":"focused","startedAt":"2026-11-03T15:17:01.811Z","finishedAt":"2026-11-03T15:18:40.963Z","checks":[{"name":"all lint and leakage gates","categories":["changeset"],"outcome":"passed","elapsedMs":8791.614308},{"name":"environment documentation","categories":["changeset"],"outcome":"passed","elapsedMs":94.48743199999808},{"name":"pending changeset package names","categories":["changeset"],"outcome":"passed","elapsedMs":1037.872212000002},{"name":"contribution changeset","categories":["changeset"],"outcome":"passed","elapsedMs":16.933732000001328},{"name":"affected fast invariants","categories":["verify","unit"],"outcome":"passed","elapsedMs":89192.416864}]}

Checklist

  • Focused local preflight/behavioral receipts and complete selected hosted proof passed; actual head/base/job checkout evidence is in Proof. (Local done; hosted proof not yet requested.)
  • Acceptance criteria from the issue are all met, or the gap is named below.
  • The PR and completed/advanced issues have verified release milestones under docs/process/github.md.
  • A changeset is included, or this changes nothing a consumer can see.
  • No personal data anywhere in the diff — the leakage gate covers the whole tree.
  • The integration contract index or relevant component is updated in this same commit, or nothing in the machine surface moved. (Nothing moved.)
  • Docs under docs/ reflect the change, or nothing documented moved.
  • Provider history stays append-only (docs/decisions/provider-history.md): the provider-history guard passes, or nothing on a turn's path changed. (No provider-facing content changed. The guard tests need user namespaces and are left to hosted CI.)

Anything left open

🤖 Generated with Claude Code

…s busy refusal (#3488)

When two concurrent resumes of one non-resident session both open a worker,
the one that returns second now decides busy or reuse (and claims a reused
entry) synchronously, then awaits its own worker's shared inactive abort
before re-raising the original outcome. The refusal settles only after the
losing worker has exited, its turn temp is closed and its stdio release has
been awaited. A refused cleanup is logged as a warning through the backend
log and cannot replace the busy refusal; the reservation stays charged and
the abort stays retryable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@schlessera schlessera added this to the 0.45.0 milestone Nov 3, 2026
@schlessera schlessera added type: fix Something is broken and must behave the way it is documented. area: backends AgentBackend implementations and the seam they share. labels Nov 3, 2026
@schlessera

Copy link
Copy Markdown
Owner Author

Collisions at head 2d80ed9409c5ee3d8f7ed3448cfbf6a60ed3b19f (bun scripts/collisions.ts --json --pr 3501):

Reporting only. None of these is a lock.

@schlessera
schlessera marked this pull request as ready for review November 3, 2026 15:25
@schlessera
schlessera merged commit 04c1928 into main Nov 3, 2026
12 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: backends AgentBackend implementations and the seam they share. type: fix Something is broken and must behave the way it is documented.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

backends: a pi duplicate-resume busy refusal settles before the losing worker's teardown

1 participant