Skip to content

Retry the atomic rename when Windows briefly locks the temp file - #40

Open
rzilem wants to merge 1 commit into
petergyang:mainfrom
rzilem:fix/windows-eperm-rename-retry
Open

Retry the atomic rename when Windows briefly locks the temp file#40
rzilem wants to merge 1 commit into
petergyang:mainfrom
rzilem:fix/windows-eperm-rename-retry

Conversation

@rzilem

@rzilem rzilem commented Aug 27, 2026

Copy link
Copy Markdown

The problem

On Windows, a virus scanner or indexer can hold a short-lived handle on a file the moment it is created. A rename against that handle fails with EPERM, EACCES or EBUSY.

atomicWrite is unusually exposed to this: it creates a temp file and immediately renames it over state.json. That is the narrowest possible window for a scanner to still be holding the file it just saw appear.

This is not a concurrency problem. It happens with a single process writing sequentially. On my machine (Windows 11, Node 24, McAfee real-time protection):

500 sequential atomicWrite calls, system temp dir : 19 failed, EPERM on rename
500 sequential atomicWrite calls, ~/.human-review : 23 failed, EPERM on rename

Roughly one in twenty-five writes, and each one is a batch of comments or edits the user had already typed into the browser.

I first noticed it because npm test was failing differently on each run — the failures moved between state.test.js, concurrent.test.js and others, which is what sent me looking at atomicWrite rather than at any one test.

The fix

The handle clears within a few milliseconds, so the write is not really lost — it just has to be asked for again. The rename retries with a short exponential backoff. Same measurement after the change:

500 sequential atomicWrite calls : 0 failed, 14 needed a retry, worst case 2 retries (~3ms)

Two things beyond the obvious retry:

Cleanup needs it too. The handle that blocks the rename blocks the unlink for exactly as long, so the single cleanup attempt could strand a .human-review.tmp file containing the newest state. Nothing ever sweeps that directory — prune() only ages out entries inside the JSON — so it would sit there indefinitely. Cleanup now retries on the same schedule, then gives up quietly: a stuck temp file must never mask the error the caller is already throwing.

Neither sleep can escape. For the same reason, an exception from the backoff would otherwise surface in place of the real write error.

What is deliberately unchanged

  • The exported signature. atomicWrite(file, data) is exactly as it was. The injection the tests use is private, so no caller can hand it a rename that quietly does nothing and reports a successful write. Exports remain atomicWrite, Store, resolveAsset.
  • Non-retryable errors still fail on the first attempt. ENOSPC and friends do not sleep and do not retry.
  • The retry is bounded, so a genuinely stuck file still surfaces its error rather than hanging.
  • The security properties: exclusive wx create, unguessable temp name, symlink resistance.

Tests

Ten new tests in test/state.test.js. They reach the retry by patching the calls atomicWrite makes — fs.renameSync, fs.unlinkSync, and Atomics.wait (which only sleepSync uses) — so every one of them drives the real two-argument atomicWrite rather than a test-only seam.

I checked they actually fail without each fix, by reverting one piece at a time:

Reverted Result
One-shot rename (no retry) 7 tests fail
RENAME_TRIES 5 → 6 3 tests fail
EBUSY dropped from the retryable set 1 test fails
Single-attempt cleanup 2 tests fail
Cleanup's final-attempt guard 1 test fails
Cleanup sleep containment 1 test fails
Retry sleep containment 1 test fails
sleepSync → no-op 7 tests fail

node --test101 tests, 100 pass, 1 skipped (the existing symlink test, which needs Windows privileges).

One note on running the suite

npm test reports 3 files as failing on Windows:

Assertion failed: !(handle->flags & UV_HANDLE_CLOSING), file src\win\async.c, line 76

Every assertion inside those files passes — it is --test-force-exit crashing libuv during teardown for the files that open a server (asset-paste, command-cache, url-review). Plain node --test exits cleanly. Unrelated to this change and not something I touched, but it may be worth knowing if Windows CI ever gets set up.

On Windows a virus scanner or indexer can hold a short-lived handle on a
file the moment it is created, and a rename against that handle fails
with EPERM, EACCES or EBUSY. atomicWrite hits this with no concurrency at
all: it creates a temp file and immediately renames it over state.json,
which is the narrowest possible window for a scanner to be holding it.

On a box running McAfee real-time protection, 19 of 500 sequential writes
to a temp directory failed this way, and 23 of 500 to ~/.human-review.
That is roughly one in twenty-five, and every one of them is a batch of
comments or edits the user already typed. The handle clears within a few
milliseconds, so the write is not really lost -- it just has to be asked
for again. Same measurement after this change: 0 failures in 500 writes,
14 needed a retry, worst case 2 retries, about 3ms.

The cleanup path needs the same treatment. The handle that blocks the
rename blocks the unlink for exactly as long, so a single attempt could
strand a .human-review.tmp file holding the newest state, and nothing
ever sweeps that directory -- prune() only ages out entries inside the
JSON. Cleanup retries on the same schedule and then gives up quietly,
since a stuck temp file must never mask the error the caller is throwing.
Neither sleep can escape for the same reason.

Non-retryable errors still fail on the first attempt, the retry is
bounded so a genuinely stuck file still surfaces, and the exported
signature is unchanged -- the injection the tests use is private, so no
caller can pass a rename that quietly does nothing and reports success.
Tests reach it by patching fs.renameSync, fs.unlinkSync and Atomics.wait,
so they all drive the real atomicWrite.

Verified by reverting each part in turn: one-shot rename, RENAME_TRIES,
the retryable set, single-attempt cleanup, the final-attempt guard, both
sleep containments, and a no-op sleepSync. All eight fail the suite.
petergyang added a commit that referenced this pull request Sep 2, 2026
petergyang added a commit that referenced this pull request Sep 3, 2026
* Wait for Send with one background poll instead of timed re-polls

The skill told agents to run poll --timeout 600 in the foreground and
re-run it on every timeout. Now it starts one open-ended poll in the
background and ends the turn; the harness wakes the agent when the
command exits, which only happens on Send or when the review is closed.

To make an open-ended wait safe, the CLI reconnects (restarting the
server if needed) when the connection drops instead of giving up after
three tries, and treats a socket close without end as a drop so the
promise cannot be left unsettled.

* Retry the atomic rename when Windows briefly locks the temp file

From community PR #40 by @rzilem.

* Harden review frame: drop popup/download rights, validate external hrefs

From community PR #37 by @dacoldest.

* Don't submit a comment on an IME composition Enter

From community PR #30 by @ulysses0604.

* Use the document filename as the browser tab title

From community PR #36 by @NuclearManatee.

* setup: treat any node_modules bin as transient; fix caret, details, heading-label bugs

- Port community PR #7 by @noammigdali-hio onto the invocation(run) signature:
  a bin under any node_modules segment is transient, not just npm's _npx cache.
- Issue #12: clicks inside the compose textarea place the caret instead of
  being swallowed by the card's focus handler.
- Issue #25: <summary> keeps its native toggle so collapsed content stays reachable.
- Issue #26: a heading labels itself instead of being named after the heading before it.

* Server: make every wait terminate, persist delivery, queue second sends, one server per state dir

- A tab that unloads tells the server; unless it reconnects within a short
  grace the review ends with reason window_closed and the waiting poll is
  released. A tab that silently vanishes ends the review after the 30-minute
  TTL instead of being deleted with the poll left hanging.
- A poll with no review open waits a short grace for one, then closes with
  reason no_review_open. Closed answers carry the unsent counts.
- A new poll for a target supersedes an older one, so stale background waits
  cannot double-apply a batch.
- delivered is persisted, so a server replaced between delivery and --ack
  honors the ack instead of re-shipping the batch. A batch written by an
  overlapping server is read from disk.
- Sending while the agent is applying a batch queues only the new items and
  carries the old cleanup forward; the agent's next --ack clears both.
- edits_saved and markdown flags per page; edit text capped at 200k with an
  explicit truncated marker instead of a silent 4k cut; Markdown and
  self-rendering pages keep unsent edit rows when the file changes on disk.
- Artifact URLs carry a per-run secret segment; a page key derived from the
  file path is no longer enough to read sibling files.
- Localhost reviews proxy the app's own requests (API calls, prefetches,
  srcset images) to the app origin instead of 404ing.
- One server per state directory: a lock file plus a health-checked takeover
  of an older-protocol server. Protocol bumped to 9.
- New routes: DELETE edit (undo), POST mode (dynamic), POST discard, POST away.
- Comment rewording updates an undelivered batch in place, or takes a fresh
  id after delivery so the rewrite ships next time.

* Browser and SDK: tab-close beacon, session re-bootstrap, undo, leftover banner, multi-block edits

- The chrome tells the server when the tab unloads (keepalive fetch on
  pagehide) and re-opens a session when the server has forgotten it, instead
  of turning into a dead tab that still looks alive.
- Undo for the two irreversible gestures: a toast with an Undo button after a
  block delete or move restores the block and drops its edit row.
- Feedback left over from a review that ended without a Send shows a
  Keep/Discard banner on the next open, noting that HTML text edits are
  already in the file either way.
- A second Send while the agent works shows as queued, not as nobody
  listening. Comments get an Edit button and an edited badge; rewording
  after delivery explains it ships with the next Send.
- Typing over a selection that spans blocks reports every block it touched,
  including a deleted row for one that vanished; a paragraph turned into a
  list keeps its label and captured original.
- CLI: an open-ended wait ends after 12 hours with a timeout status that
  says to check status and start again; superseded polls just print.
- SKILL.md: closed/superseded handling, edits_saved, deleted and truncated
  rows, rendered quotes with anchors, staged assets before ack, note-only
  batches. Codex block updated to match.
- srcset URLs are absolutized for localhost reviews.

* Fix phantom sessions after a review ends; pin edit-list ordinals to load order

- After the server ended a session, the tab's event stream reconnected, got
  a 404, and the new re-bootstrap path opened a fresh session with no tab
  behind it, which kept the review open and the poll waiting. The chrome
  now never re-bootstraps once the review ended or the tab is unloading,
  and the server drops a session no browser connected to within a minute.
- Label ordinals ("p 3") come from sibling order at load, so deleting a
  paragraph does not hand two blocks the same label within one review.
- A list command now retracts the row the fresh <li> emitted about itself
  and reports the change under the paragraph's own label with its original
  text.

* Make the reconnect test fail fast instead of stalling CI; per-test timeout

On a slow Windows runner the poll's reconnect could spawn its own server
before the test's replacement announced itself, and then wait forever for a
batch sent elsewhere. The first reconnect now waits 3s, the test races the
poll against a 30s timer, and every test gets a 60s timeout so a hang fails
the file instead of the whole job.

* agent-loop: bound the reconnect test's setup and instrument it for the Windows stall

* agent-loop: bound child shutdown waits; log cleanup state

* agent-loop: a signal-killed child counts as stopped; drop the debug markers

* SKILL: background wait is Claude Code only; Codex and others stay on the poll in the foreground

Codex returns a detached session id and nothing wakes the agent when that
session finishes, so a Send sat unread until the user nudged. The skill and
the AGENTS.md block now say which harness gets which wait, and how to cope
with a shell tool that caps command duration.

* SKILL: stay on the foreground poll through incoming messages; state the ended-turn limit plainly

* setup: own the AGENTS.md block with markers and replace a legacy one

A project set up months ago kept the instructions from that day forever,
because setup skipped any AGENTS.md that already mentioned human-review.
The block is now written between marker comments and rewritten on every
setup; the unmarked block an older setup wrote is recognized by its heading
and replaced; a project that wrote its own human-review guidance is left
alone.

* v0.7.0

* state: keep each page's pristine copy in its own file

Every edit flush rewrote state.json in full, and it embedded the complete
HTML of every page reviewed in the last month. The copy now lives in
pristine/<key>.html, is written only when it changes, and is removed with
its page. A state file that still embeds copies loads as before and moves
them out on its next save.

* Editor smoke tests under jsdom; fix ordinal snapshot missing the body

Three tests boot the SDK against a jsdom document: block delete with undo,
typing over a selection that spans blocks, and stable labels after a
deletion. The last one caught that the load-order snapshot skipped the body
element, so top-level paragraphs still renumbered.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant