Skip to content

codex bridge on Windows: watch-once deadline computed after startup causes constant exit 124 (slow-startup hosts) #558

Description

@fujibee

Reported by @8CEVSmSRMT32119 (東リ屋) with a detailed writeup, repro steps, and measured numbers: https://note.com/samehadaonsen/n/ne275f93e406c — thank you. The root cause below is confirmed against main (62450b3b41f0c4d8ec80b7a6ded5ec09b92e9174); items 2–4 are recorded from the report.

Environment (reporter)

Windows 11 Pro (10.0.26200), Git for Windows (MSYS2) bash, agmsg v1.1.11, @openai/codex in a VS Code terminal, working directory on Google Drive (DriveFS) with a Japanese path. MSYS fork emulation measured at 55–300 ms per process spawn (30–50× Linux), so any fork-heavy startup is dominated by that coefficient (identities.sh 6–29 s scaling with team count, whoami.sh 5–20 s).

Bug 1 (root cause, CONFIRMED against main): watch-once deadline is computed after startup, so a slow-startup host always hits exit 124

watch-once.sh computes its polling deadline after the startup work (project resolution, subscription-pair resolution, filter building — all fork-heavy), at watch-once.sh:82: deadline=$(( $(date +%s) + TIMEOUT )).

The bridge, meanwhile, force-kills the child at a ceiling measured from spawn time: codex-bridge.js:1036: timeoutMs: (this.opts.timeout + this.opts.interval + 10) * 1000.

So the child's real wall time is startup + TIMEOUT, but the bridge kills at TIMEOUT + interval + 10 from spawn. When startup exceeds interval + 10 (~12 s at defaults), the child is killed before it can reach its own deadline and the clean exit 2 — it exits 124 every time. On the reporter's host startup alone is ~29 s, so it is structural, not flaky. On Linux startup is ~0.2 s, well under the ceiling, so it never surfaces.

Downstream effects the reporter observed: the bridge self-destructs after 3 consecutive failures, codex-bridge-launcher.sh restarts it, and orphan watch-once processes accumulate (up to 98 processes / 56% CPU observed; the orphan-retention part is the same root as #149).

Reporter's fix (2 lines, decisive): capture the start timestamp at the top of watch-once.sh, before the startup work, and base the deadline on it:

_AGMSG_WO_START="$(date +%s)"
# ...
deadline=$(( _AGMSG_WO_START + TIMEOUT ))   # was: $(date +%s) + TIMEOUT, after startup

This makes the deadline lifetime-based (startup counts toward TIMEOUT), so total wall time fits under the bridge ceiling. Harmless on Linux. Verified by the reporter: exit 124 stopped completely (6+ min monitored).

Supporting knob the reporter used: AGMSG_WATCH_ONCE_TIMEOUT=240 (re-arm period; internal polling stays 2 s, so receive latency is unchanged). On Windows each re-arm pays the ~29 s startup fork storm, so a 30 s period spends ~50% of uptime in startup; 240 s brings that to ~11%. This argues for reconsidering the default and/or documenting that "re-arm period" and "poll interval" are separate.

Bug 2 (reported): codex join step 5 (seat record) is skipped under slow startup

codex-type join requires a separate codex-record-session.sh step (template.md step 5). When the scripts are slow (Bug/§ above), the codex agent times out and improvises, skipping step 5 — reproducibly. Without the seat record, _session-start.sh excludes the role ("No recorded seat") and the bridge never starts while the launcher retries forever. Reporter's local fix wraps join.sh to auto-run the recorder for codex joins, and judges success by the record file's existence rather than exit code (the recorder returns exit 0 even when it writes nothing). Making join and seat-record inseparable (or an opt-in on join) would remove the agent-improvisation failure structurally. Related sender-side note: codex-record-session.sh's poison-record handling was #472/#473.

Bug 3 (reported): identities.sh slowness itself induces failures

The fork cost makes identities.sh take 6–29 s, which in turn induces the step-5 improvisation above and occasional mis-created teams from misread instructions. Reporter added a local cache layer (invalidated by teams/*/config.json mtime + count, 20 s → 1.2 s), but notes it thrashes when join rewrites team config during retry storms. Relates to #449 (identity/inbox hot-path subprocess count on Windows).

Suggested upstream actions (reporter's priority order)

  1. watch-once deadline fix (2 lines) — fatal on slow-startup hosts, harmless elsewhere. This is the one to take first.
  2. Integrate the seat record into codex join, or make record failure an explicit error (and fix the recorder returning exit 0 on no-op).
  3. Reconsider the AGMSG_WATCH_ONCE_TIMEOUT default, or document re-arm-period vs poll-interval in the README.
  4. (Medium-term) reduce startup fork count / consider a resident polling loop — high value for Windows/Git Bash users.

The reporter has settled on: codex delivery via an external compiled monitor, agmsg retained as the messaging substrate for claude-code/antigravity. Their local fixes are overwritten by install.sh --update, so they re-apply manually — which is itself a reason to land 1–2 upstream.

Related: #149, #449, #415, #459, #458.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions