Skip to content

feat(loop): crash recovery on boot — resume features stranded mid-drive#9

Merged
mabry1985 merged 1 commit into
mainfrom
feat/crash-recovery-on-boot
Jun 13, 2026
Merged

feat(loop): crash recovery on boot — resume features stranded mid-drive#9
mabry1985 merged 1 commit into
mainfrom
feat/crash-recovery-on-boot

Conversation

@mabry1985

Copy link
Copy Markdown
Member

What

The final P0 resilience port. A drive doesn't survive a server restart, so every in_progress feature the previous run was building was stranded forever (the puller only claims ready). This was the single biggest autonomy gap.

The loop now runs a one-time _recover() before the puller spins (top of _run): for each in_progress feature —

  • if its PR actually got opened (a crash between open_pr and open_review) → adopt it → in_review;
  • otherwise → reset to ready for a clean rebuild (a stale worktree is cleaned when the puller re-claims).

in_review features are left alone — they have a PR and the webhook/merge-poll resolves them. Recovery is best-effort per feature — one failure doesn't strand the rest or stop the loop from starting.

worktree.pr_url_for_branch() is the probe (the open PR for feat/<id>, or "").

This completes the P0 resilience batch

#7 failure classification + backoff (transient retries, terminal blocks)
#8 stuck-coder watchdog (every await in a drive is bounded)
this crash recovery on boot

Together: the loop now survives restarts, hung coders, and transient failures unattended — the difference between "drives features" and "drives a board for days without a babysitter."

Tests

+3 (98 total), all green: pr_url_for_branch found/absent; _recover adopts-PR-else-requeue, and stays resilient to a per-feature error. ruff clean.

🤖 Generated with Claude Code

A drive doesn't survive a server restart, so every in_progress feature the previous
run was building was stranded forever (the puller only claims `ready`). The loop now
runs a one-time recovery before the puller spins (at the top of _run): for each
in_progress feature, if its PR actually got opened (a crash between open_pr and
open_review) adopt it → in_review; otherwise reset it to ready for a clean rebuild
(a stale worktree is cleaned when the puller re-claims). in_review features are left
alone — they have a PR and the webhook/merge-poll resolves them. Recovery is
best-effort per feature (one failure doesn't strand the rest or stop the loop).

worktree.pr_url_for_branch() is the probe (the open PR for feat/<id>, or "").

This completes the P0 resilience batch (failure classification #7, stuck-coder
watchdog #8, crash recovery here): the loop now survives restarts, hung coders, and
transient failures unattended.

Tests: +3 (98 total) — pr_url_for_branch found/absent; _recover adopts-PR-else-requeue
and is resilient to a per-feature error.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@protoquinn

protoquinn Bot commented Jun 13, 2026

Copy link
Copy Markdown

👀 Quinn is reviewing — verdict (PASS / WARN / FAIL) + findings to follow.

@mabry1985
mabry1985 merged commit 62c0ab6 into main Jun 13, 2026
1 check passed
@mabry1985
mabry1985 deleted the feat/crash-recovery-on-boot branch June 13, 2026 05:25

@protoquinn protoquinn Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

QA Audit — PR #9 | feat(loop): crash recovery on boot — resume features stranded mid-drive

VERDICT: WARN (non-blocking — CI still running; re-review pending terminal green)


CI Status

  • test: ⏳ in_progress

Diff Review

  • _recover() in loop.py: correctly scoped — per-feature try/except so one failure never strands others, plus an outer catch in _run() so recovery can never stop the loop from starting. Only touches in_progress features; in_review left alone.
  • pr_url_for_branch() in worktree.py: follows the same _gh / rc == 0 pattern as the adjacent pr_is_merged(). Clean, consistent.

Observations

  • LOW: clawpatch structural review unavailable — repo not in project registry. Diff is small enough (~80 LOC) that manual review is sufficient.
  • Tests (+3, 98 total) cover found/absent PR, adopt-vs-reset, and per-feature error resilience. All green locally per PR description; awaiting CI confirmation.

— Quinn, QA Engineer

@protoquinn

protoquinn Bot commented Jun 13, 2026

Copy link
Copy Markdown

Submitted COMMENT review on protoLabsAI/projectBoard-plugin#9.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant