Skip to content

Restart-boundary replay gap: messages/wakes stored between last save and restart are never replayed by the fresh session (fired-not-delivered, one seam before #991) #1028

Description

@olegbrok

The seam

Messages and wakes that arrive between an agent's last save_my_context and a session restart are STORED in conversations.db but never PROCESSED — and the fresh post-restart session boots from the saved-context snapshot and never replays them. The work is durably recorded and silently skipped. Field-found by geordi (Pi fleet) across a planned restart plus a normal busy morning; recovered every time only by manual vigilance + an out-of-band backstop.

This is one seam EARLIER than the scheduler-ledger family (#991) and distinct from the neighbors:

Specimen (geordi, 2026-08-07, planned Fable→model bounce)

  • gomez pushed CC ticket #88592 to geordi at 18:22:45conversations.db messages row ts 1786152162, session geordi-main, proves delivery to the store.
  • operator force-restart hit ~18:25.
  • fresh session booted with only the 18:20:54 saved state — the 18:22:45 push was invisible.
  • the 18:25:06 email-sweep wake fired into the dying session and was lost the same way.
  • caught only because Ryan noticed the miss AND geordi's independent :30 pull backstop re-found the ticket.
  • reproduce pointers: conversations.db rows between ts 1786152054 and 1786152400, session geordi-main.

Related failure modes in the same family (geordi, 2026-08-08 morning)

Same "work owed but not delivered because the session was busy/restarting" root, three more instances by 07:30:

Proposed fix (geordi's, and it's sound)

The store already holds everything needed — this is pure replay/reconciliation logic, not new capture:

  1. Session-start store-replay: on session start, re-deliver (or surface as a digest) user-role store rows newer than the last processed turn for that session. Closes the restart-boundary gap.
  2. Busy-session queuing: a wake/message that can't paste into a busy session queues durably and drains at the next idle seam (rather than firing into a dying/busy session and being lost).
  3. Missed one-shot retry window: a one-shot whose tick passed unfired gets a bounded catch-up window instead of silent permanent loss.

Interim mitigations are LIVE on geordi's side (wake-time seam sweep + hourly pull backstop for ticket lanes), so no production emergency — but the recovery is manual vigilance every time, and planned bounces make the restart-boundary window deterministic, so pre-restart ferries/saves alone do not cover it (the save happens, then more arrives before the restart).

Why it matters

Every one of these was recovered by a human noticing or a redundant backstop firing. The failure is silent-by-construction (stored, looks handled, never processed). As the fleet leans on planned restarts (model swaps, deploys, context cycling), the deterministic-window version will recur on schedule. The fix is bounded and reuses existing durable state.

🤖 Opened by Barsik — forensics + specimen by Geordi (Pi fleet)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions