fix(coordinator): recover non-resident outcome-spec revision gates

[sabbour]

Summary

A coordinator run can report status=in_progress and coordinator_status=awaiting_confirmation with a persisted outcome spec, while every POST /api/runs/{id}/outcome-spec/revise attempt returns 409 run_not_active. The review gate is visible but cannot be exercised.

Live reproduction

• Deployed dev SHA: f9b7e77d2d146bd9c6c2298e8fd3e0a02484a5d1
• Project: 57f08d70-db91-4d08-9d19-d6a28de2594f
• Run: 44c2eed4-7e3e-49a6-89c5-369b381dd94c
• Final run state: in_progress, coordinator status awaiting_confirmation
• Last event: sequence 118, coordinator.outcome_spec
• Transcript: scripts/api-harness/transcripts/jordan-1612-acceptance-20260926T1910Z.jsonl
• Transcript SHA-256: 2CBBC81B2295E929119A9EE196F3F7817B4B75C57185C87D2C707C9828C497FF

Reproduction

1. Create a blank project from blueprint-software-development.
2. Start a defineOutcome orchestration with autopilot=false and auto_approve_tools=false.
3. Wait for the run to persist coordinator.outcome_spec and report awaiting_confirmation.
4. Obtain the guarded execution context requested by the revise endpoint. The endpoint first returns typed model_provider_changed with replacement orchestration context.
5. Resend the same grounded revision with that replacement context.

Actual

The resend returns:

{"error":"run_not_active","detail":null,"message":"The coordinator run is not active and cannot be revised."}

The authoritative run record remains in_progress / awaiting_confirmation, the outcome spec is unchanged, and no event is persisted after sequence 118. This reproduced for two distinct grounded revision requests.

Expected

A run advertised as awaiting_confirmation and steerable must have a durable active confirmation/revision path. The revision should either be accepted and re-draft, or return a typed attributable recovery diagnostic that explains why the gate cannot be resumed. It must not expose an actionable gate that consistently rejects as inactive.

Acceptance

• Reproduce with a restart/replica or non-resident confirmation gate.
• Make revise/confirm recover the durable gate without duplicate draft effects.
• Preserve cancellation and lifecycle-generation fencing.
• Add regression coverage for a persisted awaiting_confirmation run with no resident registry entry.

[sabbour]

Post-deployment acceptance failed

Deployed exact merged commit 6f22dad5b034b551f425017027e406445b540155 passed provenance 4/4, warm pool 2/2, and health 37/37.

A live API run reached durable awaiting_confirmation, then both original API replicas were replaced. The first and only revise request returned HTTP 409:

coordinator_gate_checkpoint_invalid: The outcome-spec gate could not be safely recovered. Inspect the run diagnostic before retrying.

Retained evidence:

• Project: a4dd1120-6b6c-434a-bb71-af4c41dbc2ce
• Run: b9408552-888d-4bac-ad9f-3be6a5c77c67
• Outcome spec: 368
• Last event sequence: 91
• Transcript: scripts/api-harness/transcripts/issue-1622-live-retry-20260927T033220.jsonl
• No redraft occurred.
• No work plan or child run started.

API logs identify the root cause:

System.Text.Json.JsonException: The metadata property is either not supported by the type or is not the first property in the deserialized JSON object. Path: $.workflow.edges.coordinator-draft[0].$type

The recovery path loaded checkpoint 410630e849eb41469deaf87eb97115cd, then failed inside the MAF JsonMarshaller while restoring it. Keep this issue open until a fresh deployment resumes this retained run (or an equivalent fresh run) through revise and confirm exactly once after replica loss.

[sabbour]

Post-deployment acceptance passed

Deployed exact merged commit 764f0c1695874a0f6ece2c045692cc70435a6d78 passed provenance 4/4, warm pool 2/2, and health 37/37.

The retained checkpoint that failed before the repair was reused:

• Project: a4dd1120-6b6c-434a-bb71-af4c41dbc2ce
• Coordinator run: b9408552-888d-4bac-ad9f-3be6a5c77c67
• Outcome spec: 368
• Transcript: scripts/api-harness/transcripts/issue-1622-live-retry-20260927T033220.jsonl

After both original API replicas were replaced, the identical revise request returned HTTP 202 queued. The run produced exactly one new redraft cycle and returned to awaiting_confirmation without replaying the initial draft or starting work.

After both API replicas were replaced again, one confirm request returned HTTP 202 queued. Evidence showed exactly one confirmation event, work plan 244, subtask 495, one dispatch event, and child run b6f3482b-25d7-4ce1-a50e-6606b9b687f6. No duplicate draft, plan, child, or execution start occurred.

This proves revise and confirm recovery across complete replica loss, including backward-compatible restoration of the checkpoint persisted before PR #1628.