Skip to content

Follow-up: capture durable QA evidence for batch closeout #389

Description

@justin808

Finding

The post-merge audit for batch cpflow-justin808-open-b8db found no high-risk merged-code issue, but the required QA lane did not leave durable, current evidence.

Observed state from the audit:

  • #347 and #352 were merged with checks green and zero unresolved review threads.
  • #363 remained open, approved, and maintainer-gated because it touches workflow/config behavior.
  • #340 and #349 were closed with evidence.
  • #365 and #366 remained open intentionally as product/platform tracks.
  • The batch required a QA lane, but durable public QA evidence was missing or stale.

Evidence

  • agent-coord-bounded --timeout 20 doctor --json timed out: agent-coord-bounded: timed out after 20.0s: agent-coord doctor --json.
  • Targeted coordination status for cpflow-justin808-open-b8db was readable, but stale:
    • pr352-grafana still showed pushed-pending-ci after #352 had merged.
    • qa still showed in_progress and blocked on lanes that had reached final public states.
  • GitHub searches found no public QA Evidence, cpflow-c304-qa, or codex-claim evidence for this batch.
  • Audit ledger: not applicable (non-release audit).

Process Gap Disposition

  • Mechanism target: checklist+replay
  • Motivating miss: required QA lane stayed stale or undocumented after merged PRs and no-PR lanes reached final states.
  • Replay evidence or park reason: replay the batch closeout against cpflow-justin808-open-b8db; the mechanism should fail if QA status remains in_progress, if coordination says a lane is pending after the linked PR is merged, or if no durable QA Evidence block names the audited range/heads.
  • Non-goal: do not require QA to duplicate every worker detail when PR checks, resolved review threads, and no-PR evidence already prove the merge gate.

Desired Outcome

Add or adjust the closeout checklist/replay mechanism so future required-QA batches cannot finish with stale coordination state and no durable QA evidence. A good result is a lightweight check that names the missing QA evidence or stale lane before archive/closeout.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions