fix(qa-gate): terminate no-FAIL diagnosis phase instead of looping (#655 sub-defect B)#657
fix(qa-gate): terminate no-FAIL diagnosis phase instead of looping (#655 sub-defect B)#657halindrome wants to merge 5 commits into
Conversation
…wt-labs#655 sub-defect B) The QA remediation gate had no clean terminal state for a phase that legitimately has zero FAIL checks and zero code changes — a read-only diagnosis / documentation-disposition round. `source_verification_missing` was exempted for `input_mode=none` only at `_gate_stage=done`, but a round must clear the `verify` stage to reach `done`. So such a round was flagged `qa_gate_source_verification_missing=true` at `verify` and routed `REMEDIATION_REQUIRED` forever; the only escape observed in the field was manually advancing `verify -> done`. Add a stage-agnostic exemption that fires only when the round genuinely has nothing to remediate, using a precise predicate (not the corroboration- dependent METADATA_ONLY_ROUND, which reads non-metadata edits as metadata-only when no git evidence is available): - input_mode=none and no source verification path, AND - zero fail-classifications and zero registry known issues, AND - the recorded files are all planning metadata, AND - the phase's authoritative verification resolves on disk and is a clean PASS with zero FAIL rows and zero pre-existing/known issues (new helper `phase_baseline_is_clean`). Rounds that carry fail-classifications, carry known issues, recorded any non-metadata edit, have no phase verification on disk (a continuation round that lost its source), or whose phase verification carries pre-existing issues do NOT satisfy the predicate and still fail closed. Scope: addresses sub-defect B only. Sub-defect A (change-evidence under planning_tracking=manual) does not reproduce on current main — plan-amendment evidence is corroborated under both tracking modes and CHANGE_EVIDENCE_ UNAVAILABLE only routes REMEDIATION when ROUND_CODE_FIX_COUNT>0. Sub-defect C (deviation_override / accepted-deviation disposition) is a separate concern. Tests: - qa-result-gate.bats: the no-classification metadata-only round on a clean baseline now PROCEEDs (was the loop); a non-metadata edit on a clean baseline still fails closed. Existing fail-closed cases (missing phase verification, missing prior-round verification, carried known issues, classified process-exceptions) remain REMEDIATION_REQUIRED. Refs swt-labs#655. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Addresses the findings from the round-1 QA review (read-only adversarial review of the sub-defect B fix): - phase_baseline_is_clean pre-existing-issues counter miscounted GitHub-style colon alignment rows and non-canonical header rows (would silently defeat the fix for those formats) — replaced with the header-state-machine pattern (skip alignment rows via [[:space:]:|-], treat the first non-separator row as the header regardless of titles). - phase_baseline_is_clean result check was narrower than the gate's authoritative RESULT parser — now normalizes with toupper() and falls back to the legacy `status:` field. - gate-level coverage for the reachable phase_baseline_is_clean fail-closed branches (absent phase verification, non-PASS result, pre-existing issues); removed a misconceived test for the count_fail==0 branch, which is unreachable through the gate because a phase verification with FAIL rows makes qa-remediation-state derive input_mode=verification (documented as defense-in-depth instead). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Round 2 QA review came back clean — no blockers or major issues. The round-1 fixes hold; 187/187 bats pass, shellcheck clean. One minor, pre-existing, non-regressing observation noted and deferred: `count_fail_rows_in_verification` keys on a literal "Status" column header, so a self-contradictory verification (frontmatter result: PASS but a FAIL row under a differently-named header) is read as 0 FAILs. This is the established gate-wide parser (not introduced here), real write-verification.sh output always uses the "Status" header, and the direction is benign — out of scope for swt-labs#655. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Round 3 QA review surfaced two minor, fail-safe findings; both addressed: - Finding 2 (placeholder pre-existing table): `phase_baseline_is_clean` counted a placeholder "None"/"N/A" row in a `## Pre-existing Issues` table as a real carried issue, so a clean no-FAIL phase that renders an explicit empty table failed closed. The counter now skips placeholder rows (None / N/A / "no ... issues"), matching the placeholder-skipping convention used elsewhere in the gate. Fail-safe direction either way (could only have caused a spurious remediation, never a false PROCEED). - Finding 1 (round-02+ coverage gap): all prior swt-labs#655 tests used round-01, leaving the new exemption's round>1 source-resolution path unverified. Added a round-02 no-classification clean-baseline test asserting PROCEED_TO_UAT, locking in the deliberate disposition broadening. 189/189 bats pass; shellcheck + bash -n clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
d7d7d39 to
b1b935c
Compare
|
QA Review — Round 1 (against base fix Addressed in |
|
QA Review — Round 2 (fresh review against Clean round → empty evidence commit |
|
QA Review — Round 3 (fresh review against Both findings addressed in |
What
Give the QA remediation gate (
scripts/qa-result-gate.sh) a clean terminal state for a phase that legitimately has zero FAIL checks and zero code changes — a read-only diagnosis / documentation-disposition round — so it terminates (PROCEED_TO_UAT) instead of looping forever. Addresses sub-defect B of #655.Why
source_verification_missingwas exempted forinput_mode=noneonly at_gate_stage=done, but a round must clear theverifystage to reachdone. A no-FAIL disposition round was therefore flaggedqa_gate_source_verification_missing=trueatverifyand routedREMEDIATION_REQUIREDforever; the only escape observed in the field was manually advancingverify → done. A reproduction confirmed the loop is real and identical under bothplanning_tracking=manualandignore.How
A stage-agnostic exemption that fires only when the round has genuinely nothing to remediate. The naive "drop the
doneconjunct" approach breaks 7 fail-closed tests, so the predicate is precise:input_mode=noneand no source verification path, andMETADATA_ONLY_ROUNDis corroboration-dependent and misreads non-metadata edits as metadata-only when no git evidence is available), andphase_baseline_is_clean).Rounds that carry fail-classifications, carry known issues, recorded any non-metadata edit, have no phase verification on disk (a continuation round that lost its source), or whose phase verification carries pre-existing issues do not satisfy the predicate and still fail closed.
Scope: sub-defect B only. Sub-defect A (change-evidence under
planning_tracking=manual) does not reproduce on currentmain— plan-amendment evidence is corroborated under both tracking modes andROUND_CHANGE_EVIDENCE_UNAVAILABLEonly routesREMEDIATIONwhenROUND_CODE_FIX_COUNT>0. Sub-defect C (deviation_override / accepted-deviation terminal disposition) is a separate concern needing a new verdict. See the issue thread for detail.One existing test legitimately changed: a no-classification metadata-only round on a clean-baseline phase with a historical phase-root SUMMARY deviation now PROCEEDs — consistent with the gate's existing treatment of phase-root deviations as historical during remediation (they are not counted in the round-scoped deviation override).
Testing
phase_baseline_is_cleanhelper + the exemption;bash -n+shellcheck -S warningclean.tests/qa-result-gate.bats: the no-FAIL disposition round now PROCEEDs (was the loop); reachablephase_baseline_is_cleanfail-closed branches covered (absent phase verification, non-PASS result, pre-existing issues); a non-metadata edit still fails closed. 187 ok / 0 fail.testing/run-lint.sh: clean.verifyunder bothplanning_trackingmodes; all prior fail-closed cases remainREMEDIATION_REQUIRED.Refs #655.
🤖 Generated with Claude Code