You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
No agent that runs guardrail gates treats a non-zero gate as passed on the theory that the failure is environmental or pre-existing. A failing gate is resolved one of two ways: fix the underlying cause, or — when the failure stems from a genuinely broken input or environment — raise a blocker. Classifying a failure as ambient (pre-existing or environmental, rather than caused by the current work) requires reproducing the identical failure on the run's base ref; without that evidence the gate counts as failed. This binds every gate-running agent — writers must not commit around such a failure, reviewers must not approve around it.
Context
Observed in the autonomous run that removed the iAPI Mini-Cart feature flag. The code-reviewer ran the e2e gate (35 pass / 17 fail), classified all 17 failures as environmental, and approved — masking a real regression (the Mini-Cart drawer lost its fixed-position overlay CSS and rendered inline at the bottom of the page). The reviewer leaned on an inverted heuristic — "all the failing tests are ones this batch didn't modify → environmental" — which is backwards: a regression is by definition a previously-passing test that now fails.
Although it surfaced at the reviewer, the same fallacy threatens any gate-running agent. Gates are run by code-writer-tdd, code-writer-e2e, code-reviewer, docs-writer, and docs-reviewer (guardrails convention). A writer faces the mirror-image temptation: commit around a failure it believes was "already broken / not my change." Today agents/code-reviewer.md says a non-zero gate is a rejection finding but doesn't forbid the "environmental" rationalization or define proof of "ambient"; the writer profiles (e.g. agents/code-writer-e2e.md) treat every non-zero gate as the writer's own work to fix and don't contemplate a genuinely pre-existing/ambient failure at all. The e2e-writer in this same run did reproduce the original blocker on the base ref — the technique already exists in the project and should become the standard for everyone.
Related to #155 (reviewers should not approve around an unverified hedge): the "these failures are ambient" dismissal is the same family — an unverified hedge on a load-bearing claim. This issue is the concrete, gate-level case, and it spans writers as well as reviewers.
Assumptions / directions to explore
Open — for later phases to confirm or revise, not requirements.
The rule must be self-contained in each gate-running agent's profile, since profiles can't reference skill files — duplicated across the relevant profiles (code-writer-tdd, code-writer-e2e, code-reviewer, docs-writer, docs-reviewer). The terminal action differs per role (writer: don't commit; reviewer: don't approve), so the principle is stated and each profile states its own action.
Name and forbid the inverted "a failing test the work didn't touch → ambient" heuristic.
Define base-ref reproduction (identical failure) as the only accepted evidence for an "ambient" classification, with a blocker as the fallback when that evidence isn't obtained.
For a reviewer, base-ref reproduction is an escape hatch, not a mandatory step — the safe default for a suspect failure is a blocker, so the env cost never forces a pass.
Standing up the base-ref environment cheaply depends on the worktree + shared-services env work tracked in Initialize worktree #65; this issue stays actionable meanwhile via the blocker fallback.
Goal
No agent that runs guardrail gates treats a non-zero gate as passed on the theory that the failure is environmental or pre-existing. A failing gate is resolved one of two ways: fix the underlying cause, or — when the failure stems from a genuinely broken input or environment — raise a blocker. Classifying a failure as ambient (pre-existing or environmental, rather than caused by the current work) requires reproducing the identical failure on the run's base ref; without that evidence the gate counts as failed. This binds every gate-running agent — writers must not commit around such a failure, reviewers must not approve around it.
Context
Observed in the autonomous run that removed the iAPI Mini-Cart feature flag. The
code-reviewerran the e2e gate (35 pass / 17 fail), classified all 17 failures as environmental, and approved — masking a real regression (the Mini-Cart drawer lost its fixed-position overlay CSS and rendered inline at the bottom of the page). The reviewer leaned on an inverted heuristic — "all the failing tests are ones this batch didn't modify → environmental" — which is backwards: a regression is by definition a previously-passing test that now fails.Although it surfaced at the reviewer, the same fallacy threatens any gate-running agent. Gates are run by
code-writer-tdd,code-writer-e2e,code-reviewer,docs-writer, anddocs-reviewer(guardrails convention). A writer faces the mirror-image temptation: commit around a failure it believes was "already broken / not my change." Todayagents/code-reviewer.mdsays a non-zero gate is a rejection finding but doesn't forbid the "environmental" rationalization or define proof of "ambient"; the writer profiles (e.g.agents/code-writer-e2e.md) treat every non-zero gate as the writer's own work to fix and don't contemplate a genuinely pre-existing/ambient failure at all. The e2e-writer in this same run did reproduce the original blocker on the base ref — the technique already exists in the project and should become the standard for everyone.Related to #155 (reviewers should not approve around an unverified hedge): the "these failures are ambient" dismissal is the same family — an unverified hedge on a load-bearing claim. This issue is the concrete, gate-level case, and it spans writers as well as reviewers.
Assumptions / directions to explore
Open — for later phases to confirm or revise, not requirements.
code-writer-tdd,code-writer-e2e,code-reviewer,docs-writer,docs-reviewer). The terminal action differs per role (writer: don't commit; reviewer: don't approve), so the principle is stated and each profile states its own action.