fix(scripts): stop an empty ci run list reading as ten unverified commits - #6477
Conversation
…mits `check-main-verified.sh` reads the run history with one `gh run list` call and defaults every commit to `missing` when no run in that list matches it. An empty list therefore matches nothing, and each of the ten commits it checks becomes a separate finding. `gh` returns `[]` with exit 0 for a transient API fault as readily as for a genuinely empty history. On 2026-09-09 one such read failed the canary's `main carries no unverified commit` job and filed a tracking issue saying nothing had verified ten commits — while `ci` on the tip, cd7d53c, had already concluded success. An auth or permission failure exits non-zero and was already caught as an unknown; this shape was not. The two cases are separable: the read asks for the last 60 `ci` runs on `main` whatever their age, and an outage creates no runs and destroys none, so it cannot empty that list. Empty means the run could not see the history, which is an unknown and exits 0 — the discipline this script's own header already states. A commit absent from a populated list is still reported, which is the gap the script watches, and a case pins that so the fix cannot buy its safety with the coverage. Witness: the three new cases in scripts/test-main-verified.sh fail on the previous script (exit 1, "FAILED — main carries commits nothing verified") and pass on this one. Suite: 38 passed, 0 failed. Closes #6475
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Sorry @macanderson, you've used your own review budget of 250,000 diff characters for the last 7 days.
You can request another review in 4 days and 1 hour by commenting @sourcery-ai review. Upgrade to get a review now.
Reviewer's GuideThe checker now distinguishes an empty, successful API response from a populated run history: empty results are treated as a loud, fail-open unknown and cannot generate false findings, while missing runs in populated results remain reportable. The test suite adds regression coverage for both behaviors. Flow diagram for empty CI run list handlingflowchart TD
A["Read CI runs with gh run list"] --> B{"gh command succeeded?"}
B -- "No" --> C["Report unknown"]
B -- "Yes" --> D{"run list empty?"}
D -- "Yes" --> C
D -- "No" --> E["Check commits against populated history"]
E --> F{"completed run found?"}
F -- "Yes" --> G["Commit verified"]
F -- "No" --> H["Report missing commit"]
File-Level Changes
Assessment against linked issues
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Answering Sourcery's ❌ row
This row asks for work that is not owed, because the condition it names never That objective was read out of the "How to fix" section of #6475. The Here is the run history for the exact ten commits #6475 named. The canary
Nine of the ten had already concluded So the correct verdict at that moment was nine verified and one pending, which Re-dispatching anything now would spend CI runs re-asking a question that has |
1 participant
What this fixes
scripts/check-main-verified.shreads the CI history with onegh run listcall, then defaults each commit to
missingwhen no run in that list matchesit. An empty list matches nothing, so all ten commits it checks become ten
separate findings at once.
ghreturns[]with exit 0 for a transient API fault as readily as for agenuinely empty history. On 2026-09-09 one such read failed
main-canary.yml'smain carries no unverified commitjob and filed #6475, which named tencommits as unverified — while
cirun 34379997528 on the tip it named,cd7d53c, had already concluded
success. Every one of those ten commits has acompleted
cirun, then and now.An auth or permission failure was already handled:
ghexits non-zero thereand the script takes its
unknownpath. I checked that directly — the samequery under an invalid token exits 1, which is why this run cannot have been a
permission problem and must have been an empty answer with exit 0.
The fix
An empty run list becomes an
unknown: reported loudly, exit 0. That is thediscipline the script's own header already states for every other unanswerable
read, and the header now names this one.
The two cases are separable rather than conflated. The read asks for the last
60
ciruns onmainwhatever their age, and an outage creates no runs anddestroys none, so it cannot empty that list — older runs stay in it. Empty
therefore means this run could not see the history at all.
The fix does not buy its safety with the coverage it exists to provide: a
commit absent from a populated list is still reported, and a new case pins
that direction alongside the three new ones.
Witness
The three new cases in
scripts/test-main-verified.shfail on the previousscript and pass on this one. Checked by restoring
HEAD's copy of the scriptand running the new suite against it:
With the fix in place:
main-verified: 38 passed, 0 failed.guard-self-tests.ymlruns this suite onpull_requestwith no paths filter,so the witness runs in CI.
Checks run locally
./scripts/test-main-verified.sh— 38 passed, 0 failedshellcheckon both changed scripts — cleanmake prose— OK, none addedmake line-citations— OK, none addedmake guards-fast— greenNothing compiled: this change is two shell scripts.
Why this closes #6475
#6475 is the false report itself, filed by the defect this PR removes. Its
Definition of done — every recent commit on
mainhas a completedcirun —was already true when the issue was filed and is true now; I re-ran the fixed
checker against the real repository to confirm before ticking the box:
Not in this PR
mainalso has a second, unrelated red: thedocsworkflow'sdeploy stella.oxagen.shjob fails withThat is IAM drift in AWS account 916294258235, not a defect in this tree —
docs.ymlnames the same instance id the infra'sterraform.tfvarsdeclares,and the Terraform does grant
ssm:SendCommandon it. Repairing it needs aproduction apply in an account this repository cannot reach, so it is filed as a handoff
in #6478 rather than fixed here.
Closes #6475