Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 24 additions & 4 deletions scripts/check-main-verified.sh
Original file line number Diff line number Diff line change
Expand Up @@ -61,10 +61,11 @@
#
# ── It fails OPEN, at every unknown ──────────────────────────────────────────
#
# No `gh`, an unreachable API, an unparseable answer: report and exit 0. This
# is a monitor, and a monitor that can itself block a merge is worse than the
# gap it watches — the same argument `main-red-claim.sh`'s header makes about
# a claim check. Every unknown is loud, never silent.
# No `gh`, an unreachable API, an unparseable answer, a run list that came back
# empty so no commit could be matched: report and exit 0. This is a monitor,
# and a monitor that can itself block a merge is worse than the gap it watches
# — the same argument `main-red-claim.sh`'s header makes about a claim check.
# Every unknown is loud, never silent.
#
# ── `--announce` reaches a person, on codeql-canary.sh's shape ───────────────
#
Expand Down Expand Up @@ -233,6 +234,25 @@ elif ! runs="$(gh run list --workflow ci.yml --branch main --limit 60 \
unknown "could not reach the Actions API"
fi

# An empty list is a blind read, not a finding about every commit at once.
#
# `gh` answers a transient API fault with `[]` and exit 0 as readily as it
# answers a genuinely empty history, and the loop below defaults each commit
# to `missing`. So on 2026-09-09 one bad read reported all ten commits as
# unverified and filed an issue saying nothing had checked a tree whose `ci`
# run on cd7d53c had already concluded success. An auth or permission failure
# exits non-zero and is already caught above; this is the shape that does not.
#
# The two cases are separable because the read asks for the last 60 `ci` runs
# on `main` whatever their age. An outage creates no runs, and destroys none,
# so it cannot empty that list — older runs stay in it, and a commit absent
# from a populated list is still reported, which is the gap this script
# watches. Empty means this run could not see the history at all, and that is
# an unknown.
if [ -z "$runs" ]; then
unknown "the ci run list came back empty, so no commit could be matched"
fi

now_epoch="$(date -u +%s 2>/dev/null || echo 0)"

# Seconds since an RFC-3339 timestamp, or 0 when this platform's `date` will
Expand Down
38 changes: 38 additions & 0 deletions scripts/test-main-verified.sh
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,44 @@ case "$out" in
*) fail=$((fail + 1)); echo "FAIL an unknown exited 0 without saying so:"; echo "$out" ;;
esac

# ── E: an empty run list is a blind read ─────────────────────────────────────
#
# `gh run list` returns `[]` with exit 0 for a transient fault as readily as
# for an empty history, and the verdict loop defaults every commit to
# `missing`. On 2026-09-09 that turned one bad read into ten findings and a
# filed issue claiming nothing had verified a tree whose `ci` run had gone
# green seventeen seconds earlier. An auth failure exits non-zero and was
# already an unknown; this shape was not.
empty_list_commits="$(commit $A 'a merge the API could not answer for')
$(commit $B 'the merge before it')"

want "an empty run list is an unknown, not a finding" expect-pass \
"$empty_list_commits" "" \
"UNKNOWN"

want "...and says the list is what came back empty" expect-pass \
"$empty_list_commits" "" \
"came back empty"

# It also files no issue, which is the half of a misreport somebody has to
# undo by hand.
out="$("$SCRIPT" --fixture-commits "$empty_list_commits" --fixture-runs "" \
--announce --dry-run 2>&1)"
case "$out" in
*"issue create"*)
fail=$((fail + 1)); echo "FAIL an empty run list filed an issue:"; echo "$out" ;;
*) pass=$((pass + 1)); echo "ok ...and files no issue over a read it could not make" ;;
esac

# The coverage this must not buy its safety with. A populated list that simply
# has no run for the commit is the real gap (f1f36660's shape above), and it
# stays a failure — the guard turns on the list being empty, never on a commit
# being absent from it.
want "a commit missing from a POPULATED list is still reported" expect-fail \
"$(commit $A 'a merge nothing ran for')" \
"$B completed success $(now_iso)" \
"missing"

# ── N: several commits, one bad ──────────────────────────────────────────────
# The reported window is a run of merges, not one; a guard that stopped at the
# first verified commit would have missed three of the four that day.
Expand Down
Loading