Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 24 additions & 4 deletions scripts/check-main-verified.sh
Original file line number Diff line number Diff line change
Expand Up @@ -61,10 +61,10 @@
#
# ── It fails OPEN, at every unknown ──────────────────────────────────────────
#
# No `gh`, an unreachable API, an unparseable answer: report and exit 0. This
# is a monitor, and a monitor that can itself block a merge is worse than the
# gap it watches — the same argument `main-red-claim.sh`'s header makes about
# a claim check. Every unknown is loud, never silent.
# No `gh`, an unreachable API, an empty run list, an unparseable answer: report
# and exit 0. This is a monitor, and a monitor that can itself block a merge is
# worse than the gap it watches — the same argument `main-red-claim.sh`'s
# header makes about a claim check. Every unknown is loud, never silent.
#
# ── `--announce` reaches a person, on codeql-canary.sh's shape ───────────────
#
Expand Down Expand Up @@ -233,6 +233,26 @@ elif ! runs="$(gh run list --workflow ci.yml --branch main --limit 60 \
unknown "could not reach the Actions API"
fi

# Zero rows answers a different question than it looks like it answers. The
# loop below opens every commit at `missing` and only a matching row moves it,
# so an empty list does not report one absence — it reports the whole window as
# absent, in one breath, with no row anywhere to contradict it. A read that
# returned nothing cannot distinguish "no `ci` run exists for any of these
# commits" from "the API declined to list them", and those are opposite states.
#
# `gh` exits 0 on an empty page, so the check above never sees it. On
# 2026-09-09 one such page filed against the ten newest commits at once; each
# of them had a completed, successful run, and the same query twenty minutes
# later returned all sixty rows. The header's rule decides it: every unknown
# exits 0 and says so, because a monitor that fabricates is one nobody reads
# the second time.
#
# This is only the empty page. A list that carries rows but none for a given
# commit is the outage this script was written to catch, and still reports.
if [ -z "$runs" ]; then
unknown "the run list came back empty — this read listed no ci run at all for main, which cannot tell an outage from an unanswered query"
fi

now_epoch="$(date -u +%s 2>/dev/null || echo 0)"

# Seconds since an RFC-3339 timestamp, or 0 when this platform's `date` will
Expand Down
29 changes: 29 additions & 0 deletions scripts/test-main-verified.sh
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,35 @@ case "$out" in
*) fail=$((fail + 1)); echo "FAIL an unknown exited 0 without saying so:"; echo "$out" ;;
esac

# Zero rows is the API declining to answer, not an answer that no `ci` run
# exists for any of these commits. Read the second way, one empty page turns
# every commit in the window into a finding at once: on 2026-09-09 that filed
# against ten commits which each had a completed, successful run, and a monitor
# that files falsely is one nobody reads twice.
empty_runs_commits="$(commit $A 'a merge whose run the API did not list')
$(commit $B 'and the merge before it')"
if out="$("$SCRIPT" --fixture-commits "$empty_runs_commits" --fixture-runs "" 2>&1)" &&
[ -z "${out##*UNKNOWN*}" ]; then
pass=$((pass + 1)); echo "ok an empty run list is UNKNOWN, not every commit missing"
else
fail=$((fail + 1)); echo "FAIL an empty run list was read as a finding:"; echo "$out"
fi

# The report names the shape it could not read, so a reader of the log can
# tell this apart from the outage the script exists to catch.
case "$out" in
*"listed no ci run at all"*) pass=$((pass + 1)); echo "ok ...and names an empty run list as the reason" ;;
*) fail=$((fail + 1)); echo "FAIL the unknown did not say the run list was empty:"; echo "$out" ;;
esac

# The other direction, so the guard above cannot be satisfied by refusing to
# answer whenever a commit has no run: a NON-empty list that simply carries
# nothing for this commit is still the outage this script was written for.
want "a run list carrying nothing for the commit is still unverified" expect-fail \
"$(commit $B 'a merge with no run of its own')" \
"$A completed success $(now_iso)" \
"missing"

# ── N: several commits, one bad ──────────────────────────────────────────────
# The reported window is a run of merges, not one; a guard that stopped at the
# first verified commit would have missed three of the four that day.
Expand Down
Loading