Skip to content

An empty releases read makes check-releases-published report every tag as never published #6481

Description

@macanderson

The problem

scripts/check-releases-published.sh cannot tell "no release exists for any of
these tags" from "the releases API returned an empty page", and it answers the
first way. One empty read turns every tag in the repository into a finding at
once.

The read is unguarded:

pages_json="$(CLICOLOR_FORCE=0 NO_COLOR=1 gh api --paginate --slurp 'repos/{owner}/{repo}/releases?per_page=100')"
...
releases_json="$(printf '%s' "$pages_json" | jq '[ .[][] | { name: .tag_name, draft: .draft } ]')"

gh exits 0 on an empty page, so set -euo pipefail does not catch it and
releases_json becomes []. select_unpublished then builds $published
from that empty list:

( [ .releases[] | select(.draft != true) | { (.name): true } ] | add // {} ) as $published
| [ .tags[] | select($published[.name] | not) | select($known[.name] | not) | ... ]

With $published empty, every tag past the grace window that is not in
scripts/releases-baseline.txt reports as absent, and the script exits 1
with a report claiming the release pipeline shipped nothing. This repository
has several hundred published releases, so that report would be wrong in a way
that is loud and alarming rather than quiet.

This is the same shape as the defect fixed in #6479, found by sweeping the
sibling guards after it: there, check-main-verified.sh read one empty
gh run list page and filed against the ten newest commits on main, each of
which had a completed, successful run.

Labelled as an inference, not an observation: I read this off the code and
confirmed the jq behaviour, but I have not observed this script
misreport. What I did observe is the same API returning an empty page under
gh run list on 2026-09-09, which is what makes the path reachable rather
than theoretical.

Why this needs a maintainer call rather than a fix in the sweeping PR

The obvious guard — refuse when zero releases come back — collides with the
script's own purpose. "Many tags, zero releases" is also the genuine
catastrophic finding it exists to report: a repository whose release pipeline
never worked looks exactly like a repository whose releases API just returned
an empty page. Nothing in the payload separates them; a broken read and a real
total failure both arrive as [] or [[]].

So the decision is which cost to pay, and it is not mine to take:

  • Refuse and say so, on the shape of the max_pages sanity ceiling six
    lines above it, which already exits 1 with ::error:: and "Investigate
    before trusting this run's answer." Cost: a genuinely release-less
    repository gets "investigate" forever instead of a report.
  • Report but mark it low-confidence, so a total-failure claim is
    distinguishable in the log from an ordinary one.
  • Re-read once before believing an empty page, which costs one API call
    and closes the transient case without touching the real-failure case.

The third looks strongest to me, and it is the only one that does not trade
away a real finding — but it is a behaviour change to a scheduled audit and
wants a decision on the record.

Files

  • scripts/check-releases-published.sh — the unguarded read, and
    select_unpublished where an empty $published fans out over every tag.
  • scripts/test-releases-published-pagination.sh — the witness seam already
    exists here: write_shim <dir> <pages> <per_page> puts a fake gh on
    PATH. write_shim "$work/empty" 1 0 produces the one-empty-page shape.
  • scripts/releases-baseline.txt — the grandfathered set that would be the
    only thing keeping tags out of the false report.

How to reproduce

Add an arm to the pagination suite with a shim returning one empty page, and a
grace window narrow enough that this checkout's real tags fall outside it (the
existing arms use ten years precisely so that no real tag is reachable):

write_shim "$work/empty" 1 0
run_with_fake_gh "$work/empty" "$SCRIPT" --grace-secs 0

On today's code that prints a report naming every non-grandfathered tag in the
repository and exits 1.

Definition of done

  • An empty releases read no longer produces a report that names every tag.
  • A genuine "tags exist, nothing was ever published" state is still
    reported — whichever option above is chosen, this case does not go
    silent.
  • A witness arm in scripts/test-releases-published-pagination.sh that
    fails on today's code and passes on the fix, plus a negative control
    keeping the real-failure case reportable.
  • The chosen behaviour is written into the script's header beside the
    max_pages reasoning, so the next reader sees why an empty page is
    treated the way it is.

Pillar

Reliability, and auditability with it. A monitor that can fabricate a
repository-wide failure is one nobody reads the second time, which is the
argument check-main-verified.sh's own header already makes and the reason
#6479 exists.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    triageUntyped request — convert by adding bug / feature / epic

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions