Skip to content

fix: triage reported its own failures as work waiting for a maintainer - #187

Merged
sjh9714 merged 1 commit into
mainfrom
fix/triage-incomplete-read
Aug 6, 2026
Merged

fix: triage reported its own failures as work waiting for a maintainer#187
sjh9714 merged 1 commit into
mainfrom
fix/triage-incomplete-read

Conversation

@sjh9714

@sjh9714 sjh9714 commented Aug 6, 2026

Copy link
Copy Markdown
Owner

What a first-time user actually got

Running the command exactly as the README documents it, on a public repository,
with no token:

20 open pull request(s) read. 14 have something a maintainer checks by hand.

#10586  gci-fun-1                    could not be read
#12046  feat/detect-nested-any-in-…  could not be read
#12086  strict-void-return-fixer     could not be read
...

With a token, the same repository returns five rows. Nine of those fourteen
were not pull requests needing attention. They were GitHub refusing the request:
60 unauthenticated requests an hour, which one queue uses up.

Why it happened

Every failure became notes: ["could not be read"], which is the same shape a
finding has. So an unreadable pull request was counted in the headline, sorted
among real findings, and the process exited 0.

This repository states that it fails closed rather than presenting a partial
pass, and ships an ANALYSIS INCOMPLETE state for exactly this situation. The
command people are told to run first was the one place that ignored it.

What changed

  • Unreadable pull requests are tracked separately from findings, reported on
    their own line, excluded from the count, and the run exits non-zero.
  • When nothing could be read at all, that goes to stderr instead of a
    description of a queue the tool never saw.
  • An exhausted quota stops the loop rather than attempting every remaining pull
    request, since they fail identically.
  • The advice differs by cause: set GH_TOKEN when unauthenticated, wait or
    lower --limit when it was the token's own quota. The listing no longer tells
    somebody who has not set a token that GitHub is "rate limiting this token".
  • The listing no longer retries an exhausted hourly quota. That backoff waits
    20, 40 and 60 seconds, so an unauthenticated caller sat through two minutes of
    silence before getting the error anyway. GitHub marks this case with
    x-ratelimit-remaining: 0; the secondary rate limit, which does recover and
    is still retried, does not.

Why the tests did not catch it

runTriageCli was never executed by the suite. Every triage test was either a
pure function or an assertion about the source text, so all 90 passed while the
command's error handling was wrong.

There is now packages/cli/test/triageRun.test.ts, which runs it against
stubbed GitHub calls. Five of its seven cases fail against the previous
code
, verified by stashing the source change and re-running:

× does not count a pull request it could not read as work waiting for a maintainer
  AssertionError: expected '3 open pull request(s) read. 3 have s…'
                  to contain '2 open pull request(s) read. 2 have s…'
× stops at an exhausted quota rather than blaming every remaining pull request
  AssertionError: expected "vi.fn()" to be called 1 times, but got 4 times

Documentation

The README, getting started, and docs/triage.md all said a token was for
private repositories and higher rate limits. For this command it is required,
and they now say so, including that a token with no scopes is enough.

Verification

pnpm build && pnpm test && pnpm typecheck && pnpm lint && pnpm format:check,
all green. 461 tests, up from 453.

Running the command exactly as the README documents it, on a public repository,
with no token:

    20 open pull request(s) read. 14 have something a maintainer checks by hand.

    #10586  gci-fun-1                    could not be read
    #12046  feat/detect-nested-any-in-…  could not be read
    #12086  strict-void-return-fixer     could not be read
    ...

With a token the same repository returns five rows. Nine of those fourteen were
not pull requests needing attention, they were GitHub refusing the request:
60 unauthenticated requests an hour, which one queue uses up.

Every failure became `notes: ["could not be read"]`, which is the same shape a
finding has. So an unreadable pull request was counted in the headline, sorted
among real findings, and the process exited 0. This repository states that it
fails closed rather than presenting a partial pass, and has an ANALYSIS
INCOMPLETE state for exactly this. The command people are told to run first was
the one place that ignored it.

Unreadable pull requests are now tracked separately from findings, reported on
their own line, excluded from the count, and the run exits non-zero. When
nothing could be read at all it says so on stderr rather than describing a queue
it never saw.

An exhausted quota now stops the loop instead of attempting every remaining pull
request, since they all fail identically, and the advice differs by cause: set
GH_TOKEN when unauthenticated, wait or lower --limit when it was the token's own
quota. The listing no longer claims to be "rate limiting this token" at somebody
who has not set one.

The listing also no longer retries an exhausted hourly quota. That backoff waits
20, 40 and 60 seconds, so an unauthenticated caller sat through two minutes of
silence before getting the error anyway. GitHub marks this case with
x-ratelimit-remaining: 0, which the secondary rate limit does not.

Tests: runTriageCli was never executed by the suite, which is why this shipped.
There is now an integration test file that runs it against stubbed GitHub calls.
Five of its cases fail against the previous code.

README, getting-started and docs/triage.md said a token was for private
repositories and higher limits. For this command it is required.
@sjh9714
sjh9714 merged commit 38dba5b into main Aug 6, 2026
8 checks passed
@sjh9714
sjh9714 deleted the fix/triage-incomplete-read branch August 6, 2026 07:05
sjh9714 added a commit that referenced this pull request Aug 6, 2026
… for its own failures (#189)

npm still serves 0.10.1, where running the command this project tells people to
run first, on a public repository, with no token, prints:

    20 open pull request(s) read. 14 have something a maintainer checks by hand.
    #10586  gci-fun-1    could not be read
    ...

and exits 0. Nine of those fourteen were GitHub refusing the request after 60
unauthenticated calls an hour, not pull requests needing attention. The fix
merged in #187 and has not reached anybody.

Also ships the README rewrite from #188: it leads with the check nothing else
appears to make, and install drops from four options to one because three of
them were doing nothing.

Verified in the order the last release taught. `pnpm audit` with no severity
filter was run before the tag, not after: 0.10.0 was tagged and then rejected by
the release gate for two moderate advisories, which is how a tag ends up
pointing at something that never shipped.

Checked against the packed tarballs rather than the source tree, since the
source has always worked and that is exactly what hid this: cold install into an
empty directory reports 0.10.2 and triages a real repository. No workspace:* and
no private runtime dependency; 190 KB and 92 KB compressed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant