docs(verify): compare artifact identity, not reported description - #17
Open
unforced wants to merge 2 commits into
Open
docs(verify): compare artifact identity, not reported description#17unforced wants to merge 2 commits into
unforced wants to merge 2 commits into
Conversation
Three failures in one day all shared a root: the tooling described intent while reality had drifted, and every surface signal stayed green. - hub served the app from bun's install cache (a months-old published version) for nine hours while `parachute status` reported the bun-linked checkout — true of resolution, false of what was served - five packages sat merged-but-unpublished; nothing asserts merged==published, so a security bump was merged, tagged, changelogged, and in effect nowhere - that release had failed at the registry PUT five days earlier — red run, no alert, everyone assumed it shipped Adds the counter-discipline: fetch the artifact and diff it (served hash vs dist/, `npm pack` tarball vs source, git tags vs npm versions), and ask which path a reported version was read from. Plus the probe gotcha that cost real time — a minified bundle is one ~540 KB line, so shell command substitution mangles it and yields false negatives shaped exactly like "not deployed." Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XLZtmuSs1RirWGMGyCB1QB
Two more from the same day, same family as the identity/description split. A test that pins an absolute date then validates against the real clock is a bomb with a fuse: hub's operator-token test minted at a pinned 2026-04-26 with a 90-day TTL and validated at real now — green until 2026-07-25, red forever after, and it failed on a commit that had nothing to do with it. Records the fix (inject the clock on both sides) and the sweep method that actually bounds the class: ask which paths enforce expiry against a clock you can't inject, then intersect with pinned mints. Listing date literals doesn't bound anything. And the condition that hid it: hub's suite runs at tag time only, so three days of red looked green. parachute-agent went red on a docs-only commit, which means its real breakage landed earlier and unobserved. Hence: never let a stable release be a line's first gate run. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XLZtmuSs1RirWGMGyCB1QB
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Captures the most reusable lesson from the 2026-07-28 cross-repo audit into the verify skill, which is where verification discipline lives.
The pattern. A version string, a status column, and a git tag all describe intent. Three separate failures in one day had drifted from reality, and each read green on every signal we normally trust:
parachute statusreportedbun-linked → <repo> @ <sha>. That was true of resolution and false of what was served. It surfaced only because the served/assets/index-<hash>.jsdidn't exist in the checkout'sdist/.The counter-discipline added: fetch the artifact and diff it rather than reading what something reports about itself — served hash vs
dist/,npm packtarball vs source,git ls-remote --tagsvsnpm view versions— and when a service reports a version, ask which path it read that from.Also records a probe gotcha that cost real time during the trace: a minified bundle is one ~540 KB line, so
$(curl …)+echo | grepmangles it and returns false negatives shaped exactly like "the fix isn't deployed." I reported "marker absent" twice from that broken instrument before checking the instrument.Docs-only; no version bump.
🤖 Generated with Claude Code
https://claude.ai/code/session_01XLZtmuSs1RirWGMGyCB1QB