Skip to content

packages: recover binary index identity from archive manifests#945

Open
brandonpayton wants to merge 1 commit into
mainfrom
fix/package-index-archive-identity
Open

packages: recover binary index identity from archive manifests#945
brandonpayton wants to merge 1 commit into
mainfrom
fix/package-index-archive-identity

Conversation

@brandonpayton

@brandonpayton brandonpayton commented Jul 14, 2026

Copy link
Copy Markdown
Member

Why

xtask build-index rebuilt package identity by splitting an archive filename at its last hyphen. That filename grammar is ambiguous because both package names and versions may contain hyphens. It silently changed spidermonkey-node@140.11.0esr-node.1 into a different package name and version.

The same recovery path also let directory traversal order decide which archive won when more than one archive existed for a package and architecture.

What changed

  • Treat the validated manifest.toml inside each archive as the authoritative package identity and compatibility record.
  • Use one canonical archive-name renderer from both archive-stage and build-index; the filename is a transport label and must match the structured manifest.
  • Reject duplicate (package, target architecture) archives before writing an index.
  • Report both filenames, cache keys, and archive SHA-256 values so an operator can choose an explicit immutable archive.
  • Classify the shared renderer as archive-producing and publication-flow code.
  • Document the recovery behavior in docs/binary-releases.md.

This changes package publication tooling only. It does not change archive bytes, package cache keys, runtime behavior, or the Kandelo ABI.

Validation

Run through scripts/dev-shell.sh on exact commit 03fb6ec878822996765699d2ee4a463be5c89ed5 / tree ccfeace9338a5e96a461d2c0260e353072d8e84b:

  • all 377 xtask tests passed on the explicit native host target, including 9 focused build_index tests;
  • change-scope classifier tests passed;
  • tests/scripts/index-update.sh passed;
  • scripts/check-abi-version.sh passed;
  • scripts/compose-initial-index.sh syntax and git diff --check passed.

The rebased commit has the same stable patch ID as the previously reviewed head. Independent adversarial review approved this exact commit with no findings.

Merge

The prior hosted run was canceled, so this refreshed exact head needs fresh hosted gates. This PR receives standalone validation because it changes the scope classifier that selects publication tests. PR #939 changes a separate release-control path and may validate concurrently.

@brandonpayton

Copy link
Copy Markdown
Member Author

Merge-order note: keep this PR without the ready-to-ship label until #946 lands. This branch predates the isolated post-merge index activation workflow; arming its current Prepare merge would allow a pre-merge package run to write the canonical ABI ledger again. The current Staging build is safe and isolated. After #946 merges, update this branch onto main, rerun exact-head CI, then arm Prepare merge for review evidence.

@github-actions

Copy link
Copy Markdown

Phase B-1 matrix build status — pr-945-staging

ABI v39. 71 built, 0 failed, 71 total.

Package Arch Status Sha
icu wasm32 built ae9436d0
libcurl wasm32 built 61d74e71
libcxx wasm32 built 5b45d53b
libcxx wasm64 built 7935cf5d
libiconv wasm32 built 1f8e50c9
libpng wasm32 built 801e4ee2
libxml2 wasm32 built eb45862d
libzip wasm32 built 9b95d070
openssl wasm32 built 72f5e269
openssl wasm64 built 3d2ede81
sqlite wasm32 built d80c4723
sqlite wasm64 built a0184469
zlib wasm32 built 7d4944be
zlib wasm64 built cc826e44
bc wasm32 built b2fe8348
bzip2 wasm32 built da26ddd6
coreutils wasm32 built 7067584f
curl wasm32 built 8760d556
dash wasm32 built 6185cf4a
diffutils wasm32 built e43ed6d6
dinit wasm32 built 1f95b09c
fbdoom wasm32 built f117afb1
file wasm32 built 96a29c22
findutils wasm32 built 9b5d86a2
gawk wasm32 built e798d30c
git wasm32 built fe6fcce1
grep wasm32 built 3a358657
gzip wasm32 built 8dcfdd62
hello wasm32 built 14a3598d
kandelo-sdk wasm32 built 18405ee6
kernel wasm32 built 2fcc4c78
less wasm32 built c54ea931
lsof wasm32 built 545fd154
m4 wasm32 built 925a623c
make wasm32 built 3849825a
mariadb wasm32 built ba61888e
mariadb wasm64 built 2f7f5f4e
modeset wasm32 built 09538e74
msmtpd wasm32 built c93670bb
nano wasm32 built 1b33e4d0
ncurses wasm32 built f71a7309
netcat wasm32 built 89d4b2a1
nginx wasm32 built 049b7f50
php wasm32 built f39e66db
posix-utils-lite wasm32 built a1c75d79
ruby wasm32 built 4d4f1bcb
sed wasm32 built 33c68bdd
spidermonkey wasm32 built 935002d0
tar wasm32 built 8a711f15
tcl wasm32 built 57b8b42f
unzip wasm32 built 6889dbd3
userspace wasm32 built e1da86ef
vim wasm32 built 5c035599
wget wasm32 built 5fffd21b
xz wasm32 built 84d4d0a3
zip wasm32 built 3c4533e7
zstd wasm32 built e847e1a8
bash wasm32 built 4fcd4793
mariadb-test wasm32 built 0089d361
mariadb-vfs wasm32 built d3f85390
mariadb-vfs wasm64 built 1e3cadc5
nethack wasm32 built 52bcfdaa
node wasm32 built 83db0122
spidermonkey-node wasm32 built 0f529045
vim-browser-bundle wasm32 built 0664361e
nethack-browser-bundle wasm32 built f0cc9db3
rootfs wasm32 built 0a05033f
shell wasm32 built d96885ae
lamp wasm32 built 65b962d3
node-vfs wasm32 built 8207665f
wordpress wasm32 built 491611d7

Auto-generated; replaced on each push. Raw data in the publish-status workflow artifact.

@brandonpayton brandonpayton force-pushed the fix/package-index-archive-identity branch from d5a4e11 to 326c780 Compare July 15, 2026 19:58
@brandonpayton brandonpayton changed the title packages: use archive manifests when rebuilding binary indexes packages: recover binary index identity from archive manifests Jul 15, 2026
Package names and versions may both contain hyphens, so build-index cannot recover their boundary from the transport filename. The old last-hyphen rule silently changed spidermonkey-node@140.11.0esr-node.1 into a different package identity.

Carry the validated archived manifest through index composition and validate the transport label with the same canonical renderer used by archive-stage. Fail closed when recovery sees multiple immutable archives for one package and target architecture, reporting both cache and archive identities instead of overwriting by directory order.

Keep the change in package publication tooling: package cache keys, archive bytes, and the Kandelo ABI remain unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant