homebrew: make guest bootstrap images reproducible and ABI-checked

[brandonpayton]

Summary

• add a committed guest Homebrew bootstrap-image builder pinned to an exact, unmodified upstream Homebrew commit
• resolve the Node kernel, canonical rootfs packages, and Homebrew bootstrap programs through the ABI-current Kandelo package resolver
• validate kernel/program ABI markers, required exports, fork instrumentation policy, and retired Asyncify absence before composing the image
• record upstream revision/archive provenance in the image and document the diagnostic artifact without claiming bottle pour or browser support

The Homebrew source archive is produced from a bare object store with git archive; no checkout state can leak into it. Both supported build modes reproduced archive SHA-256 318d9e0fc96b52893a2a5555b0990a3471308cc120938ce369df1b12314606a8.

Validation

• ./scripts/dev-shell.sh scripts/build-homebrew-bootstrap.sh
  • resolved the Node kernel, 14 canonical rootfs packages, and 8 Homebrew bootstrap packages
  • validated 1 kernel and 71 program Wasm artifacts at ABI 18
  • generated 334 rootfs paths and target/homebrew-bootstrap/homebrew-bootstrap.vfs (805,312,928 bytes)
• ./scripts/dev-shell.sh scripts/build-homebrew-bootstrap.sh --skip-package-resolve
  • passed the same ABI/artifact checks and reproduced the same image inputs
• focused tests/package-system/validate-wasm-artifacts.test.ts: 3/3 passed inside scripts/dev-shell.sh
• standalone validator TypeScript check passed with the repository's host TypeScript toolchain
• structured mkrootfs inspect check confirmed image ABI 18, executable brew owned by 1000:1000, executable Ruby, and embedded provenance metadata
• bash -n scripts/build-homebrew-bootstrap.sh
• git diff --check

Current boundary

This PR codifies the reproducible bootstrap artifact; it does not claim guest Homebrew is fixed. Running upstream brew --version from the generated image reaches the known ABI 18 continuation-save overflow and traps at the already diagnosed Bash path. PR #845 makes that overflow truthful; the capacity fix remains the ABI/relink work in #862. No ABI change or binary publication occurs in this PR.

shellcheck was not run because it is not available in the declared Nix dev shell.

[brandonpayton]

Exact-head CI rerun classification for aa1c84368988808bd776e610e95f668d2711a10e:

The first full Vitest attempt had two unrelated timing failures: fork-from-thread.test.ts:57 exceeded its 5-second limit and teardown-reclaim.test.ts:71 hit the readiness race addressed by #889. I reran that exact failed job without changing the head. Attempt 2 passed 118/118 test files and 873 tests (plus the focused 2-file/3-test slice); all browser, kernel, fork-instrument, libc, POSIX, and Sortix jobs also pass.

Passing rerun: https://github.com/Automattic/kandelo/actions/runs/29196248678/job/86661597571

This supports a timing/load classification for the first attempt; it does not broaden this PR's runtime claim beyond its reproducible bootstrap-builder and artifact-validation scope.

[brandonpayton]

Independent devil review of exact head aa1c84368988808bd776e610e95f668d2711a10e found two blockers; please do not merge this head yet.

1. mkrootfs ZIP ingestion exposes ZipEntry.isSymlink but currently creates every non-directory entry as a regular file. The pinned upstream Homebrew tree contains 167 symlinks; retained-image inspection confirms tool shims such as Library/Homebrew/shims/linux/super/{cc,curl,git} were flattened to 100644 files instead of their relative symlink targets. This contradicts the unmodified-upstream claim and can break Homebrew execution. I am preparing a dedicated generic mkrootfs symlink/path-integrity PR.
2. The builder resolves into binaries/, but generate-rootfs-package-manifest.mjs can prefer local-binaries/. That permits eager local bytes or lazy local sizes to diverge from the resolver path that is validated and recorded. homebrew: make guest bootstrap images reproducible and ABI-checked #902 needs an explicit resolver input root/tier before its normal and --skip-package-resolve provenance claims are valid.

The exact retained ABI 18 build was not contaminated (this worktree has no local overrides), pinned archive provenance is sound, and the branch applies textually cleanly to locked #907. After both fixes and #907 publication, rebuild both modes at ABI 39, inspect all upstream symlink targets/modes/owners, and run real brew --version through NodeKernelHost. No 805 MB rebuild was performed during review.

[brandonpayton]

The dedicated generic fix for blocker 1 is now open for maintainer review as #917 at exact head 17e924a1a6f9f4eab3da52cbda4d6b886b0d4f31.

Fresh security review accepted the corrected head after reproducing and closing both the original symlink flattening and a manifest-path alias write-through exploit. On #902's exact pinned Homebrew archive it planned 3,511 entries, preserved all 167 symlinks (11 parent-relative), and found zero target/mode/uid/gid mismatches; the explicit bin/brew override retained its exact bytes and metadata.

#902 remains blocked: #917 must land, and the separate artifact-root provenance change is still in exact local review/integration. No ABI 39 bootstrap image rebuild has been claimed yet.

[brandonpayton]

The two generic blockers identified while validating this bootstrap now have dedicated main-repo PRs:

• mkrootfs: preserve ZIP symlinks without path escape #917 makes mkrootfs ZIP ingestion preserve and validate Unix symlinks, metadata, path identity, and override semantics.
• packages: bind rootfs manifests to one artifact tree #919 adds an exclusive --binaries-dir manifest input so the image cannot mix worktree-local and fetched package artifacts.

After both land, this branch will be restacked onto their exact commits, pass --binaries-dir "$REPO_ROOT/binaries", resolve the durable ABI 39 package set, and rebuild/validate the Homebrew image through both normal and --skip-package-resolve paths. #902 remains intentionally gated until then.

[brandonpayton]

ABI 39 integration update (no change to this PR head):

I assembled locked #907 (0f464442) + this commit + exact #917/#919 + the --binaries-dir "$REPO_ROOT/binaries" caller change. The normal resolver and --skip-package-resolve paths both completed against durable binaries-abi-v39, each validating 1 kernel and 72 program artifacts and producing an 805,316,862-byte image. The image has ABI 39 metadata; the Homebrew subtree contains 3,516 entries and all 167 upstream symlinks; bin/brew is the exact 8,671-byte launcher with mode 0755 and owner 1000:1000.

Two boundaries remain before this PR can truthfully claim a reproducible, usable bootstrap:

1. The two logical images inspect identically, but raw SHA-256 differs (468d1508... vs bf8863a1...). All 41,709 differing bytes begin in SharedFS inode timestamp fields: mkrootfs currently serializes wall-clock atime/mtime/ctime. A generic deterministic-image repair is being isolated; this is not an artifact-provenance difference.
2. Running unmodified upstream brew --version and brew config --no-install-from-api from the generated image reaches the truthful fork guard: Ruby needs a 20,012-byte continuation while ABI 39 reserves 16,384 bytes, so both commands exit 139. fork: size continuation save buffers from live frames #862 is not in Batch PHP support and bundle ABI updates with their runtime changes #907 and its current opt-in only covers GTK/LXDE programs, not Ruby. Clearing this requires an explicit next-ABI Ruby relink/rebuild decision.

The combined contract suites are otherwise green: mkrootfs 160/160 after adapting the ABI 39 lazy-identity assertion, host ZIP/lazy 33/33, manifest/wasm validator 26/26, host ESM/CJS/DTS builds, TypeScript, shell syntax, and ABI snapshot checks.

[brandonpayton]

Replacement batch #921 preserves this PR exact patch as commit 83d8fe034 on merged ABI39; the dependent artifact-tree caller is a separate commit in the same linear stack. Keep this PR open until the combined batch CI and maintainer review complete, then close it as superseded.

[brandonpayton]

Superseded by merged #921. The original commit is patch-equivalent to main commit be77b14 (git cherry origin/main aa1c8436 reports -), and #921 adds the follow-up manifest provenance binding and trusted executable-bit preservation in 6cf0e62 and f693b58. No behavior or test from this PR remains unmerged.