Skip to content

feat(gcp-pipeline): build each platform on a native runner and merge the manifest - #71

Open
MaximusHaximus wants to merge 2 commits into
mainfrom
feat/native-multiarch-docker-release
Open

feat(gcp-pipeline): build each platform on a native runner and merge the manifest#71
MaximusHaximus wants to merge 2 commits into
mainfrom
feat/native-multiarch-docker-release

Conversation

@MaximusHaximus

@MaximusHaximus MaximusHaximus commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

What

  • gcp_pipeline_release_image.yaml now builds each platform listed in platforms on a runner of that platform's own CPU architecture (a matrix job), pushes each result by digest, and a final job merges the digests into one tagged multi-platform manifest with docker buildx imagetools create. QEMU emulation is removed.
  • Two new optional inputs: runner_amd64 (default ubuntu-latest) and runner_arm64 (default ubuntu-24.04-arm, GitHub's standard arm64 runner, available to private repositories since January 2026). Every existing input and default is unchanged, so callers need no changes.
  • Build provenance (GIT_SHA / BUILT_AT) is resolved once, so every platform's image carries identical values. The GitHub Actions layer cache is scoped per image and per platform so parallel legs, and sibling images released from the same repository, do not overwrite each other's cache index.
  • Every third-party action is pinned to the commit SHA of its current release, with the version alongside (checkout v7.0.1, setup-buildx v4.3.0, google-github-actions/auth v3.0.0, metadata-action v6.2.0, build-push-action v7.3.0, upload-artifact v7.0.1, download-artifact v8.0.1). This resolves the open actions/unpinned-tag code-scanning alerts on this file. Each input and output the workflow relies on was checked against the new versions.
  • README row for the workflow updated.

Why

  • Emulating linux/arm64 under QEMU on the amd64 runner made a typical service's TypeScript compile about 8x slower than native (471s vs 58s on identical source). The emulated leg accounted for roughly 11 of a 12-minute build step while the finished amd64 image sat waiting. Every release of every consumer of this workflow pays that cost.
  • The arm64 image is the one our GKE workloads actually run, since they schedule onto arm64 node pools, so dropping the platform is not an option. Building it natively is.
  • Tags are applied only after every platform has succeeded, so a failed leg leaves the tag unpublished instead of half-published.

Verification

Live: a consuming monorepo pinned its release trigger to this branch and cut two real releases of five service images each through it, one before and one after the action-version bump. All ten runs succeeded, the pushed tags are OCI image indexes listing linux/amd64 and linux/arm64 (plus the usual BuildKit attestation entries), and the services were deployed from those images. The second release ran at this PR's head and downloaded exactly the pinned action commits.

Per image, across both releases (the first on a cold layer cache, since the cache scopes are new):

Job Duration
Resolve platforms, tags and build provenance 5s to 9s
Build linux/amd64 (ubuntu-latest) 2m06s to 3m01s
Build linux/arm64 (ubuntu-24.04-arm, 2 vCPU) 1m44s to 2m13s
Push multi-platform manifest 22s to 47s
Whole release workflow, including the consumer's own test job 5m35s to 9m55s (the spread is the consumer's test job, 2m18s to 5m47s)

The same five images on the previous single-job QEMU build, over their last ten releases: build step 10m38s to 15m23s, whole workflow 14m24s to 19m09s. The native arm64 pnpm run build took 24s against 471s emulated.

CodeQL's Analyze (actions) passes on the head with zero actions/unpinned-tag alerts remaining on this file, where main carries five. actionlint passes apart from pre-existing shellcheck quoting notes on the carried-over tag-construction step and the intentional unquoted $(...) expansions on the imagetools create line, which must word-split to produce one argument per tag and digest.

Notes for reviewers

  • Consumers see three jobs (prepare, build (<platform>), merge) instead of one. Digests travel between jobs as a one-day-retention artifact named digests-<image>-<platform>.
  • On ubuntu-24.04-arm private repositories get 2 vCPU. It was still the faster leg in the runs above; pass a larger-runner label through runner_arm64 for more.

…the manifest

Building linux/arm64 under QEMU on the amd64 runner made a typical
service's TypeScript compile ~8x slower than native (471s vs 58s on
identical source) and left the finished amd64 image waiting ~9 minutes
for it on every release. Every web-apps namespace schedules onto arm64
(C4A) nodes, so the arm64 image is the one production runs and cannot
be dropped.

Each platform in `platforms` now builds on a runner of its own
architecture (`runner_amd64`, `runner_arm64` inputs; arm64 defaults to
GitHub's `ubuntu-24.04-arm`), pushes by digest, and a final job merges
the digests into one tagged multi-platform manifest. Tags are applied
only after every platform succeeds, so a failed leg leaves the tag
unpublished instead of half-published. Build provenance is resolved once
so all platforms stamp the same GIT_SHA/BUILT_AT. Cache scope is per
image and platform so parallel legs and sibling images do not overwrite
each other's cache index.

Callers need no changes; the inputs and defaults they already use are
unchanged.

Claude-Session: https://claude.ai/code/session_01BVWnz2gcXRyRoA9vefSFXM
Comment thread .github/workflows/gcp_pipeline_release_image.yaml Fixed
Comment thread .github/workflows/gcp_pipeline_release_image.yaml Fixed
Comment thread .github/workflows/gcp_pipeline_release_image.yaml Fixed
@MaximusHaximus
MaximusHaximus marked this pull request as ready for review September 10, 2026 21:14
@MaximusHaximus
MaximusHaximus marked this pull request as draft September 10, 2026 21:16
…release

Code scanning flags each `uses:` pinned to a mutable tag on this file
(actions/unpinned-tag). Every third-party action now references the
commit of its current release with the version alongside, and the
releases are current majors: checkout v7.0.1, setup-buildx v4.3.0,
google-github-actions/auth v3.0.0, metadata-action v6.2.0,
build-push-action v7.3.0, upload-artifact v7.0.1, download-artifact
v8.0.1. All run on Node 24, which GitHub-hosted runners provide. Each
input and output this workflow relies on was checked against the new
version's action.yml: auth keeps token_format, workload_identity_provider
and service_account; build-push keeps outputs, digest, labels,
build-args, cache-from/to and platforms; metadata-action still exports
DOCKER_METADATA_OUTPUT_JSON; download-artifact v8's single-artifact path
change does not apply to pattern downloads with merge-multiple.

Claude-Session: https://claude.ai/code/session_01BVWnz2gcXRyRoA9vefSFXM
@MaximusHaximus
MaximusHaximus marked this pull request as ready for review September 10, 2026 22:23
@MaximusHaximus
MaximusHaximus requested review from a team and rgelinas-polygon September 10, 2026 22:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants