Skip to content

Add a Docker continuity test #2798

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
alixander merged 1 commit into from
Aug 3, 2026
Merged

Add a Docker continuity test #2798

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
318 changes: 318 additions & 0 deletions .github/workflows/docker-continuity-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,318 @@
name: Docker continuity test

on:
workflow_dispatch:
inputs:
version:
description: Existing published D2 release tag to rebuild
required: true
type: string
default: v0.7.1

permissions:
contents: read

concurrency:
group: docker-continuity-test
cancel-in-progress: false

jobs:
validate:
runs-on: ubuntu-24.04
timeout-minutes: 5
outputs:
version: ${{ steps.release.outputs.version }}
steps:
- name: Validate published release
id: release
env:
GH_TOKEN: ${{ github.token }}
VERSION: ${{ inputs.version }}
run: |
set -euo pipefail

if ! printf '%s\n' "$VERSION" | grep -Eq '^v(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(-[0-9A-Za-z]+([.-][0-9A-Za-z]+)*)?$'; then
echo "version must be a v-prefixed semantic version" >&2
exit 1
fi

release_json=$(gh api "repos/$GITHUB_REPOSITORY/releases/tags/$VERSION")
if [ "$(printf '%s' "$release_json" | jq -r '.draft')" != false ]; then
echo "$VERSION is a draft release" >&2
exit 1
fi
if [ "$(printf '%s' "$release_json" | jq -r '.published_at')" = null ]; then
echo "$VERSION has not been published" >&2
exit 1
fi
if [ "$(printf '%s' "$release_json" | jq -r '.tag_name')" != "$VERSION" ]; then
echo "release tag does not match $VERSION" >&2
exit 1
fi

gh api "repos/$GITHUB_REPOSITORY/git/ref/tags/$VERSION" >/dev/null
for arch in amd64 arm64; do
asset="d2-$VERSION-linux-$arch.tar.gz"
if ! printf '%s' "$release_json" | jq -e --arg asset "$asset" \
'any(.assets[]; .name == $asset)' >/dev/null; then
echo "$VERSION does not contain $asset" >&2
exit 1
fi
done

echo "version=$VERSION" >>"$GITHUB_OUTPUT"

build-amd64:
needs: validate
runs-on: ubuntu-24.04
timeout-minutes: 45
environment: docker-release
env:
DOCKERHUB_USERNAME: ${{ vars.DOCKERHUB_USERNAME }}
PLATFORM: linux/amd64
ARCH: amd64
VERSION: ${{ needs.validate.outputs.version }}
outputs:
digest: ${{ steps.build.outputs.digest }}
steps:
- uses: actions/checkout@v6
with:
ref: ${{ needs.validate.outputs.version }}
persist-credentials: false

- name: Download release archive
env:
GH_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
asset="d2-$VERSION-linux-$ARCH.tar.gz"
mkdir -p "$RUNNER_TEMP/docker-context"
gh release download "$VERSION" \
--repo "$GITHUB_REPOSITORY" \
--pattern "$asset" \
--dir "$RUNNER_TEMP/docker-context"
tar -tzf "$RUNNER_TEMP/docker-context/$asset" >/dev/null
cp ci/release/docker/entrypoint.sh "$RUNNER_TEMP/docker-context/entrypoint.sh"

- name: Log in to Docker Hub
env:
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
run: |
set -euo pipefail
test -n "$DOCKERHUB_USERNAME"
test -n "$DOCKERHUB_TOKEN"
printf '%s' "$DOCKERHUB_TOKEN" | \
docker login --username "$DOCKERHUB_USERNAME" --password-stdin

- name: Build and push by digest
id: build
run: |
set -euo pipefail
image="$DOCKERHUB_USERNAME/d2"
metadata="$RUNNER_TEMP/build-metadata.json"

docker buildx create --name d2-continuity --driver docker-container --use
docker buildx inspect --bootstrap
docker buildx build \
--platform "$PLATFORM" \
--provenance=true \
--output "type=image,name=$image,push-by-digest=true,name-canonical=true,push=true" \
--metadata-file "$metadata" \
--file ci/release/docker/Dockerfile \
"$RUNNER_TEMP/docker-context"

digest=$(jq -er '."containerimage.digest"' "$metadata")
if ! printf '%s\n' "$digest" | grep -Eq '^sha256:[0-9a-f]{64}$'; then
echo "build did not return a valid image digest" >&2
exit 1
fi
docker buildx rm d2-continuity
echo "digest=$digest" >>"$GITHUB_OUTPUT"

- name: Verify image natively
env:
DIGEST: ${{ steps.build.outputs.digest }}
run: |
set -euo pipefail
image="$DOCKERHUB_USERNAME/d2@$DIGEST"
smoke="$RUNNER_TEMP/smoke"
mkdir -p "$smoke"
printf 'x -> y\n' >"$smoke/input.d2"

version_output=$(docker run --rm --platform "$PLATFORM" "$image" --version)
printf '%s\n' "$version_output"
printf '%s\n' "$version_output" | grep -F "$VERSION"

docker run --rm --platform "$PLATFORM" \
-u "$(id -u):$(id -g)" \
-v "$smoke:/home/debian/src" \
"$image" input.d2 output.svg
test -s "$smoke/output.svg"
grep -q '<svg' "$smoke/output.svg"

docker run --rm --platform "$PLATFORM" \
-u "$(id -u):$(id -g)" \
-v "$smoke:/home/debian/src" \
"$image" input.d2 output.png
test -s "$smoke/output.png"
test "$(od -An -tx1 -N8 "$smoke/output.png" | tr -d ' \n')" = 89504e470d0a1a0a

- name: Clean up Docker credentials
if: always()
run: |
docker logout >/dev/null 2>&1 || true
docker buildx rm d2-continuity >/dev/null 2>&1 || true

build-arm64:
needs: validate
runs-on: ubuntu-24.04-arm
timeout-minutes: 45
environment: docker-release
env:
DOCKERHUB_USERNAME: ${{ vars.DOCKERHUB_USERNAME }}
PLATFORM: linux/arm64
ARCH: arm64
VERSION: ${{ needs.validate.outputs.version }}
outputs:
digest: ${{ steps.build.outputs.digest }}
steps:
- uses: actions/checkout@v6
with:
ref: ${{ needs.validate.outputs.version }}
persist-credentials: false

- name: Download release archive
env:
GH_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
asset="d2-$VERSION-linux-$ARCH.tar.gz"
mkdir -p "$RUNNER_TEMP/docker-context"
gh release download "$VERSION" \
--repo "$GITHUB_REPOSITORY" \
--pattern "$asset" \
--dir "$RUNNER_TEMP/docker-context"
tar -tzf "$RUNNER_TEMP/docker-context/$asset" >/dev/null
cp ci/release/docker/entrypoint.sh "$RUNNER_TEMP/docker-context/entrypoint.sh"

- name: Log in to Docker Hub
env:
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
run: |
set -euo pipefail
test -n "$DOCKERHUB_USERNAME"
test -n "$DOCKERHUB_TOKEN"
printf '%s' "$DOCKERHUB_TOKEN" | \
docker login --username "$DOCKERHUB_USERNAME" --password-stdin

- name: Build and push by digest
id: build
run: |
set -euo pipefail
image="$DOCKERHUB_USERNAME/d2"
metadata="$RUNNER_TEMP/build-metadata.json"

docker buildx create --name d2-continuity --driver docker-container --use
docker buildx inspect --bootstrap
docker buildx build \
--platform "$PLATFORM" \
--provenance=true \
--output "type=image,name=$image,push-by-digest=true,name-canonical=true,push=true" \
--metadata-file "$metadata" \
--file ci/release/docker/Dockerfile \
"$RUNNER_TEMP/docker-context"

digest=$(jq -er '."containerimage.digest"' "$metadata")
if ! printf '%s\n' "$digest" | grep -Eq '^sha256:[0-9a-f]{64}$'; then
echo "build did not return a valid image digest" >&2
exit 1
fi
docker buildx rm d2-continuity
echo "digest=$digest" >>"$GITHUB_OUTPUT"

- name: Verify image natively
env:
DIGEST: ${{ steps.build.outputs.digest }}
run: |
set -euo pipefail
image="$DOCKERHUB_USERNAME/d2@$DIGEST"
smoke="$RUNNER_TEMP/smoke"
mkdir -p "$smoke"
printf 'x -> y\n' >"$smoke/input.d2"

version_output=$(docker run --rm --platform "$PLATFORM" "$image" --version)
printf '%s\n' "$version_output"
printf '%s\n' "$version_output" | grep -F "$VERSION"

docker run --rm --platform "$PLATFORM" \
-u "$(id -u):$(id -g)" \
-v "$smoke:/home/debian/src" \
"$image" input.d2 output.svg
test -s "$smoke/output.svg"
grep -q '<svg' "$smoke/output.svg"

docker run --rm --platform "$PLATFORM" \
-u "$(id -u):$(id -g)" \
-v "$smoke:/home/debian/src" \
"$image" input.d2 output.png
test -s "$smoke/output.png"
test "$(od -An -tx1 -N8 "$smoke/output.png" | tr -d ' \n')" = 89504e470d0a1a0a

- name: Clean up Docker credentials
if: always()
run: |
docker logout >/dev/null 2>&1 || true
docker buildx rm d2-continuity >/dev/null 2>&1 || true

combine:
needs: [validate, build-amd64, build-arm64]
runs-on: ubuntu-24.04
timeout-minutes: 10
environment: docker-release
env:
AMD64_DIGEST: ${{ needs.build-amd64.outputs.digest }}
ARM64_DIGEST: ${{ needs.build-arm64.outputs.digest }}
DOCKERHUB_USERNAME: ${{ vars.DOCKERHUB_USERNAME }}
STAGE_TAG: continuity-test-${{ github.run_id }}
steps:
- name: Log in to Docker Hub
env:
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
run: |
set -euo pipefail
test -n "$DOCKERHUB_USERNAME"
test -n "$DOCKERHUB_TOKEN"
printf '%s' "$DOCKERHUB_TOKEN" | \
docker login --username "$DOCKERHUB_USERNAME" --password-stdin

- name: Create and verify continuity-test manifest
run: |
set -euo pipefail
image="$DOCKERHUB_USERNAME/d2"
for digest in "$AMD64_DIGEST" "$ARM64_DIGEST"; do
if ! printf '%s\n' "$digest" | grep -Eq '^sha256:[0-9a-f]{64}$'; then
echo "invalid architecture digest" >&2
exit 1
fi
done

docker buildx imagetools create \
--tag "$image:$STAGE_TAG" \
"$image@$AMD64_DIGEST" \
"$image@$ARM64_DIGEST"
docker buildx imagetools inspect --raw "$image:$STAGE_TAG" \
>"$RUNNER_TEMP/manifest.json"

test "$(jq '[.manifests[] | select(.platform.os == "linux" and .platform.architecture == "amd64")] | length' "$RUNNER_TEMP/manifest.json")" -eq 1
test "$(jq '[.manifests[] | select(.platform.os == "linux" and .platform.architecture == "arm64")] | length' "$RUNNER_TEMP/manifest.json")" -eq 1
docker buildx imagetools inspect "$image:$STAGE_TAG"

{
echo "Published and verified \`$image:$STAGE_TAG\`."
echo
echo "Delete this continuity-test tag in Docker Hub after review."
} >>"$GITHUB_STEP_SUMMARY"

- name: Clean up Docker credentials
if: always()
run: docker logout >/dev/null 2>&1 || true
18 changes: 18 additions & 0 deletions ci/release/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,24 @@ Use `--host-only` to build only the release for the host's `$OS-$ARCH` pair.
Helper script called by build.sh to build D2 on each linux runner inside Docker.
The Dockerfile is in ./linux/Dockerfile

### Docker continuity test

The manually dispatched `Docker continuity test` GitHub workflow proves that an existing,
published D2 release can be rebuilt for Docker Hub without the legacy AWS builders. It
downloads the release's exact Linux archives, builds on native GitHub-hosted amd64 and arm64
runners, verifies the images, and publishes only
`terrastruct/d2:continuity-test-<workflow-run-id>`. It never updates a release version tag or
`latest`.

Before dispatching it, configure the `docker-release` GitHub environment with a
`DOCKERHUB_USERNAME` variable and a `DOCKERHUB_TOKEN` secret that can write to that user's
`d2` repository. Dispatch the workflow from the protected `master` branch. The version input
must name a published, non-draft GitHub release with both Linux archives; `v0.7.1` is the
default continuity fixture. Delete the continuity-test tag in Docker Hub after reviewing the
workflow summary and manifest.

This test does not disable or replace the existing release script's Docker publishing path.

### _build.sh

Called by build.sh (with --local or macOS) or build_docker.sh (on linux) to create the
Expand Down
Loading