feat(*): add node installer dir and workflow #26

vdice · 2024-03-06T16:42:52Z

Adds node-installer.yaml to publish a custom node installer image with only the Spin shim baked in
- Call this workflow for both PRs (ci.yaml) and main/tag release (release.yaml)
Also run release.yaml on merges to main (albeit sans GH release when push to main)
Update RELEASE_VERSION logic to support other refs (eg refs/heads/main where the 10 char split breaks)

I believe the intention is to eventually have spinkube/runtime-class-manager produce this image via fetching versioned shim releases from this repo, but this gets us an image with the latest shim version that we/users can plug in when installing the kwasm-operator chart, near-term.

Tested on fork eg https://github.com/vdice/containerd-shim-spin/actions/runs/8181643173

kwasm-node-installer/Dockerfile

Mossaka · 2024-03-06T18:39:22Z

nit: consider renaming kwasm to runtime class manager

vdice · 2024-03-06T20:29:57Z

@Mossaka agreed it might be better to use a different name. Slightly hesitant to use runtime-class-manager since https://github.com/spinkube/runtime-class-manager in its current form appears to hold the operator logic as opposed to the node-installer logic... what about shim-node-installer here; or just node-installer? The latter as oci ref would be something like ghcr.io/spinkube/containerd-shim-spin/node-installer. Thoughts?

radu-matei · 2024-03-06T21:26:23Z

Would it be possible to generate the node installer image on PRs as well?
It would be fantastic to be able to create a test cluster based on a not yet merged branch, and not have to build everything yourself.

vdice · 2024-03-06T22:13:38Z

Would it be possible to generate the node installer image on PRs as well?

We could; working on a re-org to support this. As a follow-up in the future, if we wanted, we could also push the k3d img on PRs while we are at it.

radu-matei · 2024-03-07T00:05:10Z

kwasm-node-installer/Dockerfile

+
+RUN mkdir -p /release/bin/ \
+    && curl -L https://github.com/containerd/runwasi/releases/download/containerd-shim-wasmedge%2Fv0.3.0/containerd-shim-wasmedge-$(uname -m | sed s/arm64/aarch64/g | sed s/amd64/x86_64/g).tar.gz | tar -xzf - -C /release/bin/ \
+    && curl -L https://github.com/containerd/runwasi/releases/download/containerd-shim-wasmtime%2Fv0.3.0/containerd-shim-wasmtime-$(uname -m | sed s/arm64/aarch64/g | sed s/amd64/x86_64/g).tar.gz | tar -xzf - -C /release/bin/ \


One point I want to raise here is — this is the custom node installer image for SpinKube and the Spin operator project, and we generate this image from the Spin shim.

I'd like to propse that this image only configures the Spin shim, for a number of reasons:

the other shims are not used by SpinKube

we are not updating them as part of this installer

we might actually overwrite someone's desired other shim version

they represent a security vulnerability area that is unnecessary because of the first two points

Managing other shims and runtime classes is the job for https://github.com/spinkube/runtime-class-manager/, and configuration there IMO is where someone would decide to bring in another shim.

Signed-off-by: Vaughn Dice <[email protected]>

vdice · 2024-03-07T02:43:25Z

In the latest commits:

Tentatively updated img name to node-installer cc @Mossaka
Broke out publishing of this image into its own workflow, so it can be called from PR and release pipelines (re: feat(*): add node installer dir and workflow #26 (comment))
Removed installation of runtimes; reduced to only installing/configuring the Spin shim (re: feat(*): add node installer dir and workflow #26 (review))

Signed-off-by: Vaughn Dice <[email protected]>

vdice · 2024-03-07T16:44:19Z

Looks like the workflow can't push the node-installer image, probably due ro running from a PR. Is there a setting we can toggle to allow publishing packages from PRs? Or perhaps I need to create the branch on origin and not my fork?

 buildx failed with: ERROR: failed to solve: failed to push ghcr.io/spinkube/containerd-shim-spin/node-installer:20240307-033658-g6a41bf5: unexpected status from POST request to https://ghcr.io/v2/spinkube/containerd-shim-spin/node-installer/blobs/uploads/: 403 Forbidden

Edit: Yes, images (packages) can only be published when the PR branch is on origin. (See #30 and https://github.com/spinkube/containerd-shim-spin/actions/runs/8191813640/job/22403982746?pr=30). I assume that's our desired state as opposed to allowing PRs from forks to publish packages - if even possible. Hence, I've gated the job appropriately in 198e74f. cc @radu-matei

Edit 2: Actually, my attempt at gating the job if a PR is from a fork isn't working as expected; looking...

Mossaka · 2024-03-07T22:45:23Z

Yeah node-installer sounds good to me

vdice · 2024-03-07T23:10:12Z

@radu-matei I don't see a great way to gate the node-installer image push job to only running when the PR is from a branch on origin, rather than a fork. The workaround to publish custom node-installer images from code changes not on main would be to push to the main branch of one's fork where the code changes are. Would this suffice near-term? If so, I'd remove logic to publish the image on PRs; we can revisit in the future.

Edit: wait, trying one more thing... yup 167ab94 works to only push the image if PR branch is on origin.

Signed-off-by: Vaughn Dice <[email protected]>

Mossaka

LGTM

Mossaka · 2024-03-08T21:07:12Z

.github/workflows/ci.yaml

+    uses: ./.github/workflows/node-installer.yaml
+    needs: build
+    # This action requires use of the GITHUB_TOKEN to publish the image
+    # By default, PRs from forks don't have access, so we only run when the PR branch is on origin.


Does this imply that maintainers of this project should push feat branches to origin?

Yes. Should I add a mention of this in other areas as well? Maybe the README?

I don't think it's necessary to put it in the README

kate-goldenring

LGTM!

vdice commented Mar 6, 2024

View reviewed changes

kwasm-node-installer/Dockerfile Outdated Show resolved Hide resolved

radu-matei reviewed Mar 7, 2024

View reviewed changes

vdice added 2 commits March 6, 2024 17:26

feat(*): add node installer dir and workflow

163101b

Signed-off-by: Vaughn Dice <[email protected]>

ref(node-installer): rm installation of runtimes

ca85084

Signed-off-by: Vaughn Dice <[email protected]>

vdice force-pushed the ci/kwasm-node-installer branch from e0eefe7 to ca85084 Compare March 7, 2024 02:41

fix(ci.yaml): update node-installer usage

132e4c2

Signed-off-by: Vaughn Dice <[email protected]>

vdice changed the title ~~feat(*): add kwasm-node-installer dir, add publishing steps to release.yaml~~ feat(*): add node installer dir and workflow Mar 7, 2024

vdice mentioned this pull request Mar 7, 2024

feat(*): node-installer #30

Closed

vdice requested review from radu-matei and Mossaka March 7, 2024 18:08

vdice requested review from kate-goldenring, jsturtevant and devigned as code owners March 7, 2024 20:09

vdice force-pushed the ci/kwasm-node-installer branch from 68f8593 to 198e74f Compare March 7, 2024 21:56

ci(node-installer): gate job to only run if branch on origin

167ab94

Signed-off-by: Vaughn Dice <[email protected]>

vdice force-pushed the ci/kwasm-node-installer branch from 198e74f to 167ab94 Compare March 7, 2024 23:28

Mossaka approved these changes Mar 8, 2024

View reviewed changes

kate-goldenring approved these changes Mar 11, 2024

View reviewed changes

kate-goldenring merged commit 1f8c36a into spinkube:main Mar 11, 2024
8 of 9 checks passed

vdice deleted the ci/kwasm-node-installer branch March 11, 2024 14:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(*): add node installer dir and workflow #26

feat(*): add node installer dir and workflow #26

vdice commented Mar 6, 2024 •

edited

Loading

Mossaka commented Mar 6, 2024

vdice commented Mar 6, 2024 •

edited

Loading

radu-matei commented Mar 6, 2024

vdice commented Mar 6, 2024

radu-matei Mar 7, 2024

vdice commented Mar 7, 2024 •

edited

Loading

vdice commented Mar 7, 2024 •

edited

Loading

Mossaka commented Mar 7, 2024

vdice commented Mar 7, 2024 •

edited

Loading

Mossaka left a comment

Mossaka Mar 8, 2024

vdice Mar 8, 2024

Mossaka Mar 8, 2024

kate-goldenring left a comment

feat(*): add node installer dir and workflow #26

feat(*): add node installer dir and workflow #26

Conversation

vdice commented Mar 6, 2024 • edited Loading

Mossaka commented Mar 6, 2024

vdice commented Mar 6, 2024 • edited Loading

radu-matei commented Mar 6, 2024

vdice commented Mar 6, 2024

radu-matei Mar 7, 2024

Choose a reason for hiding this comment

vdice commented Mar 7, 2024 • edited Loading

vdice commented Mar 7, 2024 • edited Loading

Mossaka commented Mar 7, 2024

vdice commented Mar 7, 2024 • edited Loading

Mossaka left a comment

Choose a reason for hiding this comment

Mossaka Mar 8, 2024

Choose a reason for hiding this comment

vdice Mar 8, 2024

Choose a reason for hiding this comment

Mossaka Mar 8, 2024

Choose a reason for hiding this comment

kate-goldenring left a comment

Choose a reason for hiding this comment

vdice commented Mar 6, 2024 •

edited

Loading

vdice commented Mar 6, 2024 •

edited

Loading

vdice commented Mar 7, 2024 •

edited

Loading

vdice commented Mar 7, 2024 •

edited

Loading

vdice commented Mar 7, 2024 •

edited

Loading