Cargo isn't able to find the spin-sdk crate during build

[divya-mohan0209]

Hello folks!

I was trying out the containerd-shim-spin quickstart and while building the container image, the below error message was thrown:

 => ERROR [build 4/4] RUN rustup target add wasm32-wasi && cargo build --target wasm32-wasi --release                                                          459.5s
------                                                                                                                                                                
 > [build 4/4] RUN rustup target add wasm32-wasi && cargo build --target wasm32-wasi --release:                                                                       
0.187 info: downloading component 'rust-std' for 'wasm32-wasi'                                                                                                        
15.78 info: installing component 'rust-std' for 'wasm32-wasi'                                                                                                         
17.35     Updating crates.io index                                                                                                                                    
423.7     Updating git repository `https://github.com/fermyon/spin`                                                                                                   
459.4 error: no matching package named `spin-sdk` found
459.4 location searched: https://github.com/fermyon/spin?tag=v2.0.1#1d72f1c4
459.4 required by package `qs-wasm-spin v0.1.0 (/opt/build)`

I'm running spin v2.0.1

spin --version
spin 2.0.1 (1d72f1c 2023-11-10)

The list of plugins that I have are:

 spin plugins list  
check-for-update 0.0.1
check-for-update 0.1.0
cloud 0.1.0
cloud 0.1.1
cloud 0.1.2
cloud 0.2.0
cloud 0.3.0
cloud 0.4.0
cloud 0.4.1
cloud 0.5.0
cloud 0.5.1 [installed]
cloud-gpu 0.1.0
js2wasm 0.1.0
js2wasm 0.2.0
js2wasm 0.3.0
js2wasm 0.4.0
js2wasm 0.5.0
js2wasm 0.5.1
js2wasm 0.6.0
js2wasm 0.6.1 [installed]
k8s 0.1.0 [installed]
pluginify 0.6.0
py2wasm 0.1.0
py2wasm 0.1.1
py2wasm 0.2.0
py2wasm 0.3.0
py2wasm 0.3.2 [installed]

Other details:

• I'm running this on my Mac (M1 chip), but I've also tried this out on Ubuntu.
• Docker Desktop version 4.25.2 and I have also enabled the beta feature for running wasm.

[divya-mohan0209]

Also, FWIW here's the content of the Dockerfile that's there:

FROM --platform=${BUILDPLATFORM} rust:1.59 AS build
WORKDIR /opt/build
COPY . .
RUN rustup target add wasm32-wasi && cargo build --target wasm32-wasi --release

FROM scratch
COPY --from=build /opt/build/target/wasm32-wasi/release/qs_wasm_spin.wasm .
COPY --from=build /opt/build/spin.toml .

The funny part is the cargo build command works like a charm when executing the spin build command and is able to find the spin-sdk. Sample output is shown below:

➜  hello-world spin build
Building component hello-world with `cargo build --target wasm32-wasi --release`
    Updating crates.io index
    Updating git repository `https://github.com/fermyon/spin`
   Compiling proc-macro2 v1.0.69
   Compiling unicode-ident v1.0.12
   Compiling serde v1.0.193
   Compiling anyhow v1.0.75
   Compiling serde_json v1.0.108
   Compiling autocfg v1.1.0
   Compiling semver v1.0.20
   Compiling equivalent v1.0.1
   Compiling itoa v1.0.9
   Compiling hashbrown v0.14.2
   Compiling leb128 v0.2.5
   Compiling ryu v1.0.15
   Compiling smallvec v1.11.2
   Compiling spdx v0.10.2
   Compiling wasm-encoder v0.38.0
   Compiling id-arena v2.2.1
   Compiling unicode-xid v0.2.4
   Compiling log v0.4.20
   Compiling futures-core v0.3.29
   Compiling slab v0.4.9
   Compiling wasm-encoder v0.36.2
   Compiling bitflags v2.4.1
   Compiling quote v1.0.33
   Compiling futures-task v0.3.29
   Compiling futures-channel v0.3.29
   Compiling version_check v0.9.4
   Compiling syn v2.0.39
   Compiling unicode-segmentation v1.10.1
   Compiling smartstring v1.0.1
   Compiling futures-util v0.3.29
   Compiling futures-sink v0.3.29
   Compiling wit-bindgen-rust v0.13.2
   Compiling heck v0.4.1
   Compiling futures-io v0.3.29
   Compiling memchr v2.6.4
   Compiling pin-utils v0.1.0
   Compiling syn v1.0.109
   Compiling static_assertions v1.1.0
   Compiling pin-project-lite v0.2.13
   Compiling fnv v1.0.7
   Compiling async-trait v0.1.74
   Compiling thiserror v1.0.50
   Compiling bytes v1.5.0
   Compiling smartcow v0.2.1
   Compiling spin-sdk v2.0.1 (https://github.com/fermyon/spin?tag=v2.0.1#1d72f1c4)
   Compiling http v0.2.11
   Compiling percent-encoding v2.3.0
   Compiling form_urlencoded v1.2.0
   Compiling indexmap v2.1.0
   Compiling routefinder v0.5.3
   Compiling wasmparser v0.118.0
   Compiling wasmparser v0.116.1
   Compiling once_cell v1.18.0
   Compiling serde_derive v1.0.193
   Compiling futures-macro v0.3.29
   Compiling thiserror-impl v1.0.50
   Compiling spin-macro v0.1.0 (https://github.com/fermyon/spin?tag=v2.0.1#1d72f1c4)
   Compiling wit-parser v0.12.2
   Compiling wasm-metadata v0.10.13
   Compiling futures-executor v0.3.29
   Compiling futures v0.3.29
   Compiling wit-component v0.17.0
   Compiling wit-bindgen-core v0.13.1
   Compiling wit-bindgen-rust-macro v0.13.1
   Compiling wit-bindgen v0.13.1
   Compiling hello-world v0.1.0 (/Users/divyamohan/hello-world)
    Finished release [optimized] target(s) in 18.36s
Finished building all Spin components

[vdice]

Hi @divya-mohan0209, thank you for raising this issue. I was able to reproduce as well.

I think the issue is due to Spin's minimum declared Rust version, which is currently 1.71. When I updated the first line of the Dockerfile to FROM --platform=${BUILDPLATFORM} rust:1.71 AS build, it was able to build. Would you mind trying this when convenient?

Also, as you've probably noticed from the quickstart, the Dockerfile image approach is just one (older) way of running the Spin app in k8s. Now that Spin natively uses OCI registries for distribution of apps, you can go straight to spin registry push once your app is built; no need for a wrapping container image.

I'm seeing a few things in the linked Quickstart that could use updates, so thanks again for relaying your experience and I'll create a PR upstream soon.

[divya-mohan0209]

Absolutely! Do you think this could be explicitly mentioned somewhere in the Quickstart? I realize that the version number cannot be static lol, but just saying that Spin has this minimum declared dependency and the version of rust, at any point in time, must be cross-checked against this version? I'm happy to submit a PR/raise an issue for that against the Quickstart, if needed!

[vdice]

@divya-mohan0209 I put up a PR with some of the things you've spotted, including the Rust version here: deislabs/containerd-wasm-shims#188. Would definitely appreciate your taking a look and trying things out to see if there are still some places to update. Thanks!

[divya-mohan0209]

So, I'm happy to report that the build works.

However, when I deploy it to the k3d cluster, the pod creation fails with an error:

no match for platform in manifest: not found.

FWIW, I even tried running it as a normal container using Docker but ended up with the same error. I cross-checked with the Dockerfile of the spin images that are used for the example deployment and the entries for the Rust-based deploy seem to be identical.

[itowlson]

@divya-mohan0209 Do you know which version of the shim you have in the k3d cluster? It's a bit hard to tell from the containerd-wasm-shims quickstart. There's a Discord thread (https://discord.com/channels/926888690310053918/1122332926244294806/1124380791846219886) where this happens when the manifest format is newer than the shim can handle, although I'm pretty sure the shim has been updated very recently. @Mossaka is probably the expert on this.

[divya-mohan0209]

@itowlson I will have to check my notes although in the Quickstart itself, there seems to be an indication of the version of spin-shim that's being used and it's a little different from the one you have on the Spin docs listed under the k3d section. FWIW, for the cluster I'm running, I followed the Quickstart (i.e.the first option) and that does seem to be the latest.

[itowlson]

@divya-mohan0209 I've raised an issue for the different versions. It looks like the shim quickstart was updated within the last hour to new versions; the previous one looks quite old but I'm not sure how k3d image versions map to shim versions...!

[divya-mohan0209]

Thanks @itowlson ! I'll try out the new quickstart guide right now & get back to you :)

[vdice]

@divya-mohan0209 checking in on this one. The containerd-shim-spin project has since migrated to https://github.com/spinkube/containerd-shim-spin and an updated quickstart exists at https://www.spinkube.dev/docs/spin-operator/quickstart/. Wondering if you'd be interested in trying the new shim/quickstart at some point?

If any issue arise, can you please file in the https://github.com/spinkube/containerd-shim-spin repo (or potentially https://github.com/spinkube/documentation)? Thank you!

[divya-mohan0209]

@vdice : I'm so sorry I dropped the ball on this! I'd definitely like to provide feedback. But I'd also be able to do so after KubeCon.

[vdice]

@divya-mohan0209 no worries, thanks so much! Enjoy KubeCon :)