Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove Go dev environment from Alpine builder #480

Merged
merged 4 commits into from
Nov 29, 2023
Merged

Remove Go dev environment from Alpine builder #480

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
dfacf50
Avoid using go-ext-builder for running Go tests
webmaster128 Nov 27, 2023
bf7e17a
Remove Go environment from Rust builder image
webmaster128 Nov 28, 2023
5f681fd
Run test_alpine_build in CI for this PR
webmaster128 Nov 29, 2023
5059f30
Add helpful error message when trying to execute test-alpine on ARM
webmaster128 Nov 29, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -457,7 +457,8 @@ workflows:
only:
# long living branches
- main
- 0.14-dev
# Development
- no-go-in-builder
# Run only on main, not on tags (auto-build on merge PR)
- deploy_to_git:
requires:
Expand Down
7 changes: 7 additions & 0 deletions Dockerfile.alpine_tester
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
FROM golang:1.21.4-alpine

RUN apk add build-base

# prepare go cache dirs
RUN mkdir -p /.cache/go-build
RUN chmod -R 777 /.cache
12 changes: 9 additions & 3 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,9 @@

# Builds the Rust library libwasmvm
BUILDERS_PREFIX := cosmwasm/go-ext-builder:0017
# Contains a full Go dev environment in order to run Go tests on the built library
ALPINE_TESTER := cosmwasm/go-ext-builder:0017-alpine
# Contains a full Go dev environment including CGO support in order to run Go tests on the built shared library
# This image is currently not published.
ALPINE_TESTER := cosmwasm/alpine-tester:local

USER_ID := $(shell id -u)
USER_GROUP = $(shell id -g)
Expand Down Expand Up @@ -123,7 +124,12 @@ release-build:
make release-build-macos
make release-build-windows

test-alpine: release-build-alpine
.PHONY: create-tester-image
create-tester-image:
docker build -t $(ALPINE_TESTER) - < ./Dockerfile.alpine_tester

test-alpine: release-build-alpine create-tester-image
Copy link
Collaborator

@chipshort chipshort Nov 29, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On my Macbook test-alpine fails with an error about incompatible libwasmvm_muslc.a (during the go build -tags muslc ./.... Any idea why?

/usr/local/go/pkg/tool/linux_arm64/link: running gcc failed: exit status 1
/usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: skipping incompatible /mnt/testrun/internal/api/libwasmvm_muslc.a when searching for -lwasmvm_muslc
/usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: cannot find -lwasmvm_muslc: No such file or directory
collect2: error: ld returned 1 exit status

I guess it's platform issues?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, nice find. It's possible I never tested that because the release-build-alpine build is so slow.

I guess the reason is that unlike before, $(ALPINE_TESTER) is not always an x86 image anymore. Instead, it's built for arm on arm machines. Will double check.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, this cannot work. I added a helpful error message and ticket here: #483. First step would be to change the output file name in the Apline builder. But that's for another day.

@[ "$(shell uname -m)" = "x86_64" ] || (echo "This test is only working on x86_64. See https://github.com/CosmWasm/wasmvm/issues/483."; exit 78)
# try running go tests using this lib with muslc
docker run --rm -u $(USER_ID):$(USER_GROUP) -v $(shell pwd):/mnt/testrun -w /mnt/testrun $(ALPINE_TESTER) go build -tags muslc ./...
# Use package list mode to include all subdirectores. The -count=1 turns off caching.
Expand Down
28 changes: 2 additions & 26 deletions builders/Dockerfile.alpine
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,26 +1,6 @@
# This image is used for two things (which is not ideal, but yeah):
# 1. Build the static Rust library
# 2. Execute Go tests that use and test this library
# For 2. we define the Go image here. For 1. we install Rust below.
#
# See also docs/COMPILER_VERSIONS.md
FROM golang:1.20.10-alpine

ENV RUSTUP_HOME=/usr/local/rustup \
CARGO_HOME=/usr/local/cargo \
PATH=/usr/local/cargo/bin:$PATH
FROM rust:1.73.0-alpine

# this comes from standard alpine nightly file
# https://github.com/rust-lang/docker-rust-nightly/blob/master/alpine3.12/Dockerfile
# with some changes to support our toolchain, etc
RUN set -eux \
&& apk add --no-cache ca-certificates build-base

RUN wget "https://static.rust-lang.org/rustup/dist/x86_64-unknown-linux-musl/rustup-init" \
&& chmod +x rustup-init \
&& ./rustup-init -y --no-modify-path --profile minimal --default-toolchain 1.73.0 \
&& rm rustup-init \
&& chmod -R a+w $RUSTUP_HOME $CARGO_HOME
RUN apk add --no-cache ca-certificates build-base

# Install C compiler for cross-compilation. This is required by
# Wasmer in https://github.com/wasmerio/wasmer/blob/2.2.1/lib/vm/build.rs.
Expand All @@ -34,10 +14,6 @@ RUN wget https://musl.cc/aarch64-linux-musl-cross.tgz \
&& /opt/aarch64-linux-musl-cross/bin/aarch64-linux-musl-gcc --version \
&& rm aarch64-linux-musl-cross.tgz

# prepare go cache dirs
RUN mkdir -p /.cache/go-build
RUN chmod -R 777 /.cache

# allow non-root user to download more deps later
RUN chmod -R 777 /usr/local/cargo

Expand Down
4 changes: 4 additions & 0 deletions builders/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,10 @@ versions of the builder images.

## Changelog

**Unreleased**

- Remove Go dev environment from `cosmwasm/go-ext-builder:XXXX-alpine`

**Version 0017:**

- Update Rust to 1.73.0.
Expand Down