Build components with GitHub Actions and buildx, add container build …

…to integration testing (#217)

* Update generate-keys to use buildx

Signed-off-by: Alessandro Pomponio <[email protected]>

* Update dataset-operator to buildx

Signed-off-by: Alessandro Pomponio <[email protected]>

* Update csi-s3 to use buildx

Signed-off-by: Alessandro Pomponio <[email protected]>

* Create scripts for building the dataset operator via build and buildx

Signed-off-by: Alessandro Pomponio <[email protected]>

* Update dataset-operator to build using Go 1.19

Signed-off-by: Alessandro Pomponio <[email protected]>

* Remove dataset-operator Makefile

Signed-off-by: Alessandro Pomponio <[email protected]>

* Create scripts to build generate keys via build and buildx

Signed-off-by: Alessandro Pomponio <[email protected]>

* Remove generate-keys Makefile

Signed-off-by: Alessandro Pomponio <[email protected]>

* Update build_components to use new build scripts

Signed-off-by: Alessandro Pomponio <[email protected]>

* Update build_components and related scripts

Signed-off-by: Alessandro Pomponio <[email protected]>

* Add building component images to integration testing

Signed-off-by: Alessandro Pomponio <[email protected]>

* Correctly switch to build tools directory

Signed-off-by: Alessandro Pomponio <[email protected]>

* Create KinD cluster after building images

Signed-off-by: Alessandro Pomponio <[email protected]>

* Stop performing integration test on every push

Signed-off-by: Alessandro Pomponio <[email protected]>

* Create new Action for building and pushing components images

Signed-off-by: Alessandro Pomponio <[email protected]>

* Change interactive login to normal login

Signed-off-by: Alessandro Pomponio <[email protected]>

* Revert "Change interactive login to normal login"

This reverts commit 9deef45.

Signed-off-by: Alessandro Pomponio <[email protected]>

* Add option to skip docker login on buildx

Signed-off-by: Alessandro Pomponio <[email protected]>

* Use environment secrets and docker login action

Signed-off-by: Alessandro Pomponio <[email protected]>

* Use env variable for registry url

Signed-off-by: Alessandro Pomponio <[email protected]>

* Change definition of REGISTRY_URL

Signed-off-by: Alessandro Pomponio <[email protected]>

* Pass REGISTRY_URL to scripts

Signed-off-by: Alessandro Pomponio <[email protected]>

* Build Operator and Generate Keys in different jobs, correctly pass REGISTRY_URL

Signed-off-by: Alessandro Pomponio <[email protected]>

* Use vars instead of env

Signed-off-by: Alessandro Pomponio <[email protected]>

* Only build and push multiarch images on commits to main branch

Signed-off-by: Alessandro Pomponio <[email protected]>

* Comment buildx instructions for csi-s3 and re-add docker build instructions

Signed-off-by: Alessandro Pomponio <[email protected]>

* Add comment explaining git checkout for goofys

Signed-off-by: Alessandro Pomponio <[email protected]>

* Add missing . in Makefile

Signed-off-by: Alessandro Pomponio <[email protected]>

* Fix use of REGISTRY_URL

Signed-off-by: Alessandro Pomponio <[email protected]>

* Do not run integration testing on PR message edits

Signed-off-by: Alessandro Pomponio <[email protected]>

Signed-off-by: Alessandro Pomponio <[email protected]>

.github/workflows/push-updated-images.yml

@@ -0,0 +1,45 @@
+name: Build and push multiarch images
+on:
+  push:
+    branches:
+      - master
+
+jobs:
+  dataset-operator:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Clone Datashim
+        uses: actions/checkout@v2
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v2
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v2
+      - name: Login to Container Registry
+        uses: docker/login-action@v2
+        with:
+          registry: ${{ vars.REGISTRY_URL }}
+          username: ${{ secrets.REGISTRY_USERNAME }}
+          password: ${{ secrets.REGISTRY_PASSWORD }}
+      - name: Build and push Dataset Operator
+        run: |
+          cd src/dataset-operator
+          ./build_and_push_multiarch_dataset_operator.sh ${{ vars.REGISTRY_URL }}
+  generate-keys:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Clone Datashim
+        uses: actions/checkout@v2
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v2
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v2
+      - name: Login to Container Registry
+        uses: docker/login-action@v2
+        with:
+          registry: ${{ vars.REGISTRY_URL }}
+          username: ${{ secrets.REGISTRY_USERNAME }}
+          password: ${{ secrets.REGISTRY_PASSWORD }}
+      - name: Build and push Generate Keys
+        run: |
+          cd src/generate-keys
+          ./build_and_push_multiarch_generate_keys.sh ${{ vars.REGISTRY_URL }}

.github/workflows/testing.yml

@@ -1,18 +1,24 @@
 name: Test on KinD
 on:
   pull_request:
-    types: [ready_for_review, review_requested, opened, edited, reopened, synchronize]
+    types: [ready_for_review, review_requested, opened, reopened, synchronize]
 
 jobs:
   integration-test:
     runs-on: ubuntu-latest
     steps:
       - name: Clone Datashim
         uses: actions/checkout@v2
-      - name: Create k8s Kind Cluster
-        uses: helm/[email protected]
+      - name: Build components
+        run: |
+          cd build-tools
+          ./build_components.sh
       - name: Make Datashim manifests
         run: make manifests
+      - name: Update manifests to use local images
+        run: "sed -i 's/imagePullPolicy: Always/imagePullPolicy: IfNotPresent/g' release-tools/manifests/dlf.yaml"
+      - name: Create k8s Kind Cluster
+        uses: helm/[email protected]
       - name: Install Datashim
         run: make deployment
       - name: Install NooBaa

build-tools/build_components.sh

@@ -1,16 +1,45 @@
 #!/bin/bash
+set -e
 
-MAKE_ARGS="build-components"
+print_usage() {
+    echo "Usage: $0 [-k] [-p] [-s]"
+    echo "Use -k to avoid creating another buildx context"
+    echo "Use -p to build and push multiarch images"
+    echo "Use -s to skip logging in to the container registry"
+}
 
-if [ -n "$DOCKER_REGISTRY" ]
-then
-      MAKE_ARGS+=" push-components"
+BUILD_AND_PUSH="no"
+CREATE_NEW_BUILDX_CONTEXT="yes"
+SKIP_LOGIN="no"
+while getopts 'kps' OPTION
+do
+    case "$OPTION" in
+        k)
+            CREATE_NEW_BUILDX_CONTEXT="no"
+            ;;
+        p)
+            BUILD_AND_PUSH="yes"
+            ;;
+        s)
+            SKIP_LOGIN="yes"
+            ;;
+        ?)
+            print_usage >&2
+            exit 1
+            ;;
+    esac
+done
+
+if [ $BUILD_AND_PUSH = "yes" ]; then
+      if [ $SKIP_LOGIN = "no" ]; then
+            echo $REGISTRY_PASSWORD | docker login -u $REGISTRY_USERNAME --password-stdin $REGISTRY_URL
+      fi
+      if [ $CREATE_NEW_BUILDX_CONTEXT = "yes" ]; then
+            docker buildx create --use
+      fi
+      (cd ../src/dataset-operator && ./build_and_push_multiarch_dataset_operator.sh $REGISTRY_URL)
+      (cd ../src/generate-keys && ./build_and_push_multiarch_generate_keys.sh $REGISTRY_URL)
 else
-      DOCKER_REGISTRY="local"
+      (cd ../src/dataset-operator && ./build_dataset_operator.sh $REGISTRY_URL)
+      (cd ../src/generate-keys && ./build_generate_keys.sh $REGISTRY_URL)
 fi
-
-export ARCH=`arch`;
-if [ "$ARCH" == "x86_64" ]; then export ARCH="amd64"; fi
-if [ "$ARCH" == "i386" ]; then export ARCH="amd64"; fi
-if [ "$ARCH" == "aarch64" ]; then export ARCH="arm64"; fi
-make ARCH=$ARCH DOCKER_REGISTRY=$DOCKER_REGISTRY $MAKE_ARGS

src/csi-s3/Makefile

@@ -39,6 +39,8 @@ test:
 container:
 #	docker build -t $(IMAGE_TAG) -f cmd/s3driver/Dockerfile .
 	docker build --network=host -t $(FULL_IMAGE_TAG) --build-arg VERSION=$(VERSION) --build-arg ARCH=$(ARCH) -f cmd/s3driver/Dockerfile.full .
+# 	docker buildx create --use --buildkitd-flags '--allow-insecure-entitlement network.host'
+#	docker buildx build --platform linux/amd64,linux/arm64,linux/ppc64le --network=host -t $(FULL_IMAGE_TAG) --build-arg VERSION=$(VERSION) -f cmd/s3driver/Dockerfile.full .
 push: container
 	docker push $(IMAGE_TAG)
 	docker push $(FULL_IMAGE_TAG)

src/csi-s3/cmd/s3driver/Dockerfile.full

@@ -1,8 +1,6 @@
 ARG ARCH
 FROM golang:1.18-alpine3.15 as base
 
-ENV ARCH=${ARCH}
-
 ARG S3BACKER_VERSION=1.5.0
 
 RUN apk add --update --no-cache \
@@ -23,16 +21,17 @@ RUN ls -l && \
 # Datashim's CSI-S3 is only tested with goofys at present so the others are redundant for now
 # Build goofys 
 # Add additional line to go.mod to deal with CVE-2021-38561
+
+# AP: NOTE: git checkout is required because as of 27/01/2023
+# goofys cannot be built.
+# This instruction can be safely removed once
+# https://github.com/kahing/goofys/issues/740 has been fixed
 WORKDIR /go/src
 RUN git clone https://github.com/kahing/goofys.git && \
     cd goofys && \
+    git checkout e903e56038fe0325e4538007b6acc064af8164c7 && \ 
     sed -i -e '/require/ a \\tgolang.org/x/text v0.3.7' go.mod && \
     go mod tidy && \
-    if [ "$ARCH" = "ppc64le" ]; then \
-	  sed -i 's/4096/65536/g' /go/pkg/mod/github.com/kahing/fuse*/internal/buffer/in_message.go ; \
-	  sed -i 's/17/21/g' /go/pkg/mod/github.com/kahing/fuse*/internal/buffer/in_message_linux.go ; \
-	  sed -i 's/17/21/g' /go/pkg/mod/github.com/kahing/fuse*/internal/buffer/out_message_linux.go ; \
-    else echo 'true'; fi && \
     go build -o /go/bin/goofys
 
 # Compile & install s3backer

src/dataset-operator/Dockerfile

@@ -1,8 +1,5 @@
 # Build the manager binary
-FROM golang:1.17 as builder
-
-ARG ARCH 
-ENV ARCH=${ARCH}
+FROM golang:1.19 as builder
 
 WORKDIR /dataset-operator
 # Copy the Go Modules manifests
@@ -20,7 +17,7 @@ COPY version/ version/
 COPY admissioncontroller/ admissioncontroller/
 
 # Build
-RUN CGO_ENABLED=0 GOOS=linux GOARCH=${ARCH} go build -a -o manager main.go
+RUN CGO_ENABLED=0 GOOS=linux go build -a -o manager main.go
 
 # Use distroless as minimal base image to package the manager binary
 # Refer to https://github.com/GoogleContainerTools/distroless for more details

src/dataset-operator/build_and_push_multiarch_dataset_operator.sh

@@ -0,0 +1,4 @@
+#!/bin/bash
+
+REGISTRY_URL="${1:-quay.io/datashim-io}"
+docker buildx build --platform linux/amd64,linux/arm64,linux/ppc64le --push -t ${REGISTRY_URL}/dataset-operator .

src/dataset-operator/build_dataset_operator.sh

@@ -0,0 +1,4 @@
+#!/bin/bash
+
+REGISTRY_URL="${1:-quay.io/datashim-io}"
+docker build -t ${REGISTRY_URL}/dataset-operator .

src/generate-keys/build_and_push_multiarch_generate_keys.sh

@@ -0,0 +1,4 @@
+#!/bin/bash
+
+REGISTRY_URL="${1:-quay.io/datashim-io}"
+docker buildx build --platform linux/amd64,linux/arm64,linux/ppc64le --push -t ${REGISTRY_URL}/generate-keys .

src/generate-keys/build_generate_keys.sh

@@ -0,0 +1,4 @@
+#!/bin/bash
+
+REGISTRY_URL="${1:-quay.io/datashim-io}"
+docker build -t ${REGISTRY_URL}/generate-keys .