Merge pull request #3 from OKDP/concurrency

Maximize number of parallel jobs and readme update

.github/workflows/main.yml

@@ -1,4 +1,4 @@
-name: Build, test, and push jupyter images
+name: "[Main] Build, test, and push jupyter images"
 
 on:
   pull_request:
@@ -97,8 +97,6 @@ jobs:
   build-base:
     name: build-test-base (python-${{ matrix.python.python_version }})
     strategy:
-      # 3 Jobs in //, the base jobs run in sequential
-      max-parallel: 3
       matrix:
         python: ${{ fromJson(needs.build-version-compatibility-matrix.outputs.python) }}
     uses: ./.github/workflows/build-test-base.yml
@@ -115,8 +113,6 @@ jobs:
   build-datascience:
     name: build-test-datascience (python-${{ matrix.python.python_version }})
     strategy:
-      # 1 matrix call = +2 jobs in // (check the number here build-datascience.yml)
-      max-parallel: 1
       matrix:
         python: ${{ fromJson(needs.build-version-compatibility-matrix.outputs.python) }}
     uses: ./.github/workflows/build-test-datascience.yml
@@ -132,8 +128,6 @@ jobs:
   build-spark:
     name: build-test-spark (python-${{ matrix.spark.python_version }})
     strategy:
-      # 2 jobs in //
-      max-parallel: 2
       matrix: 
         spark: ${{ fromJson(needs.build-version-compatibility-matrix.outputs.spark) }}
     uses: ./.github/workflows/build-test-spark.yml
@@ -158,7 +152,6 @@ jobs:
     if: github.ref == 'refs/heads/main'
     name: push-base (python-${{ matrix.python.python_version }})
     strategy:
-      max-parallel: 1
       matrix:
         image: [ docker-stacks-foundation, base-notebook, minimal-notebook, scipy-notebook,]
         python: ${{ fromJson(needs.build-version-compatibility-matrix.outputs.python) }}
@@ -176,7 +169,6 @@ jobs:
     if: github.ref == 'refs/heads/main'
     name: push-datascience (python-${{ matrix.python.python_version }})
     strategy:
-      max-parallel: 1
       matrix:
         image: [r-notebook, datascience-notebook, ]
         python: ${{ fromJson(needs.build-version-compatibility-matrix.outputs.python) }}
@@ -193,7 +185,6 @@ jobs:
   push-spark:
     if: github.ref == 'refs/heads/main'
     strategy:
-      max-parallel: 1
       matrix: 
         image: [ pyspark-notebook, all-spark-notebook,]
         spark: ${{ fromJson(needs.build-version-compatibility-matrix.outputs.spark) }}

README.md

@@ -27,7 +27,7 @@ The [main](.github/workflows/main.yml) build pipeline contains 6 main reusable w
 
 The build is based on the [version compatibility matrix](.build/.versions.yml).
 
-The [build-matrix](.build/.versions.yml#L42) section defines the components versions to build. It behaves like a filter of the parent [version compatibility matrix](.build/.versions.yml) to limit the versions combintations to build. The build process ensures only the compatible versions are built:
+The [build-matrix](.build/.versions.yml#L42) section defines the components versions to build. It behaves like a filter of the parent [compatibility-matrix](.build/.versions.yml#L5) section to limit the versions combintations to build. The build process ensures only the compatible versions are built:
 
 For example, the following build-matrix:
 
@@ -62,7 +62,7 @@ Development images with tags ```-<GIT-BRANCH>-latest``` suffix (ex.: spark3.2.4-
 The [official images](#tagging) are pushed to the [container registry](https://github.com/orgs/OKDP/packages) when:
 
 1. The workflow is triggered on the main branch only and
-2. The [tests](#build/test) are completed successfully
+2. The [tests](docker-stacks/tests) are completed successfully
 
 This prevents pull requests or developement branchs to push the official images before they are reviewed or tested. It also provides the flexibility to test against developement images ```-<GIT-BRANCH>-latest``` before they are officially pushed.
 
@@ -99,7 +99,24 @@ Here are some examples:
 
 Please, check the [container registry](https://github.com/orgs/OKDP/packages) for more images and tags.
 
-# Build locally with Act
+# Running github actions
+## Github container registry credentials
+
+Create the following [secrets and configuration variables](https://docs.github.com/en/actions/learn-github-actions/variables#creating-configuration-variables-for-a-repository) when running with your own github account or organization:
+
+| Variable               | Type                    | Default  | Description                                 |
+| -----------------------|-------------------------| ---------| ------------------------------------------- |
+| `REGISTRY`             | Configuration variable  | ghcr.io  | Container registry                          |
+| `REGISTRY_USERNAME`    | Secret variable         |          | Container registry username                 |
+| `REGISTRY_ROBOT_TOKEN` | Secret variable         |          | Container registry password or access token `(Scopes: write:packages/delete:packages)` |
+## Running with Github
+
+By default, the [workflow](.github/workflows/main.yml) runs automatically on the following events:
+
+- Push on the main branch with changes on the configured `paths` filters
+- Pull request on any branch
+
+## Running locally with act
 
 [Act](https://github.com/nektos/act) can be used to build and test locally.