Skip to content

Commit

Permalink
[GHA] Introduce reusable matrix workflow
Browse files Browse the repository at this point in the history
# Motivation

We have a few checks that we want to run across all supported Swift versions e.g. unit tests and benchmarks. Right now we already added two matrix based jobs to the reusable workflow. This leads to a lot of duplication and harder maintenance.

# Modification

This PR splits off the current reusable workflow to only contain soundness related checks and introduces a new reusable matrix based workflow which can be customized to execute different commands.

# Result

It is now easy to setup multiple matrix based workflows.
  • Loading branch information
FranzBusch committed Jul 15, 2024
1 parent 7b03cb2 commit a8ed467
Show file tree
Hide file tree
Showing 5 changed files with 153 additions and 72 deletions.
22 changes: 18 additions & 4 deletions .github/workflows/pull_request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,24 @@ on:
types: [opened, reopened, synchronize, ready_for_review]

jobs:
call-reusable-pull-request-workflow:
name: Checks
uses: ./.github/workflows/reusable_pull_request.yml
call-pull-request-soundness-workflow:
name: Soundness
uses: ./.github/workflows/pull_request_soundness.yml
with:
benchmarks_linux_package_path: "Benchmarks"
license_header_check_project_name: "SwiftNIO"
format_check_enabled: false

call-pull-request-unit-tests-workflow:
name: Unit tests
uses: ./.github/workflows/pull_request_swift_matrix.yml
with:
name: "Unit tests"
matrix_linux_command: "swift test -Xswiftc -warnings-as-errors --explicit-target-dependency-import-check error"

call-pull-request-benchmark-workflow:
name: Benchmarks
uses: ./.github/workflows/pull_request_swift_matrix.yml
with:
name: "Benchmarks"
matrix_linux_command: "apt-get update -y -q && apt-get install -y -q libjemalloc-dev && swift package --package-path Benchmarks/ --disable-sandbox benchmark baseline check --check-absolute-path Benchmarks/Thresholds/${SWIFT_VERSION}/
"
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,6 @@ name: Pull Request
on:
workflow_call:
inputs:
unit_tests_linux_enabled:
type: boolean
description: "Boolean to enable the unit tests linux job. Defaults to true."
default: true
benchmarks_linux_enabled:
type: boolean
description: "Boolean to enable the benchmarks linux job. Defaults to true."
default: true
benchmarks_linux_package_path:
type: string
description: "Path to the package containing the benchmarks. Defaults to the repository root."
default: "."
api_breakage_check_enabled:
type: boolean
description: "Boolean to enable the API breakage check job. Defaults to true."
Expand Down Expand Up @@ -50,64 +38,10 @@ on:

## We are cancelling previously triggered workflow runs
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
group: ${{ github.workflow }}-${{ github.ref }}-soundness
cancel-in-progress: true

jobs:
unit-tests-linux:
name: Unit tests
if: ${{ inputs.unit_tests_linux_enabled }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
swift:
- image: swift:5.8-jammy
- image: swift:5.9-jammy
- image: swift:5.10-jammy
- image: swiftlang/swift:nightly-6.0-jammy
- image: swiftlang/swift:nightly-main-jammy
container:
image: ${{ matrix.swift.image }}
timeout-minutes: 20
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Run tests
run: swift test

benchmarks-linux:
name: Benchmarks
if: ${{ inputs.benchmarks_linux_enabled }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
swift:
- image: swift:5.8-jammy
swift_version: "5.8"
- image: swift:5.9-jammy
swift_version: "5.9"
- image: swift:5.10-jammy
swift_version: "5.10"
- image: swiftlang/swift:nightly-6.0-jammy
swift_version: "nightly-next"
- image: swiftlang/swift:nightly-main-jammy
swift_version: "nightly-main"
container:
image: ${{ matrix.swift.image }}
timeout-minutes: 20
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Run tests
env:
PACKAGE_PATH: ${{ inputs.benchmarks_linux_package_path }}
SWIFT_VERSION: ${{ matrix.swift.swift_version }}
run: |
apt-get update -y -q && apt-get install -y -q libjemalloc-dev
swift package --package-path ${PACKAGE_PATH} --disable-sandbox benchmark baseline check --check-absolute-path ${PACKAGE_PATH}/Thresholds/${SWIFT_VERSION}/
api-breakage-check:
name: API breakage check
if: ${{ inputs.api_breakage_check_enabled }}
Expand Down
84 changes: 84 additions & 0 deletions .github/workflows/pull_request_swift_matrix.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
name: Pull Request

on:
workflow_call:
inputs:
name:
type: string
description: "The name of the workflow used for the concurrency group."
required: true
matrix_linux_command:
type: string
description: "The command of the current Swift version linux matrix job to execute."
required: true
matrix_linux_5_8_enabled:
type: string
description: "The command of the 5.8 Swift version linux matrix job to execute."
matrix_linux_5_8_command_override:
type: string
description: "The command of the 5.8 Swift version linux matrix job to execute."
matrix_linux_5_9_enabled:
type: string
description: "The command of the 5.9 Swift version linux matrix job to execute."
matrix_linux_5_9_command_override:
type: string
description: "The command of the 5.9 Swift version linux matrix job to execute."
matrix_linux_5_10_enabled:
type: string
description: "The command of the 5.10 Swift version linux matrix job to execute."
matrix_linux_5_10_command_override:
type: string
description: "The command of the 5.10 Swift version linux matrix job to execute."
matrix_linux_nightly_next_enabled:
type: string
description: "The command of the nightly next Swift version linux matrix job to execute."
matrix_linux_nightly_next_command_override:
type: string
description: "The command of the nightly next Swift version linux matrix job to execute."
matrix_linux_nightly_main_enabled:
type: string
description: "The command of the nightly main Swift version linux matrix job to execute."
matrix_linux_nightly_main_command_override:
type: string
description: "The command of the nightly main Swift version linux matrix job to execute."

## We are cancelling previously triggered workflow runs
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}-${{ inputs.name }}
cancel-in-progress: true

jobs:
linux:
name: ${{ matrix.swift.swift_version }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
swift:
- image: swift:5.8-jammy
swift_version: "5.8"
- image: swift:5.9-jammy
swift_version: "5.9"
- image: swift:5.10-jammy
swift_version: "5.10"
- image: swiftlang/swift:nightly-6.0-jammy
swift_version: "nightly-next"
- image: swiftlang/swift:nightly-main-jammy
swift_version: "nightly-main"
container:
image: ${{ matrix.swift.image }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Run matrix job
env:
SWIFT_VERSION: ${{ matrix.swift.swift_version }}
COMMAND: ${{ inputs.matrix_linux_command }}
COMMAND_OVERRIDE_5_8: ${{ inputs.matrix_linux_5_8_command_override }}
COMMAND_OVERRIDE_5_9: ${{ inputs.matrix_linux_5_9_command_override }}
COMMAND_OVERRIDE_5_10: ${{ inputs.matrix_linux_5_10_command_override }}
COMMAND_OVERRIDE_NIGHTLY_NEXT: ${{ inputs.matrix_linux_nightly_next_command_override }}
COMMAND_OVERRIDE_NIGHTLY_MAIN: ${{ inputs.matrix_linux_nightly_main_command_override }}
run: |
apt-get -qq update && apt-get -qq -y install curl
curl -s https://raw.githubusercontent.com/apple/swift-nio/main/scripts/check-matrix-job.sh | bash
49 changes: 49 additions & 0 deletions scripts/check-matrix-job.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
#!/bin/bash
##===----------------------------------------------------------------------===##
##
## This source file is part of the SwiftNIO open source project
##
## Copyright (c) 2024 Apple Inc. and the SwiftNIO project authors
## Licensed under Apache License v2.0
##
## See LICENSE.txt for license information
## See CONTRIBUTORS.txt for the list of SwiftNIO project authors
##
## SPDX-License-Identifier: Apache-2.0
##
##===----------------------------------------------------------------------===##
set -euo pipefail

log() { printf -- "** %s\n" "$*" >&2; }
error() { printf -- "** ERROR: %s\n" "$*" >&2; }
fatal() { error "$@"; exit 1; }

test -n "${SWIFT_VERSION:-}" || fatal "SWIFT_VERSION unset"
test -n "${COMMAND:-}" || fatal "COMMAND unset"
swift_version="$SWIFT_VERSION"
command="$COMMAND"
command_5_8="$COMMAND_OVERRIDE_5_8"
command_5_9="$COMMAND_OVERRIDE_5_9"
command_5_10="$COMMAND_OVERRIDE_5_10"
command_nightly_next="$COMMAND_OVERRIDE_NIGHTLY_NEXT"
command_nightly_main="$COMMAND_OVERRIDE_NIGHTLY_MAIN"

if [ "$swift_version" == "5.8" ] && [ -n "$command_5_8" ]; then
log "Running 5.8 command override"
eval "$command_5_8"
elif [ "$swift_version" == "5.9" ] && [ -n "$command_5_9" ]; then
log "Running 5.9 command override"
eval "$command_5_9"
elif [ "$swift_version" == "5.10" ] && [ -n "$command_5_10" ]; then
log "Running 5.10 command override"
eval "$command_5_10"
elif [ "$swift_version" == "nightly-next" ] && [ -n "$command_nightly_next" ]; then
log "Running nightly next command override"
eval "$command_nightly_next"
elif [ "$swift_version" == "nightly-main" ] && [ -n "$command_nightly_main" ]; then
log "Running nightly main command override"
eval "$command_nightly_main"
else
log "Running default command"
eval "$command"
fi
2 changes: 1 addition & 1 deletion scripts/soundness.sh
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ for language in swift-or-c bash dtrace python; do
matching_files=( -name '*' )
case "$language" in
swift-or-c)
exceptions=( -name c_nio_llhttp.c -o -name c_nio_api.c -o -name c_nio_http.c -o -name c_nio_llhttp.h -o -name cpp_magic.h -o -name Package.swift -o -name 'Package@*.swift' -o -path './Snippets/*' -o -name CNIOSHA1.h -o -name c_nio_sha1.c -o -name ifaddrs-android.c -o -name ifaddrs-android.h)
exceptions=( -name c_nio_llhttp.c -o -name c_nio_api.c -o -name c_nio_http.c -o -name c_nio_llhttp.h -o -name cpp_magic.h -o -name Package.swift -o -name 'Package@*.swift' -o -path './Snippets/*' -o -name CNIOSHA1.h -o -name c_nio_sha1.c -o -name ifaddrs-android.c -o -name ifaddrs-android.h -o -name pull_request_soundness.yml)
matching_files=( -name '*.swift' -o -name '*.c' -o -name '*.h' )
cat > "$tmp" <<"EOF"
//===----------------------------------------------------------------------===//
Expand Down

0 comments on commit a8ed467

Please sign in to comment.