Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[POC] Implement caching of .build for unit-tests #2874

Closed
wants to merge 1 commit into from
Closed

[POC] Implement caching of .build for unit-tests #2874

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 22 additions & 1 deletion .github/workflows/swift_matrix.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ on:
type: string
description: "The name of the workflow used for the concurrency group."
required: true
cache_key_prefix:
type: string
description: "The prefix for the cache key. If empty, caching is disabled"
default: ''
matrix_linux_command:
type: string
description: "The command of the current Swift version linux matrix job to execute."
Expand Down Expand Up @@ -71,7 +75,7 @@ on:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}-${{ inputs.name }}
cancel-in-progress: true

jobs:
linux:
name: Linux (${{ matrix.swift.swift_version }})
Expand Down Expand Up @@ -108,6 +112,16 @@ jobs:
if: ${{ matrix.swift.enabled }}
# https://github.com/actions/checkout/issues/766
run: git config --global --add safe.directory ${GITHUB_WORKSPACE}

- name: Restore .build
if: ${{ matrix.swift.enabled && (inputs.cache_key_prefix != '') }}
id: "restore-cache"
uses: actions/cache/restore@v4
with:
path: .build
key: "${{ inputs.cache_key_prefix }}${{ github.event.repository.name }}-build-${{ matrix.swift.swift_version }}-${{ runner.os }}-${{ (github.event.pull_request.base.sha != null) && github.event.pull_request.base.sha || github.event.after }}"
Copy link
Contributor Author

@MahdiBM MahdiBM Sep 11, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

${{ (github.event.pull_request.base.sha != null) && github.event.pull_request.base.sha || github.event.after }}
is the main thing that can be different for a specific swift version.
It allows the cache to be updated after each push to the main branch, since it'll resolve to github.event.after.

It either resolves to the base branch's SHA in PR events, or to the new SHA of the branch in push events.
So each PR will be looking for their base branch's cached .build.
And on a push to the main branch, the cache is refreshed.
It will be suboptimal, but even the event isn't a push or PR, the CI should still use the cache. But maybe also waste some seconds on trying to re-save the cache.

One confusing behavior of the caches action is that in a PR, you cannot create a cache that the next commits in a PR can use. That's why the cache "key" doesn't try to do something like that.
You can only access the repo's primary-branch's caches. [Edit: actually not 100% sure about this, this "rule" might be even more weird, but still, what is in this PR makes sense]

FYI another gotcha of the caches action is that you cannot delete OR override value of a cache "key" (not programmatically, otherwise there is a delete button in GitHub web UI).

restore-keys: "${{ inputs.cache_key_prefix }}${{ github.event.repository.name }}-build-${{ matrix.swift.swift_version }}-${{ runner.os }}"

- name: Run matrix job
if: ${{ matrix.swift.enabled }}
env:
Expand All @@ -121,3 +135,10 @@ jobs:
run: |
apt-get -qq update && apt-get -qq -y install curl
curl -s https://raw.githubusercontent.com/apple/swift-nio/main/scripts/check-matrix-job.sh | bash

- name: Cache .build
if: ${{ matrix.swift.enabled && (inputs.cache_key_prefix != '') && (steps.restore-cache.outputs.cache-hit != 'true') }}
uses: actions/cache/save@v4
with:
path: .build
key: "${{ inputs.cache_key_prefix }}${{ github.event.repository.name }}-build-${{ matrix.swift.swift_version }}-${{ runner.os }}-${{ (github.event.pull_request.base.sha != null) && github.event.pull_request.base.sha || github.event.after }}"
3 changes: 2 additions & 1 deletion .github/workflows/unit_tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,13 +43,14 @@ on:
type: string
description: "The arguments passed to swift test in the Linux nightly main Swift version matrix job."
default: ""

jobs:
unit-tests:
name: Unit tests
uses: ./.github/workflows/swift_matrix.yml
with:
name: "Unit tests"
cache_key_prefix: "unit-tests-"
matrix_linux_command: "swift test"
matrix_linux_5_8_enabled: ${{ inputs.linux_5_8_enabled }}
matrix_linux_5_8_command_override: "swift test ${{ inputs.linux_5_8_arguments_override }}"
Expand Down