Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reduce CI usage #83

Merged
merged 3 commits into from
Jul 25, 2023
Merged

Reduce CI usage #83

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
63dca0e
CI: Test multiple Pythons per OS
kurtmckee Feb 5, 2023
965c86a
Build a wheel once
kurtmckee Feb 5, 2023
1086eee
CI: Cache virtual environments
kurtmckee Feb 6, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 29 additions & 15 deletions .github/workflows/tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ env:

jobs:
tests:
name: "${{ matrix.python-version }} on ${{ matrix.os }}"
name: "Test on ${{ matrix.os }}"
runs-on: "${{ matrix.os }}-latest"

strategy:
Expand All @@ -31,29 +31,43 @@ jobs:
- ubuntu
- macos
- windows
python-version:
- "3.7"
- "3.8"
- "3.9"
- "3.10"
- "3.11"
- "pypy-3.9"

steps:
- name: "Check out the repo"
uses: "actions/checkout@v3"

- name: "Set up Python"
id: "setup-python"
uses: "actions/setup-python@v4"
with:
python-version: "${{ matrix.python-version }}"
cache: pip
cache-dependency-path: 'requirements/*.txt'
# The last listed Python version is the default.
python-version: |
pypy-3.9
3.7
3.8
3.9
3.10
3.11

- name: "Restore cache"
id: "restore-cache"
uses: "actions/cache@v3"
with:
path: |
.tox/
.venv/
key: "cache-python-${{ steps.setup-python.outputs.python-version }}-os-${{ runner.os }}-hash-${{ hashFiles('tox.ini', 'requirements/*.txt') }}"
Comment on lines +52 to +59
Copy link
Contributor Author

@kurtmckee kurtmckee Feb 7, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By caching the .tox/ and .venv/ directories, the virtual environment and the Tox environments can be restored as filesystem operations, rather than as an installation of Python packages.

The cache key includes a hash of tox.ini and all of the requirements/*.txt files changes. This ensures that a cache miss occurs if any of those files changes.

Note that it may be helpful to modify the cache key in the future. In particular, if the last-listed Python version above ("3.11" currently) doesn't change versions but any of the other versions updates (say, from "3.10.x" to "3.10.y"), then:

  1. This would result in a cache hit.
  2. However, Tox would detect that the 3.10 version had changed and would rebuild the py310 test environment.
  3. The cache would not be updated with this new environment, resulting in an increase in test times.

This situation can be improved by adding a step that runs each Python version and writes its exact version to a file. Then, this file could be included in the list of files passed to hashFiles().

I haven't learned how to do this yet, so this improvement isn't included. The only thing that will happen in this situation is that Tox has to discard the Python 3.10.x test environment and rebuild it with 3.10.y, so test times will increase until the cache is invalidated by some other means, like when the requirements/*.txt files are updated.


- name: "Identify venv path"
shell: "bash"
run: "echo 'venv-path=${{ runner.os == 'Windows' && '.venv/Scripts' || '.venv/bin' }}' >> $GITHUB_ENV"
Comment on lines +61 to +63
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step establishes the relative path to the Python executable and writes it to $GITHUB_ENV, which expects INI-like lines that establish an environment variable name and its value. Subsequent steps will have access to a venv-path environment variable.

Although the default shell is specified above to be bash, note that the forward slashes would also work in Windows' default shell, Powershell, so no changes are needed here.

However, I explicitly included the shell: "bash" configuration because the test suite failed for me locally with "path not found" errors while running in Powershell, so I'm anticipating the possibility that using bash as the default shell even when running on Windows is masking a failure. Investigating that was not in-scope for this work, but I tried to anticipate a future change to running Powershell on Windows in CI.


- name: "Install dependencies"
if: "steps.restore-cache.outputs.cache-hit == false"
run: |
python -m pip install -U setuptools
python -m pip install -r requirements/tox.txt
python -m venv .venv
${{ env.venv-path }}/python -m pip install -U setuptools
${{ env.venv-path }}/python -m pip install -r requirements/tox.txt
Comment on lines 65 to +70
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step is expensive, so it only happens when there's a cache miss. A virtual environment is used because it's far easier to cache a virtual environment at a path of our choosing than to try caching items in an OS-specific location.


- name: "Install pandoc on Linux"
# sudo apt-get pandoc: will install a version from 2018!
Expand All @@ -72,9 +86,9 @@ jobs:
run: |
choco install -y -r --no-progress pandoc

- name: "Run tox for ${{ matrix.python-version }}"
- name: "Run tox"
run: |
python -m tox
${{ env.venv-path }}/python -m tox -m ci-tests
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Tox invocation changes from a bare run of Tox (which was intercepted by tox-gh and transformed to a single run of the default test environment) to use Tox 4's new labels. This allows the tox.ini file to control which environments the ci-tests label is associated with.


- name: "Upload coverage data"
uses: actions/upload-artifact@v3
Expand Down
3 changes: 0 additions & 3 deletions requirements/dev.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -366,9 +366,6 @@ tomlkit==0.11.8
tox==4.6.4
# via
# -r requirements/tox.txt
# tox-gh
tox-gh==1.2.0
# via -r requirements/tox.txt
twine==4.0.2
# via -r requirements/quality.txt
typed-ast==1.5.5
Expand Down
1 change: 0 additions & 1 deletion requirements/tox.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,3 @@
-c constraints.txt

tox # Virtualenv management for tests
tox-gh # Run tox environment based on the GitHub action matrix
3 changes: 0 additions & 3 deletions requirements/tox.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,6 @@ tomli==2.0.1
tox==4.6.4
# via
# -r requirements/tox.in
# tox-gh
tox-gh==1.2.0
# via -r requirements/tox.in
typing-extensions==4.7.1
# via
# importlib-metadata
Expand Down
14 changes: 4 additions & 10 deletions tox.ini
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,12 @@ envlist =
py3{7,11}-no_extras,
# And the rest:
pypy3, coverage, docs, quality
labels =
ci-tests = py3{7,8,9,10,11},pypy3

[testenv]
package = wheel
wheel_build_env = .pkg
deps =
-r{toxinidir}/requirements/test.txt
extras =
Expand Down Expand Up @@ -61,13 +65,3 @@ commands =
commands =
python -m pip install -U pip
make upgrade

[gh]
# https://pypi.org/project/tox-gh/
python =
3.7 = py37, py37-no_extras
3.8 = py38
3.9 = py39
3.10 = py310
3.11 = py311, py311-no_extras
pypy3 = pypy3