Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions .github/workflows/cache-pixi-lock.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Adapted from https://raw.githubusercontent.com/Parcels-code/Parcels/58cdd6185b3af03785c567914a070288ffd804e0/.github/workflows/cache-pixi-lock.yml
name: Generate and cache Pixi lockfile

on:
workflow_call:
outputs:
cache-id:
description: "The lock file contents"
value: ${{ jobs.cache-pixi-lock.outputs.cache-id }}

jobs:
cache-pixi-lock:
name: Generate output
runs-on: ubuntu-latest
outputs:
cache-id: ${{ steps.restore.outputs.cache-primary-key }}
steps:
- uses: actions/checkout@v5
with:
fetch-depth: 0
submodules: recursive
- name: Get current date
id: date
run: echo "date=$(date +'%Y-%m-%d')" >> "$GITHUB_OUTPUT"
- uses: actions/cache/restore@v4
id: restore
with:
path: |
pixi.lock
key: ${{ steps.date.outputs.date }}_${{hashFiles('pixi.toml')}}
- uses: prefix-dev/[email protected]
if: ${{ !steps.restore.outputs.cache-hit }}
with:
pixi-version: v0.56.0
run-install: false
- name: Run pixi lock
if: ${{ !steps.restore.outputs.cache-hit }}
run: pixi lock
- uses: actions/cache/save@v4
if: ${{ !steps.restore.outputs.cache-hit }}
id: cache
with:
path: |
pixi.lock
key: ${{ steps.restore.outputs.cache-primary-key }}
- name: Upload pixi.lock
uses: actions/upload-artifact@v4
with:
name: pixi-lock
path: pixi.lock
49 changes: 27 additions & 22 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,56 +21,61 @@ env:
FORCE_COLOR: 3

jobs:
cache-pixi-lock:
uses: ./.github/workflows/cache-pixi-lock.yml

tests:
name: tests (${{ matrix.runs-on }} | Python ${{ matrix.python-version }})
name: "Unit tests: ${{ matrix.runs-on }} | pixi run -e ${{ matrix.pixi-environment }} tests"
runs-on: ${{ matrix.runs-on }}
needs: cache-pixi-lock
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.12"]
pixi-environment: ["test-py310", "test-py312"]
runs-on: [ubuntu-latest, windows-latest, macos-14]

steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: mamba-org/setup-micromamba@v2
submodules: recursive
- uses: actions/cache/restore@v4
with:
environment-name: ship
environment-file: environment.yml
create-args: >-
python=${{matrix.python-version}}

- run: pip install . --no-deps
path: pixi.lock
key: ${{ needs.cache-pixi-lock.outputs.cache-id }}
- uses: prefix-dev/[email protected]
with:
cache: true
cache-write: ${{ github.event_name == 'push' && github.ref_name == 'main' }}

- name: Test package
run: >-
python -m pytest -ra --cov --cov-report=xml --cov-report=term
run:
pixi run -e ${{ matrix.pixi-environment }} tests -ra --cov --cov-report=xml --cov-report=term
--durations=20

- name: Upload coverage report
uses: codecov/[email protected]
with:
token: ${{ secrets.CODECOV_TOKEN }}
typechecking:
name: mypy
name: "TypeChecking: pixi run typing"
runs-on: ubuntu-latest
needs: cache-pixi-lock
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: mamba-org/setup-micromamba@v2
submodules: recursive
- uses: actions/cache/restore@v4
with:
environment-name: ship
environment-file: environment.yml
create-args: >-
python=3.12

- run: pip install . --no-deps
- run: conda install lxml # dep for report generation
path: pixi.lock
key: ${{ needs.cache-pixi-lock.outputs.cache-id }}
- uses: prefix-dev/[email protected]
with:
cache: true
cache-write: ${{ github.event_name == 'push' && github.ref_name == 'main' }}
- name: Typechecking
run: |
mypy --install-types --non-interactive src/virtualship --html-report mypy-report
run: pixi run typing --non-interactive --html-report mypy-report
- name: Upload test results
if: ${{ always() }} # Upload even on mypy error
uses: actions/upload-artifact@v4
Expand Down
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -178,3 +178,9 @@ src/virtualship/_version_setup.py

.vscode/
.DS_Store

# Ignore pixi.lock file for this project. The con of 22k lines of noise it adds to diffs is not worth
# the minor benefit of perfectly reproducible environments for all developers (and all the tooling that would
# be required to support that - see https://github.com/pydata/xarray/issues/10732#issuecomment-3327780806
# for more details)
pixi.lock
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "Parcels"]
path = Parcels
url = [email protected]:Parcels-code/Parcels.git
27 changes: 13 additions & 14 deletions .readthedocs.yaml
Original file line number Diff line number Diff line change
@@ -1,18 +1,17 @@
# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details

version: 2
sphinx:
configuration: docs/conf.py
build:
os: ubuntu-22.04
os: ubuntu-lts-latest
tools:
python: mambaforge-22.9
python: "latest" # just so RTD stops complaining
jobs:
pre_build:
- pip install .
- sphinx-build -b linkcheck docs/ _build/linkcheck
- sphinx-apidoc -o docs/api/ --module-first --no-toc --force src/virtualship

conda:
environment: environment.yml
create_environment:
- asdf plugin add pixi
- asdf install pixi latest
- asdf global pixi latest
install:
- pixi install -e docs
build:
html:
- pixi run -e docs sphinx-build -T -b html docs $READTHEDOCS_OUTPUT/html
sphinx:
configuration: docs/conf.py
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@

<!-- Badges -->

[![Pixi Badge](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/prefix-dev/pixi/main/assets/badge/v0.json)](https://pixi.sh)
[![Anaconda-release](https://anaconda.org/conda-forge/virtualship/badges/version.svg)](https://anaconda.org/conda-forge/virtualship/)
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/virtualship)
[![DOI](https://zenodo.org/badge/682478059.svg)](https://doi.org/10.5281/zenodo.14013931)
Expand Down
81 changes: 60 additions & 21 deletions docs/contributing/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,36 +8,75 @@ We have a design document providing a conceptual overview of VirtualShip. This d

### Development installation

We use `conda` to manage our development installation. Make sure you have `conda` installed by following [the instructions here](https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html) and then run the following commands:
```{note}
VirtualShip uses [Pixi](https://pixi.sh) to manage environments and run developer tooling. Pixi is a modern alternative to Conda and also includes other powerful tooling useful for a project like VirtualShip. It is our sole development workflow - we do not offer a Conda development workflow. Give Pixi a try, you won't regret it!
```

To get started contributing to VirtualShip:

**Step 1:** [Install Pixi](https://pixi.sh/latest/).

**Step 2:** [Fork the repository](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo#forking-a-repository)

**Step 3:** Clone your fork with submodules and `cd` into the repository.

```bash
git clone --recurse-submodules [email protected]:YOUR_USERNAME/virtualship.git
cd virtualship
```

```{note}
The `--recurse-submodules` flag is required to clone the Parcels submodule, which is used for testing and development.
```

**Step 4:** Install the Pixi environment

```bash
conda create -n ship python=3.10
conda activate ship
conda env update --file environment.yml
pip install -e . --no-deps --no-build-isolation
pixi install
```

This creates an environment, and installs all the dependencies that you need for development, including:
Now you have a development installation of VirtualShip, as well as a bunch of developer tooling to run tests, check code quality, and build the documentation! Simple as that.

- core dependencies
- development dependencies (e.g., for testing)
- documentation dependencies
### Pixi workflows

then installs the package in editable mode.
You can use the following Pixi commands to run common development tasks.

### Useful commands
**Testing**

The following commands are useful for local development:
- `pixi run tests` - Run the full test suite using pytest with coverage reporting
- `pixi run tests-notebooks` - Run notebook tests

- `pytest` to run tests
- `pre-commit run --all-files` to run pre-commit checks
- `pre-commit install` (optional) to install pre-commit hooks
- this means that every time you commit, pre-commit checks will run on the files you changed
- `sphinx-autobuild docs docs/_build` to build and serve the documentation
- `sphinx-apidoc -o docs/api/ --module-first --no-toc --force src/virtualship` (optional) to generate the API documentation
- `sphinx-build -b linkcheck docs/ _build/linkcheck` to check for broken links in the documentation
**Documentation**

The running of these commands is useful for local development and quick iteration, but not _vital_ as they will be run automatically in the CI pipeline (`pre-commit` by pre-commit.ci, `pytest` by GitHub Actions, and `sphinx` by ReadTheDocs).
- `pixi run docs` - Build the documentation using Sphinx
- `pixi run docs-watch` - Build and auto-rebuild documentation when files change (useful for live editing)

**Code quality**

- `pixi run lint` - Run pre-commit hooks on all files (includes formatting, linting, and other code quality checks)
- `pixi run typing` - Run mypy type checking on the codebase

**Different environments**

VirtualShip supports testing against different environments (e.g., different Python versions) with different feature sets. In CI we test against these environments, and you can too locally. For example:

- `pixi run -e test-py311 tests` - Run tests using Python 3.11
- `pixi run -e test-py312 tests` - Run tests using Python 3.12
- `pixi run -e test-latest tests` - Run tests using latest Python

The name of the workflow on GitHub contains the command you have to run locally to recreate the workflow - making it super easy to reproduce CI failures locally.

**Typical development workflow**

1. Make your code changes
2. Run `pixi run lint` to ensure code formatting and style compliance
3. Run `pixi run tests` to verify your changes don't break existing functionality
4. If you've added new features, run `pixi run typing` to check type annotations
5. If you've modified documentation, run `pixi run docs` to build and verify the docs

```{tip}
You can run `pixi info` to see all available environments and `pixi task list` to see all available tasks across environments.
```

## For maintainers

Expand All @@ -52,5 +91,5 @@ The running of these commands is useful for local development and quick iteratio

When adding a dependency, make sure to modify the following files where relevant:

- `environment.yml` for core and development dependencies (important for the development environment, and CI)
- `pixi.toml` for core and development dependencies (important for the development environment, and CI)
- `pyproject.toml` for core dependencies (important for the pypi package, this should propagate through automatically to `recipe/meta.yml` in the conda-forge feedstock)
43 changes: 0 additions & 43 deletions environment.yml

This file was deleted.

Loading
Loading