Skip to content

Commit

Permalink
Update dev env instructions (#2990)
Browse files Browse the repository at this point in the history
Close #2793 

- Add pixi, first mentioned in
#2707 (comment)
- Use tabs in docs to help to point out only one option is needed
- Default recommendation is conda, add `dev.yml` with dev dependencies
  • Loading branch information
luizirber authored Jul 28, 2024
1 parent d516811 commit 272a97f
Show file tree
Hide file tree
Showing 9 changed files with 12,476 additions and 32 deletions.
7 changes: 6 additions & 1 deletion .envrc
Original file line number Diff line number Diff line change
@@ -1 +1,6 @@
use flake
#use flake

PATH=~/.pixi/bin:$PATH

watch_file pixi.lock
eval "$(pixi shell-hook)"
40 changes: 29 additions & 11 deletions .github/workflows/dev_envs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@ on:
pull_request:
push:
branches: [latest]
concurrency:
group: ${{ github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/latest' }}
jobs:
nix:
runs-on: ${{ matrix.os }}
Expand All @@ -23,8 +26,32 @@ jobs:

- run: nix develop --command bash -c "tox -e py310"

pixi:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, macos-14]
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- name: set up pixi
uses: prefix-dev/[email protected]
with:
pixi-version: v0.26.1
cache: true
frozen: true

- name: run tests for 3.10
shell: pixi run bash {0}
run: tox -e py310

mamba:
runs-on: ubuntu-latest
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, macos-14]
steps:
- uses: actions/checkout@v4
with:
Expand All @@ -42,20 +69,11 @@ jobs:
- name: setup conda
uses: conda-incubator/setup-miniconda@a4260408e20b96e80095f42ff7f1a15b27dd94ca
with:
auto-update-conda: true
python-version: "3.10"
channels: conda-forge,bioconda
miniforge-variant: Mambaforge
miniforge-version: latest
use-mamba: true
mamba-version: "*"
environment-file: dev.yml
activate-environment: sourmash_dev
auto-activate-base: false
use-only-tar-bz2: true

- name: install dependencies
shell: bash -l {0}
run: mamba install 'tox>=3.27,<4' tox-conda rust git compilers pandoc libstdcxx-ng

- name: run tests for 3.10
shell: bash -l {0}
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,6 @@ wasm-pack.log
# nix
.direnv/
result

# pixi
.pixi/
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ dist: FORCE
$(PYTHON) -m build --sdist

test: .PHONY
tox -e py39
tox -e py310
cargo test

doc: .PHONY
Expand Down
12 changes: 12 additions & 0 deletions dev.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
name: sourmash_dev
channels:
- conda-forge
dependencies:
- clangdev >=16.0.6,<16.1
- cxx-compiler >=1.7.0,<1.8
- git 2.41.0.*
- libclang >=16.0.6,<16.1
- pandoc 3.1.3.*
- python 3.10.*
- rust >=1.80.0,<1.81
- tox >=4.11.0,<5
5 changes: 5 additions & 0 deletions doc/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@
"nbsphinx",
"IPython.sphinxext.ipython_console_highlighting",
"myst_parser",
"sphinx_design",
]

# Add any paths that contain templates here, relative to this directory.
Expand Down Expand Up @@ -314,3 +315,7 @@

autodoc_mock_imports = ["sourmash.minhash"]
myst_heading_anchors = 3
myst_enable_extensions = [
"html_admonition",
"colon_fence",
]
60 changes: 43 additions & 17 deletions doc/developer.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,33 +12,52 @@ git clone https://github.com/sourmash-bio/sourmash.git
```
sourmash runs under Python 3.10 and later.

We recommend using `conda` or `Nix` for setting up an environment for developing
We recommend using `conda` for setting up an environment for developing
new features, running tests and code quality checks.
Here are some suggestions on how to set them up (note: you only need one =])
Here are some suggestions on how to set them up

### Using mamba (conda alternative)
:::{note}
You only need one of these, no need to install them all!
:::

::::{tab-set}

:::{tab-item} conda

Follow the [installation instructions](https://github.com/conda-forge/miniforge#install) for
installing `mambaforge` (a conda distribution that uses
[`mamba`](https://github.com/TheSnakePit/mamba)
and the [`conda-forge`](https://conda-forge.org/) channel by default).
installing `Miniforge3` (a conda distribution that uses
the [`conda-forge`](https://conda-forge.org/) channel by default).

Once `mamba` is installed, run
Once `conda` is installed, run
```
mamba create -n sourmash_dev 'tox>=3.27,<4' tox-conda rust git compilers pandoc libstdcxx-ng
conda env create -n sourmash_dev --file dev.yml
```
to create an environment called `sourmash_dev` containing the programs needed
for development.

To activate the new environment, run
Once the environment is created, you can activate it for development with
```
conda activate sourmash_dev
```

:::

:::{tab-item} pixi

Follow the [installation instructions](https://github.com/prefix-dev/pixi) for
installing `pixi`.

To activate the new environment, run
```
pixi shell
```
and proceed to the ["Running tests and checks"](#running-tests-and-checks) section.

### Using Nix
:::

:::{tab-item} nix

Follow the [installation instructions](https://nixos.org/manual/nix/stable/#chap-installation)
Follow the [installation instructions](https://zero-to-nix.com/start/install)
for setting up Nix in your system (Linux or macOS).

Once Nix is installed, run
Expand All @@ -47,18 +66,22 @@ nix develop
```
to start an environment ready for [running tests and checks](#running-tests-and-checks).

### General instructions
:::

:::{tab-item} General instructions

As long as you have `tox` and a Rust compiler available,
you can skip `mamba` or `Nix`.
you can skip `conda`, `pixi`, or `nix`.

For Rust, we suggest using `rustup` to install the Rust environment:
```
curl https://sh.rustup.rs -sSf | sh
```
And for `tox` you can run

And for `tox` you can use [pipx](https://pipx.pypa.io/latest/installation/) for
installing it without interfering with other Python environments:
```
python -m pip install tox
pipx install tox
```

We suggest working on sourmash in a virtualenv; e.g. from within the
Expand All @@ -74,12 +97,15 @@ pip install -r requirements.txt
```
(but they are already installed in the virtualenv created with `tox -e dev`).

## Updating your developer environment

To update rust to the latest version, use `rustup update`.

To update your Python dependencies to the latest required for sourmash, you can run `pip install -r requirements.txt`.

:::

::::


## Running tests and checks

We use [`tox`](https://tox.readthedocs.io) for managing dependencies and
Expand Down
Loading

0 comments on commit 272a97f

Please sign in to comment.