Skip to content

Commit

Permalink
Replace vcr-py for tests (#1198)
Browse files Browse the repository at this point in the history
* Adapt test.sh to use proxay.

* wip replace vcr-py with proxy.

* working on formatting and cleaning responses.

* Add type hint.

* Update uv lock.

* Change API requests in tests to use local webknossos.

* Experiment with failing tests.

* Remove download from pytest fixture.

* Update test-cassettes and fix some tests.

* Run linter.

* Remove time strings from tests.

* Adapt tests with zarr streaming.

* Working on tests that use annotations.

* Add debugging test mode that starts replay proxy.

* Add fixture for aiohttp use env variables.

* Set l4_sample public for running the remote_mags tests.

* Add cassette files for tests.

* Move access to remote MagViews out of fixture to capture requests in proxay.

* Run linter.

* Update cassette for annotation_upload_download_roundtrip.

* Update readme and contribution guideline.

* Update ci.

* Remove groups from CI.

* Update cassettes.

* Revert changes of ci and add timeout to test.sh.

* Update timeout in test.sh.

* Update test_duration file.

* Adapt test examples.

* Remove unused import.

* Skip flaky tests for now.

* Update tests to use l4_sample dataset.

* Update cassettes.

* Fix test for apply_merger_mode.

* Fix remote_datasets test.

* Fix tests for adding remote mag and layer.

* Run linter and typechecker.

* Fix test learned_segmenter.
  • Loading branch information
markbader authored Dec 3, 2024
1 parent 33d2ba6 commit 1eee692
Show file tree
Hide file tree
Showing 101 changed files with 86,177 additions and 101,293 deletions.
5 changes: 4 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,10 @@ jobs:
with:
# Install a specific version of uv.
version: "0.4.22"


- name: Install proxay
run: npm install -g proxay

- name: Set up Python ${{ matrix.python-version }}
run: uv python install ${{ matrix.python-version }}

Expand Down
13 changes: 11 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,16 +133,25 @@ Internal workflows for scalable minds:

The `webknossos` folder contains examples, which are not part of the package, but are tested via `tests/test_examples.py` and added to the documentation (see `docs/src/webknossos-py/examples`).

To run the `./test.sh` script it is necessary to install `proxay`. This is either done with [NPM](https://www.npmjs.com) or [yarn](https://yarnpkg.com/getting-started/install):
```bash
npm install --global proxay

# or if you're using yarn
yarn global add proxay
```

The tests also contain functionality for the WEBKNOSSOS client. There a two modes to run the tests:

1. `./test.sh --refresh-snapshots`, sending network requests to a WEBKNOSSOS instance:
This expects a local WEBKNOSSOS setup with specific test data, which is shipped with WEBKNOSSOS. If you're starting and running WEBKNOSSOS manually, please use port 9000 (the default) and run the `tools/postgres/dbtool.js prepare-test-db` script in the WEBKNOSSOS repository (⚠️ this overwrites your local WEBKNOSSOS database). Alternatively, a `docker compose` setup is started automatically for the tests, see `./test.sh` and `tests/docker-compose.yml` for details. The network requests & response are recorded as "cassettes" by [vcr.py](https://vcrpy.readthedocs.io), see next point:
2. `./test.sh` replays responses from previous network snapshots using [vcr.py](https://vcrpy.readthedocs.io) via [pytest-recording](https://github.com/kiwicom/pytest-recording). No additional network requests are allowed in this mode.
This expects a local WEBKNOSSOS setup with specific test data, which is shipped with WEBKNOSSOS. If you're starting and running WEBKNOSSOS manually, please use port 9000 (the default) and run the `tools/postgres/dbtool.js prepare-test-db` script in the WEBKNOSSOS repository (⚠️ this overwrites your local WEBKNOSSOS database). Alternatively, a `docker compose` setup is started automatically for the tests, see `./test.sh` and `tests/docker-compose.yml` for details. The network requests & response are recorded as "cassettes" by [proxay](https://github.com/airtasker/proxay), see next point:
2. `./test.sh` replays responses from previous network snapshots using [proxay](https://github.com/airtasker/proxay).

`./test.sh --store-durations` updates the durations for
[`pytest-split`](https://jerry-git.github.io/pytest-split),
which is used in the CI to split the tests for different runners.


#### `cluster_tools` package

For testing the `slurm` setup a `docker compose` setup is available. Please see the [respective Readme](https://github.com/scalableminds/webknossos-libs/blob/master/cluster_tools/README.md) for details.
Expand Down
10 changes: 4 additions & 6 deletions cluster_tools/dockered-slurm/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
version: "2.2"

services:
mysql:
image: mysql:5.7
Expand All @@ -16,7 +14,7 @@ services:

slurmdbd:
image: scalableminds/slurm-docker-cluster:master__11274637426
command: ["slurmdbd"]
command: [ "slurmdbd" ]
container_name: slurmdbd
hostname: slurmdbd
volumes:
Expand All @@ -32,7 +30,7 @@ services:

slurmctld:
image: scalableminds/slurm-docker-cluster:master__11274637426
command: ["slurmctld"]
command: [ "slurmctld" ]
container_name: slurmctld
environment:
USER: "root"
Expand All @@ -53,7 +51,7 @@ services:

c1:
image: scalableminds/slurm-docker-cluster:master__11274637426
command: ["slurmd"]
command: [ "slurmd" ]
hostname: c1
container_name: c1
volumes:
Expand All @@ -71,7 +69,7 @@ services:

c2:
image: scalableminds/slurm-docker-cluster:master__11274637426
command: ["slurmd"]
command: [ "slurmd" ]
hostname: c2
container_name: c2
volumes:
Expand Down
2 changes: 0 additions & 2 deletions docs/uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 1eee692

Please sign in to comment.