Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🚧 switch mamba installation #81

Draft
wants to merge 30 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
442ad85
:construction: switch mamba installation
enryH Oct 7, 2024
057647f
:bug: specify python version, move ls
enryH Oct 7, 2024
66dbb88
:construction: deactivate some workflow, run relatvie ls command
enryH Oct 7, 2024
9de1e37
try not to cache
enryH Oct 7, 2024
4e4965c
:construction: test using venv created by codespace with python 3.12
enryH Oct 7, 2024
7dc1168
try to use full snakemake installation
enryH Oct 7, 2024
be2699e
:construction: use miniconda for pypi installation test
enryH Oct 7, 2024
0f011a1
try miniconda again
enryH Oct 7, 2024
1b35c28
install build dependencies, fix ubuntu first
enryH Oct 7, 2024
e732f8c
:bug: try to put mamba below 2.0
enryH Oct 7, 2024
98b7279
test should be activate per default
enryH Oct 7, 2024
180141c
:construction: conda env not activated...
enryH Oct 8, 2024
9d2c4a4
:bug: pip does not install in environment
enryH Oct 8, 2024
2b90edd
:construction: experiment
enryH Oct 8, 2024
1b9f2ac
:bug: shell was not iniated
enryH Oct 8, 2024
86f57b3
:bug: test installing njab separately
enryH Oct 8, 2024
a97968d
:bug: order matters!
enryH Oct 8, 2024
49f4902
try again new order, add umap-learn explicitly
enryH Oct 8, 2024
4d202cb
:bug: do not re-install njab
enryH Oct 8, 2024
6a17b90
restrict scipy (trapz missing in lifelines)
enryH Oct 8, 2024
85b386c
:bug: exclude numpy 2.0 for now
enryH Oct 8, 2024
e6db81f
numpy try two
enryH Oct 8, 2024
481ac9f
swap numpy and njab, adapt other pkg to what it was before
enryH Oct 8, 2024
8326220
add back umap learn, relax constraints
enryH Oct 9, 2024
47988cf
:construction: in package single requirement
enryH Oct 9, 2024
d3fa813
:heavy_minus_sign: remove scipy dependency
enryH Oct 9, 2024
f0ea205
:arrow_up: remove support for python 3.8 (end-of-life)
enryH Oct 9, 2024
f4f06ab
:art: setuptools_scm uses tags to determine version, add tags
enryH Oct 9, 2024
b57415a
:bug: tags not fetched without entire history
enryH Oct 9, 2024
62e30ea
:art: clean-up workflow file
enryH Oct 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 42 additions & 22 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,38 +18,29 @@ jobs:
fail-fast: false
matrix:
os: ["ubuntu-latest",
"macos-13",
#"macos-13",
# "windows-latest" # rrcovNA cannot be build from source on windows-server
]
python-version: ["3.8", "3.9", "3.10"]
python-version: ["3.9", "3.10", "3.11", "3.12"]
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up Miniconda
# ! change action https://github.com/mamba-org/setup-micromamba
uses: conda-incubator/setup-miniconda@v3
with:
miniforge-variant: Mambaforge
# miniforge-version: latest
use-mamba: true
channel-priority: disabled
python-version: ${{ matrix.python-version }}
channel-priority: strict
environment-file: snakemake_env.yml
activate-environment: snakemake
auto-activate-base: true
# auto-update-conda: true
auto-update-conda: true
- name: inspect-conda-environment
run: |
conda info
conda list
conda env export --from-history --no-builds > environment.yml
conda env export --no-builds
conda env export --no-builds > environment_w_versions.yml
# - name: test-r-kernel-imports
# run: |
# Rscript -e "library(stringi)"
# Rscript -e "library(stringr)"
# Rscript -e "library(reshape2)"
- name: Dry-Run demo workflow (integration test)
run: |
cd project
Expand All @@ -59,6 +50,8 @@ jobs:
run: |
cd project
snakemake -p -c4 -k --configfile config/single_dev_dataset/example/config.yaml --use-conda
- name: Show current files in conda snakemake folder
run: ls -l project/.snakemake/conda/
- name: Run demo workflow again (in case of installation issues)
continue-on-error: true
run: |
Expand All @@ -75,35 +68,62 @@ jobs:
name: ${{ matrix.os }}-${{ matrix.python-version }}-example-workflow-results
path: |
project/runs/example/
environment.yml
environment_w_versions.yml
snakemake_env
project/.snakemake/conda/

run-unit-local-pip-installation:
runs-on: ${{ matrix.os }}
name: test-pip-installation
defaults:
run:
shell: bash -el {0}
strategy:
fail-fast: false
matrix:
os: ["ubuntu-latest", "macos-latest", "windows-latest"]
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
os: ["ubuntu-latest",]
# "macos-latest", "windows-latest"]
python-version: ["3.9", "3.10", "3.11", "3.12"]
steps:
- uses: actions/checkout@v4
with:
fetch-tags: true
fetch-depth: 0

- uses: actions/setup-python@v5
# - uses: actions/setup-python@v5
# with:
# python-version: ${{ matrix.python-version }}

- uses: conda-incubator/setup-miniconda@v3
with:
python-version: ${{ matrix.python-version }}
auto-update-conda: true
activate-environment: test
auto-activate-base: true

- name: check conda
run: |
conda info
conda list

- name: Install build dependencies explicitly
run: |
pip install setuptools_scm
conda list

- name: install pimms
run: python -m pip install .
run: pip install .

- name: install njab
run: pip install njab

- name: Install pytest
run: python -m pip install pytest pytest-cov
run: pip install pytest pytest-cov

- name: Run pytest
run: pytest .

- name: Install papermill
run: python -m pip install papermill ipykernel
run: pip install papermill ipykernel

- name: View papermill help message for notebooks (as scripts)
run: |
Expand Down Expand Up @@ -141,4 +161,4 @@ jobs:
- uses: pypa/gh-action-pypi-publish@release/v1
with:
user: __token__
password: ${{ secrets.PYPI_API_TOKEN }}
password: ${{ secrets.PYPI_API_TOKEN }}
4 changes: 2 additions & 2 deletions .github/workflows/ci_workflow.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ name: run workflow with conda envs
on:
push:
branches: [main, dev]
pull_request:
branches: [main, dev]
# pull_request:
# branches: [main, dev]
release:
types: [published]
schedule:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/test_pkg_on_colab.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ name: Test that tutorial runs on latest colab image
on:
# push:
# branches: [main]
pull_request:
branches: [main]
# pull_request:
# branches: [main]
schedule:
- cron: '0 2 3 * *'

Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/workflow_website.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name: Build workflow website on public Alzheimer dataset (for protein groups)
on:
pull_request:
branches: [main, dev]
# pull_request:
# branches: [main, dev]
release:
types: [published]
schedule:
Expand Down Expand Up @@ -73,4 +73,4 @@ jobs:
uses: peaceiris/actions-gh-pages@v4
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: project/runs/alzheimer_study/_build/
publish_dir: project/runs/alzheimer_study/_build/
4 changes: 2 additions & 2 deletions .readthedocs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ version: 2
build:
os: ubuntu-22.04
tools:
python: "3.8"
python: "3.10"
# You can also specify other tool versions:
# nodejs: "19"
# rust: "1.64"
Expand All @@ -32,4 +32,4 @@ python:
- method: pip
path: .
extra_requirements:
- docs
- docs
2 changes: 1 addition & 1 deletion environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ channels:
- plotly
# - defaults
dependencies:
- python>=3.8,<=3.12
- python>=3.9,<=3.12
- numpy
- pandas>=1
- scipy>=1.6
Expand Down
Loading
Loading