Skip to content

Commit

Permalink
Add support for Python 3.12, relax numcodecs pin (#458)
Browse files Browse the repository at this point in the history
Contributes to rapidsai/build-planning#40

This PR adds support for Python 3.12.

## Notes for Reviewers

This is part of ongoing work to add Python 3.12 support across RAPIDS.
It temporarily introduces a build/test matrix including Python 3.12, from rapidsai/shared-workflows#213.

A follow-up PR will revert back to pointing at the `branch-24.10` branch of `shared-workflows` once all
RAPIDS repos have added Python 3.12 support.

### This will fail until all dependencies have been updates to Python 3.12

CI here is expected to fail until all of this project's upstream dependencies support Python 3.12.

This can be merged whenever all CI jobs are passing.

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Bradley Dice (https://github.com/bdice)

URL: #458
  • Loading branch information
jameslamb authored Sep 4, 2024
1 parent 0e9ebcb commit 3d48f2a
Show file tree
Hide file tree
Showing 13 changed files with 45 additions and 39 deletions.
16 changes: 8 additions & 8 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ concurrency:
jobs:
cpp-build:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-build.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-build.yaml@python-3.12
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -37,7 +37,7 @@ jobs:
python-build:
needs: [cpp-build]
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-python-build.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/conda-python-build.yaml@python-3.12
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -46,7 +46,7 @@ jobs:
upload-conda:
needs: [cpp-build, python-build]
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-upload-packages.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/conda-upload-packages.yaml@python-3.12
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -56,7 +56,7 @@ jobs:
if: github.ref_type == 'branch'
needs: python-build
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@python-3.12
with:
arch: "amd64"
branch: ${{ inputs.branch }}
Expand All @@ -68,7 +68,7 @@ jobs:
sha: ${{ inputs.sha }}
wheel-build-cpp:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@python-3.12
with:
matrix_filter: group_by([.ARCH, (.CUDA_VER|split(".")|map(tonumber)|.[0])]) | map(max_by(.PY_VER|split(".")|map(tonumber)))
build_type: ${{ inputs.build_type || 'branch' }}
Expand All @@ -79,7 +79,7 @@ jobs:
wheel-build-python:
needs: wheel-build-cpp
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@python-3.12
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -89,7 +89,7 @@ jobs:
wheel-publish-cpp:
needs: wheel-build-cpp
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-publish.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-publish.yaml@python-3.12
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -100,7 +100,7 @@ jobs:
wheel-publish-python:
needs: wheel-build-python
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-publish.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-publish.yaml@python-3.12
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand Down
22 changes: 11 additions & 11 deletions .github/workflows/pr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,38 +23,38 @@ jobs:
- wheel-python-build
- wheel-python-tests
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/pr-builder.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/pr-builder.yaml@python-3.12
checks:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/checks.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/checks.yaml@python-3.12
conda-cpp-build:
needs: checks
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-build.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-build.yaml@python-3.12
with:
build_type: pull-request
conda-cpp-tests:
needs: conda-cpp-build
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-tests.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-tests.yaml@python-3.12
with:
build_type: pull-request
conda-python-build:
needs: conda-cpp-build
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-python-build.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/conda-python-build.yaml@python-3.12
with:
build_type: pull-request
conda-python-tests:
needs: conda-python-build
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-python-tests.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/conda-python-tests.yaml@python-3.12
with:
build_type: pull-request
docs-build:
needs: conda-python-build
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@python-3.12
with:
build_type: pull-request
node_type: "gpu-v100-latest-1"
Expand All @@ -63,7 +63,7 @@ jobs:
run_script: "ci/build_docs.sh"
devcontainer:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/build-in-devcontainer.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/build-in-devcontainer.yaml@python-3.12
with:
arch: '["amd64"]'
cuda: '["12.5"]'
Expand All @@ -73,22 +73,22 @@ jobs:
sccache -s;
wheel-cpp-build:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@python-3.12
with:
matrix_filter: group_by([.ARCH, (.CUDA_VER|split(".")|map(tonumber)|.[0])]) | map(max_by(.PY_VER|split(".")|map(tonumber)))
build_type: pull-request
script: ci/build_wheel_cpp.sh
wheel-python-build:
needs: wheel-cpp-build
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@python-3.12
with:
build_type: pull-request
script: ci/build_wheel_python.sh
wheel-python-tests:
needs: wheel-python-build
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@python-3.12
with:
build_type: pull-request
script: ci/test_wheel.sh
4 changes: 2 additions & 2 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,15 @@ on:
jobs:
cpp-tests:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-tests.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-tests.yaml@python-3.12
with:
build_type: nightly
branch: ${{ inputs.branch }}
date: ${{ inputs.date }}
sha: ${{ inputs.sha }}
python-tests:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-python-tests.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/conda-python-tests.yaml@python-3.12
with:
build_type: nightly
branch: ${{ inputs.branch }}
Expand Down
4 changes: 2 additions & 2 deletions conda/environments/all_cuda-118_arch-aarch64.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ dependencies:
- doxygen=1.9.1
- gcc_linux-aarch64=11.*
- ninja
- numcodecs <0.12.0
- numcodecs !=0.12.0
- numpy>=1.23,<3.0a0
- numpydoc
- nvcc_linux-aarch64=11.8
Expand All @@ -27,7 +27,7 @@ dependencies:
- pre-commit
- pytest
- pytest-cov
- python>=3.10,<3.12
- python>=3.10,<3.13
- rapids-build-backend>=0.3.0,<0.4.0.dev0
- scikit-build-core>=0.10.0
- sphinx
Expand Down
4 changes: 2 additions & 2 deletions conda/environments/all_cuda-118_arch-x86_64.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ dependencies:
- libcufile-dev=1.4.0.31
- libcufile=1.4.0.31
- ninja
- numcodecs <0.12.0
- numcodecs !=0.12.0
- numpy>=1.23,<3.0a0
- numpydoc
- nvcc_linux-64=11.8
Expand All @@ -29,7 +29,7 @@ dependencies:
- pre-commit
- pytest
- pytest-cov
- python>=3.10,<3.12
- python>=3.10,<3.13
- rapids-build-backend>=0.3.0,<0.4.0.dev0
- scikit-build-core>=0.10.0
- sphinx
Expand Down
4 changes: 2 additions & 2 deletions conda/environments/all_cuda-125_arch-aarch64.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,15 +19,15 @@ dependencies:
- gcc_linux-aarch64=11.*
- libcufile-dev
- ninja
- numcodecs <0.12.0
- numcodecs !=0.12.0
- numpy>=1.23,<3.0a0
- numpydoc
- nvcomp==3.0.6
- packaging
- pre-commit
- pytest
- pytest-cov
- python>=3.10,<3.12
- python>=3.10,<3.13
- rapids-build-backend>=0.3.0,<0.4.0.dev0
- scikit-build-core>=0.10.0
- sphinx
Expand Down
4 changes: 2 additions & 2 deletions conda/environments/all_cuda-125_arch-x86_64.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,15 +19,15 @@ dependencies:
- gcc_linux-64=11.*
- libcufile-dev
- ninja
- numcodecs <0.12.0
- numcodecs !=0.12.0
- numpy>=1.23,<3.0a0
- numpydoc
- nvcomp==3.0.6
- packaging
- pre-commit
- pytest
- pytest-cov
- python>=3.10,<3.12
- python>=3.10,<3.13
- rapids-build-backend>=0.3.0,<0.4.0.dev0
- scikit-build-core>=0.10.0
- sphinx
Expand Down
2 changes: 1 addition & 1 deletion conda/recipes/kvikio/meta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ requirements:
- cupy >=12.0.0
- zarr
# See https://github.com/zarr-developers/numcodecs/pull/475
- numcodecs <0.12.0
- numcodecs !=0.12.0
- packaging
- {{ pin_compatible('cuda-version', max_pin='x', min_pin='x') }}
{% if cuda_major == "11" %}
Expand Down
4 changes: 2 additions & 2 deletions cpp/doxygen/main_page.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,9 +47,9 @@ Install the **nightly release** from the ``rapidsai-nightly`` channel with the f
# Install in existing environment
mamba install -c rapidsai-nightly -c conda-forge libkvikio
# Create new environment (CUDA 11.8)
mamba create -n libkvikio-env -c rapidsai-nightly -c conda-forge python=3.11 cuda-version=11.8 libkvikio
mamba create -n libkvikio-env -c rapidsai-nightly -c conda-forge python=3.12 cuda-version=11.8 libkvikio
# Create new environment (CUDA 12.5)
mamba create -n libkvikio-env -c rapidsai-nightly -c conda-forge python=3.11 cuda-version=12.5 libkvikio
mamba create -n libkvikio-env -c rapidsai-nightly -c conda-forge python=3.12 cuda-version=12.5 libkvikio
```

---
Expand Down
8 changes: 6 additions & 2 deletions dependencies.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -286,8 +286,12 @@ dependencies:
packages:
- python=3.11
- matrix:
py: "3.12"
packages:
- python>=3.10,<3.12
- python=3.12
- matrix:
packages:
- python>=3.10,<3.13
rapids_build_skbuild:
common:
- output_types: [conda, requirements, pyproject]
Expand All @@ -306,7 +310,7 @@ dependencies:
- numpy>=1.23,<3.0a0
- zarr
# See https://github.com/zarr-developers/numcodecs/pull/475
- numcodecs <0.12.0
- numcodecs !=0.12.0
- packaging
test_python:
common:
Expand Down
8 changes: 4 additions & 4 deletions docs/source/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@ Install the **stable release** from the ``rapidsai`` channel like:
# Install in existing environment
mamba install -c rapidsai -c conda-forge kvikio
# Create new environment (CUDA 11.8)
mamba create -n kvikio-env -c rapidsai -c conda-forge python=3.11 cuda-version=11.8 kvikio
mamba create -n kvikio-env -c rapidsai -c conda-forge python=3.12 cuda-version=11.8 kvikio
# Create new environment (CUDA 12.5)
mamba create -n kvikio-env -c rapidsai -c conda-forge python=3.11 cuda-version=12.5 kvikio
mamba create -n kvikio-env -c rapidsai -c conda-forge python=3.12 cuda-version=12.5 kvikio
Install the **nightly release** from the ``rapidsai-nightly`` channel like:

Expand All @@ -27,9 +27,9 @@ Install the **nightly release** from the ``rapidsai-nightly`` channel like:
# Install in existing environment
mamba install -c rapidsai-nightly -c conda-forge kvikio
# Create new environment (CUDA 11.8)
mamba create -n kvikio-env -c rapidsai-nightly -c conda-forge python=3.11 cuda-version=11.8 kvikio
mamba create -n kvikio-env -c rapidsai-nightly -c conda-forge python=3.12 cuda-version=11.8 kvikio
# Create new environment (CUDA 12.5)
mamba create -n kvikio-env -c rapidsai-nightly -c conda-forge python=3.11 cuda-version=12.5 kvikio
mamba create -n kvikio-env -c rapidsai-nightly -c conda-forge python=3.12 cuda-version=12.5 kvikio
.. note::
Expand Down
3 changes: 2 additions & 1 deletion python/kvikio/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ license = { text = "Apache 2.0" }
requires-python = ">=3.10"
dependencies = [
"cupy-cuda11x>=12.0.0",
"numcodecs <0.12.0",
"numcodecs !=0.12.0",
"numpy>=1.23,<3.0a0",
"packaging",
"zarr",
Expand All @@ -33,6 +33,7 @@ classifiers = [
"Programming Language :: Python",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
]

[project.optional-dependencies]
Expand Down
1 change: 1 addition & 0 deletions python/libkvikio/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ classifiers = [
"Programming Language :: Python",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
]

[project.urls]
Expand Down

0 comments on commit 3d48f2a

Please sign in to comment.