wanda python-agnostic ray-cpp wheel (py3-none tag)(x86_64) #59969
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Author
Contributor
Author
|
This was referenced Jan 8, 2026
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces Wanda-based builds for Ray C++ wheels on x86_64, adding new CI configurations and Dockerfiles. The changes are well-structured, but there are opportunities to improve maintainability and ensure correctness across different Python versions. My review includes suggestions to refactor the Buildkite configuration to reduce duplication and to address a potential issue with hardcoded Python dependency files in the Wanda configuration.
Comment on lines
78
to
126
| - name: ray-cpp-core-build | ||
| label: "wanda: cpp core py{{matrix}} (x86_64)" | ||
| wanda: ci/docker/ray-cpp-core.wanda.yaml | ||
| matrix: | ||
| - "3.10" | ||
| - "3.11" | ||
| - "3.12" | ||
| env: | ||
| PYTHON_VERSION: "{{matrix}}" | ||
| ARCH_SUFFIX: "" | ||
| HOSTTYPE: "x86_64" | ||
| MANYLINUX_VERSION: "260103.868e54c" | ||
| tags: | ||
| - release_wheels | ||
| - oss | ||
|
|
||
| - name: ray-cpp-wheel-build | ||
| label: "wanda: cpp wheel py{{matrix}} (x86_64)" | ||
| wanda: ci/docker/ray-cpp-wheel.wanda.yaml | ||
| matrix: | ||
| - "3.10" | ||
| - "3.11" | ||
| - "3.12" | ||
| env: | ||
| PYTHON_VERSION: "{{matrix}}" | ||
| ARCH_SUFFIX: "" | ||
| HOSTTYPE: "x86_64" | ||
| MANYLINUX_VERSION: "260103.868e54c" | ||
| tags: | ||
| - release_wheels | ||
| - linux_wheels | ||
| - oss | ||
| depends_on: | ||
| - ray-core-build | ||
| - ray-cpp-core-build | ||
| - ray-java-build | ||
| - ray-dashboard-build | ||
|
|
||
| # Upload cpp wheels to S3 | ||
| # Gating: only on releases/* OR (master AND nightly) | ||
| - label: ":s3: upload: cpp wheel py{{matrix}} (x86_64)" | ||
| key: linux_cpp_wheels_upload | ||
| instance_type: small | ||
| commands: | ||
| - ./ci/build/extract_wanda_wheels.sh ray-cpp-wheel-py{{matrix}} --upload | ||
| matrix: | ||
| - "3.10" | ||
| - "3.11" | ||
| - "3.12" | ||
| depends_on: | ||
| - ray-cpp-wheel-build | ||
| tags: | ||
| - release_wheels | ||
| - skip-on-premerge | ||
| - oss |
Contributor
There was a problem hiding this comment.
There's significant duplication in the newly added build steps. The matrix for Python versions and the env configuration are repeated. To improve maintainability and reduce redundancy, consider using YAML anchors and aliases.
For example, you could define:
x-python-versions: &python-versions
- "3.10"
- "3.11"
- "3.12"
x-cpp-build-env: &cpp-build-env
PYTHON_VERSION: "{{matrix}}"
ARCH_SUFFIX: ""
HOSTTYPE: "x86_64"
MANYLINUX_VERSION: "260103.868e54c"And then reference them in your build steps:
- name: ray-cpp-core-build
...
matrix: *python-versions
env: *cpp-build-env
- name: ray-cpp-wheel-build
...
matrix: *python-versions
env: *cpp-build-env
- label: ":s3: upload: cpp wheel py{{matrix}} (x86_64)"
...
matrix: *python-versionsThis would make the configuration cleaner and easier to update in the future.
| - java/dependencies.bzl | ||
| - release/BUILD.bazel | ||
| - release/requirements.txt | ||
| - release/requirements_py310.txt |
Contributor
There was a problem hiding this comment.
This configuration hardcodes release/requirements_py310.txt as a source file. However, the build matrix in .buildkite/build.rayci.yml includes Python 3.10, 3.11, and 3.12. Using Python 3.10-specific requirements for 3.11 and 3.12 builds might lead to dependency issues if there are version-specific packages.
If different Python versions require different dependencies, consider making this path dynamic based on the Python version. For example, if the build system supports it, you could use a variable like release/requirements_py$PYTHON_VERSION.txt and ensure corresponding files exist for each Python version.
andrew-anyscale
force-pushed
the
andrew/revup/master/ray-cpp-wheel
branch
from
bf5b228 to
ed903b7
Compare
andrew-anyscale
force-pushed
the
andrew/revup/master/ray-wheel
branch
2 times, most recently
from
8a4bdf5 to
a0b03e9
Compare
andrew-anyscale
force-pushed
the
andrew/revup/master/ray-cpp-wheel
branch
2 times, most recently
from
21500cc to
cf7cab6
Compare
andrew-anyscale
force-pushed
the
andrew/revup/master/ray-wheel
branch
from
a0b03e9 to
32347fa
Compare
andrew-anyscale
added
go
andrew-anyscale
force-pushed
the
andrew/revup/master/ray-cpp-wheel
branch
2 times, most recently
from
8d1057a to
d8901f7
Compare
andrew-anyscale
force-pushed
the
andrew/revup/master/ray-wheel
branch
from
32347fa to
9074421
Compare
Contributor
Author
|
Verifying cpp wheel found no diffs between the old and this method: Remote URL : f6c2b5f Local wheel: 029272617770.dkr.ecr.us-west-2.amazonaws.com/rayproject/citemp@sha256:922a6d572dc2508d00cf247487facb2cbe918d3c924bc00ff53fd99726af5eb3 |
andrew-anyscale
force-pushed
the
andrew/revup/master/ray-cpp-wheel
branch
from
d8901f7 to
9e43825
Compare
andrew-anyscale
force-pushed
the
andrew/revup/master/ray-wheel
branch
from
9074421 to
1b2c341
Compare
andrew-anyscale
force-pushed
the
andrew/revup/master/ray-cpp-wheel
branch
from
9e43825 to
6801f72
Compare
andrew-anyscale
force-pushed
the
andrew/revup/master/ray-wheel
branch
from
1b2c341 to
05785cb
Compare
The ray-cpp wheel contains only C++ headers, libraries, and executables with no Python-specific code. Previously we built 4 identical wheels (one per Python version: cp310, cp311, cp312, cp313), wasting CI time and storage. This change produces a single wheel tagged py3-none-manylinux2014_* that works with any Python 3.x version. Changes: - Add ray-cpp-core.wanda.yaml and Dockerfile for cpp core - Add ray-cpp-wheel.wanda.yaml for cpp wheel builds - Add ci/build/build-ray-cpp-wheel.sh for Python-agnostic wheel builds - Add RayCppBdistWheel class to setup.py that forces py3-none tags (necessary because BinaryDistribution.has_ext_modules() causes bdist_wheel to use interpreter-specific ABI tags by default) - Update ray-cpp-wheel.wanda.yaml to build single wheel per architecture - Update .buildkite/build.rayci.yml to remove Python version matrix for cpp wheel build/upload steps Topic: ray-cpp-wheel Relative: ray-wheel Signed-off-by: andrew <[email protected]>
andrew-anyscale
force-pushed
the
andrew/revup/master/ray-cpp-wheel
branch
from
6801f72 to
9834ea3
Compare
andrew-anyscale
force-pushed
the
andrew/revup/master/ray-wheel
branch
from
05785cb to
37410f9
Compare
andrew-anyscale
force-pushed
the
andrew/revup/master/ray-cpp-wheel
branch
3 times, most recently
from
2a36172 to
b81d4cb
Compare
andrew-anyscale
requested review from
aslonnie,
edoakes and
richardliaw
as code owners
andrew-anyscale
changed the title
Contributor
Author
|
Converting to draft. Apologies for the new Codeowner pings |
Contributor
Author
|
Validation of commit f6c2b5f for this diff for this looked great!
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The ray-cpp wheel contains only C++ headers, libraries, and executables
with no Python-specific code. Previously we built 4 identical wheels
(one per Python version: cp310, cp311, cp312, cp313), wasting CI time
and storage.
This change produces a single wheel tagged py3-none-manylinux2014_*
that works with any Python 3.x version.
Changes:
(necessary because BinaryDistribution.has_ext_modules() causes
bdist_wheel to use interpreter-specific ABI tags by default)
for cpp wheel build/upload steps
Topic: ray-cpp-wheel
Relative: ray-wheel
Signed-off-by: andrew [email protected]