From edeccfe12707f5f4adba7d7cc2a3a304c0390a08 Mon Sep 17 00:00:00 2001 From: shoumikhin Date: Mon, 10 Aug 2026 19:56:25 -0700 Subject: [PATCH] Build and publish CUDA wheels ## The problem The wheel can carry the CUDA delegate, but nothing builds one: there is no CUDA row in any workflow, so a GPU user still has to build from source. ## The change Add the workflows that build and publish CUDA wheels for Linux x86_64 and aarch64, and a smoke test that checks each wheel from the artifact itself. The build machines for these rows have no GPU, so the smoke test does not execute a model; it verifies the CUDA libraries are present, that the declared runtime matches the wheel's CUDA version, that nothing resolves through the build machine's toolkit, and that the shipped device code covers every GPU architecture the row claims. ``` executorch-1.5.0-cp312-cp312-manylinux_2_28_x86_64.whl +cu130 ``` A release publishes CUDA 12.6, 13.0 and 13.2, for Python 3.10 through 3.13. A pull request builds a single row instead of all twelve, because a full matrix costs hours for little extra signal. Which GPU architectures each row compiles for is chosen per row rather than detected on the builder. Detecting it would produce a wheel carrying device code for whatever machine happened to build it, which installs fine and then fails at the first GPU call. The aarch64 CUDA 12.6 row also compiles for compute capability 8.7, which is an embedded module. Every other row lists only the architectures the published PyTorch build for that train covers, and by that rule 8.7 would be left out, because the generic aarch64 build of this train carries 8.0 and 9.0 only. It is included because this is the only row whose CUDA major version matches what that module's software release ships, and because this wheel declares no PyTorch dependency: a user there supplies the build that carries their architecture. Leaving 8.7 out does not protect them from a bad pairing, it only removes the device code they need. Without it, a model reaching one of the shipped optional operators, quantized matrix multiply, sort or random number generation, fails at the first launch on that device. Two guards keep a release honest: - if the shared matrix generator stops offering a combination this policy advertises, the step fails instead of quietly publishing fewer wheels. A missing job is otherwise a green check for a wheel that was never built. - if a row reaches the architecture list with no CUDA version, the build refuses rather than falling back to the builder's GPU. A TORCH_CUDA_ARCH_LIST that holds only named GPU families PyTorch accepts, such as "Hopper", now fails to configure instead of quietly leaving CMAKE_CUDA_ARCHITECTURES unset and taking the compiler default. The three named forms CMake itself understands, "native", "all" and "all-major", pass straight through, since CMake reduces them to the architectures the toolkit knows about. Windows CUDA is deliberately absent. The separate shared libraries this wheel exists to ship are Linux only today, so a Windows CUDA wheel would carry a delegate a C++ application still could not link. ## Test plan - built the full release matrix, twelve wheels, and confirmed each one's contents match the row it claims: the CUDA libraries present, the CUDA runtime declared, and device code for every GPU architecture the row advertises. - ran a GPU model end to end from a CI-built wheel on three NVIDIA GPUs covering three device architectures, with output identical to eager PyTorch on each (largest absolute difference 0), and inspected the wheel for a fourth device it cannot execute on. - ran the matrix filter over generated inputs, including incomplete and malformed ones, and confirmed it refuses rather than publishing a partial release: a missing CUDA version, a missing python, or a python present on rows this policy does not build are each reported by name. - confirmed a CPU row still produces a CPU wheel on a builder that happens to have a CUDA toolkit installed. - the newest architecture also ships in its portable form, so a GPU newer than any in the row can still run by having the driver compile it at load time. Checked with `cuobjdump --list-ptx`, since `--list-elf` prints identical output whether or not the portable form is present. - every library that carries GPU device code covers the whole row on its own. - the declared CUDA packages are compared against the expected set in BOTH directions. A one-way comparison accepted a wheel that omitted required packages, and a name-suffix comparison accepted cross-train names because for CUDA 13 the suffix is empty. - the python axis is an allowlist, matching the CUDA axis. Testing only the disabled list let any python not on it through: a 3.9 row was emitted successfully. - `install_utils.py` is in both CUDA workflows' path filters. It owns the supported CUDA train list and the toolkit detection, so a change there previously ran no CUDA wheel job. - requesting the JetPack rows fails with its own reason instead of the generic empty-matrix message, since both of its lists are deliberately empty and no workflow asks for them. - torchao keeps its CUDA channel where that channel exists. Falling back to the plain nightly index was needed only on aarch64, where the CUDA channel publishes nothing, and doing it everywhere changed which torchao an x86_64 install resolves. - the CUDA smoke test now asserts the QnnBackend and OpenvinoBackend registrations that a CPU Linux row asserts. The CUDA build enables OpenVINO on every Linux architecture and downloads the QNN SDK on x86_64, so a CUDA wheel carries both backends; a previous premise that "a CUDA row is not built with them" was false, and dropping the checks meant those two backends were unverified on every CUDA wheel. Known gap: no automated job runs a CUDA model on real hardware before publication. Running a model on real hardware is a separate release-time step that a person owns today, not an automated job wired into these workflows. ghstack-source-id: aced89e101e29ef9a19d7fad519fdff78ccd01ff ghstack-comment-id: 5220374521 Pull-Request: https://github.com/pytorch/executorch/pull/21668 --- .ci/scripts/wheel/cuda_arch_list.sh | 131 +++++++ .ci/scripts/wheel/envvar_cuda_linux.sh | 42 +++ .ci/scripts/wheel/test_cuda_linux.py | 340 ++++++++++++++++++ .ci/scripts/wheel/test_shared_libraries.py | 71 +++- .github/scripts/filter_cuda_matrix.py | 238 ++++++++++++ .../build-wheels-cuda-aarch64-linux.yml | 100 ++++++ .github/workflows/build-wheels-cuda-linux.yml | 96 +++++ backends/cuda/CMakeLists.txt | 80 +++++ install_requirements.py | 11 +- 9 files changed, 1105 insertions(+), 4 deletions(-) create mode 100644 .ci/scripts/wheel/cuda_arch_list.sh create mode 100644 .ci/scripts/wheel/envvar_cuda_linux.sh create mode 100644 .ci/scripts/wheel/test_cuda_linux.py create mode 100644 .github/scripts/filter_cuda_matrix.py create mode 100644 .github/workflows/build-wheels-cuda-aarch64-linux.yml create mode 100644 .github/workflows/build-wheels-cuda-linux.yml diff --git a/.ci/scripts/wheel/cuda_arch_list.sh b/.ci/scripts/wheel/cuda_arch_list.sh new file mode 100644 index 00000000000..17aefaf98f6 --- /dev/null +++ b/.ci/scripts/wheel/cuda_arch_list.sh @@ -0,0 +1,131 @@ +#!/usr/bin/env bash +# Copyright (c) Meta Platforms, Inc. and affiliates. +# All rights reserved. +# +# This source code is licensed under the BSD-style license found in the +# LICENSE file in the root directory of this source tree. + +# GPU architectures to compile device code for, chosen per release row rather than detected from +# the build machine. +# +# Without this the build compiles for whichever GPU the builder happens to have. The wheel then +# installs on every machine the row claims and fails when a model runs on a different generation, +# with an error that looks like a model problem rather than a packaging one. Detection is the right +# default for a local build and the wrong one for a published artifact. +# +# The value is published as TORCH_CUDA_ARCH_LIST rather than CMAKE_CUDA_ARCHITECTURES, because +# PyTorch's CMake rejects the latter and overrides it, so setting only that reduces the build to a +# single detected architecture. + +# The architectures each row serves. Two rules decide the list, and they pull in opposite directions. +# +# The upper end follows the published PyTorch build for that train, read from its own library rather than +# chosen by reasoning about which GPUs matter. A delegate is only useful where torch already runs, and an +# architecture torch supports but this wheel omits produces a wheel that installs and then fails at the +# first kernel launch. Two omissions found that way were the GPU on the runner that tests these wheels, +# and a common desktop card. +# +# The lower end does NOT follow torch. It stops at 8.0 even though torch reaches further down, because one +# source here compiles an integer matrix-multiply path only at 8.0 and above. Below that a user gets a +# delegate that loads, runs most models, and fails on one needing that operator, which is worse than a row +# that never claimed the device. So these lists are narrower than torch at the bottom on purpose. +_cuda_arch_x86_64_cu130="8.0 8.6 8.9 9.0 10.0 12.0" +_cuda_arch_x86_64_cu132="${_cuda_arch_x86_64_cu130}" + +# The architectures the published aarch64 PyTorch CUDA build covers, read from its own library on an ARM +# machine, for the same reason as the x86_64 rows above. Includes the ARM module whose train matches. +_cuda_arch_aarch64_cu130="8.0 9.0 10.0 11.0 12.0" +_cuda_arch_aarch64_cu132="${_cuda_arch_aarch64_cu130}" + +# The older CUDA train. +# +# The two architectures do not carry identical lists, because each covers what the published PyTorch +# build for that architecture covers, and those differ. Matching them to each other instead would mean +# advertising a GPU on one architecture that PyTorch cannot serve there. +# +# The smaller embedded modules are deliberately absent, with one exception. An embedded-only +# architecture in a generic wheel would advertise a device the row cannot otherwise serve, since +# those devices also need the CUDA, TensorRT and PyTorch pinned by their own software release +# rather than the ones a generic wheel resolves. +# +# 8.7 is that exception. This is the only row whose CUDA major matches what that module's software +# release ships, and the wheel declares no PyTorch, so the user supplies the build that carries +# their architecture. Omitting it does not protect them from a bad pairing, it only removes the +# device code they need. +# +# The floor is 8.0 rather than the oldest architecture PyTorch still carries. One of these sources compiles +# an integer matrix-multiply path only at 8.0 and newer, so an older architecture would get a delegate that +# loads, runs most models, and fails on one that needs that operator. Claiming hardware the delegate only +# partly serves is the same problem the embedded modules have, so the row leaves it out for the same reason. +_cuda_arch_x86_64_cu126="8.0 8.6 8.9 9.0" +_cuda_arch_aarch64_cu126="8.0 8.7 9.0" + +# A CUDA train with no architecture list would leave the build detecting the builder's GPU, which is +# the failure this file exists to prevent. Adding a train to the release matrix without adding its +# architectures should fail loudly rather than silently produce a single-GPU wheel. +_executorch_unknown_train() { + echo "cuda_arch_list.sh: no GPU architecture list for CUDA train '$1' on $(uname -m)." >&2 + echo "Add one before building this row, or the wheel ships device code for one GPU only." >&2 + return 64 +} + +# The architectures for the current row, space separated in the dotted form PyTorch expects. +executorch_cuda_arch_list() { + local machine + machine="$(uname -m)" + # The wheel build exports the row's CUDA train as CU_VERSION. DESIRED_CUDA is the name of the + # matrix field rather than of the variable, so reading only that leaves every row falling back to + # detecting the builder's GPU. + local train="${CU_VERSION:-${DESIRED_CUDA:-}}" + # A CPU row names no CUDA train and needs no architectures, so it is not an error. + # + # A CUDA row always names one, so an empty value there means the row lost it. Treating that as a CPU + # row let the build fall back to detecting the builder's GPU, which produces a wheel carrying device + # code for whatever machine happened to build it while every check still reports green. + case "${train}" in + "" | cpu | CPU | none | NONE) + if [ "${EXECUTORCH_BUILD_CUDA:-}" = "1" ]; then + echo "this is a CUDA build but the row's CUDA version is '${train}', which names no CUDA" >&2 + echo "train. Refusing to detect the builder GPU instead." >&2 + return 65 + fi + return 0 + ;; + esac + # The value arrives as cu130, while some callers pass 13.0 instead. + train="${train#cu}" + train="${train//./}" + + case "${machine}" in + aarch64 | arm64) + case "${train}" in + 126) printf '%s' "${_cuda_arch_aarch64_cu126}" ;; + 130) printf '%s' "${_cuda_arch_aarch64_cu130}" ;; + 132) printf '%s' "${_cuda_arch_aarch64_cu132}" ;; + *) _executorch_unknown_train "${train}" ;; + esac + ;; + x86_64) + case "${train}" in + 126) printf '%s' "${_cuda_arch_x86_64_cu126}" ;; + 130) printf '%s' "${_cuda_arch_x86_64_cu130}" ;; + 132) printf '%s' "${_cuda_arch_x86_64_cu132}" ;; + *) _executorch_unknown_train "${train}" ;; + esac + ;; + *) _executorch_unknown_train "${train}" ;; + esac +} + +# The same list with a portable form appended for the newest architecture, so a GPU newer than any +# in the row can still run the wheel by compiling that form at load time. Without it a newer GPU +# gets no usable code at all. +executorch_cuda_arch_list_with_ptx() { + local dotted top + # Propagate a failed lookup rather than reporting an empty list, since a caller cannot tell an + # unknown row from a CPU row and the unknown one must not pass silently. + dotted="$(executorch_cuda_arch_list)" || return $? + [ -n "${dotted}" ] || return 0 + top="${dotted##* }" + printf '%s %s+PTX' "${dotted}" "${top}" +} diff --git a/.ci/scripts/wheel/envvar_cuda_linux.sh b/.ci/scripts/wheel/envvar_cuda_linux.sh new file mode 100644 index 00000000000..d66ae3f2d22 --- /dev/null +++ b/.ci/scripts/wheel/envvar_cuda_linux.sh @@ -0,0 +1,42 @@ +# Copyright (c) Meta Platforms, Inc. and affiliates. +# All rights reserved. +# +# This source code is licensed under the BSD-style license found in the +# LICENSE file in the root directory of this source tree. + +# This file is sourced into the environment before building a pip wheel. It +# should typically only contain shell variable assignments. Be sure to export +# any variables so that subprocesses will see them. + +source "${GITHUB_WORKSPACE}/${REPOSITORY}/.ci/scripts/wheel/envvar_base.sh" + +# Ask for the CUDA delegate explicitly rather than letting the build detect a toolkit. A detected +# build is fine locally, but a release row states what it is producing, and a row that silently +# produced a CPU wheel because the toolkit was missing would publish under a CUDA name. +export EXECUTORCH_BUILD_CUDA=1 +export CMAKE_ARGS="${CMAKE_ARGS} -DEXECUTORCH_BUILD_CUDA=ON" + +# Fail the build if CUDA is not actually present. Without this the packaging step would look for +# CUDA libraries that were never built and report a confusing missing-file error several minutes +# after the real problem. +if [ ! -x "${CUDA_HOME:-/usr/local/cuda}/bin/nvcc" ]; then + echo "EXECUTORCH_BUILD_CUDA is set but no nvcc was found. This row cannot build a CUDA wheel." >&2 + exit 1 +fi + +# Compile device code for the GPUs this release row claims, rather than for whichever GPU the +# builder happens to have. A wheel built by detection alone installs on every machine the row covers +# and then fails when a model runs on a different generation. +source "${GITHUB_WORKSPACE}/${REPOSITORY}/.ci/scripts/wheel/cuda_arch_list.sh" +# The status is checked rather than only the output. An unrecognised row makes the lookup fail, and +# this file is sourced rather than run under a failing-command shell, so ignoring the status would +# leave the variable unset and let the build fall back to detecting the builder's own GPU. That is +# exactly the outcome this is meant to prevent, and it would ship quietly. +if ! _executorch_cuda_arch="$(executorch_cuda_arch_list_with_ptx)"; then + echo "could not resolve GPU architectures for CU_VERSION=${CU_VERSION:-unset}" >&2 + exit 1 +fi +if [ -n "${_executorch_cuda_arch}" ]; then + export TORCH_CUDA_ARCH_LIST="${_executorch_cuda_arch}" + echo "building device code for: ${TORCH_CUDA_ARCH_LIST}" +fi diff --git a/.ci/scripts/wheel/test_cuda_linux.py b/.ci/scripts/wheel/test_cuda_linux.py new file mode 100644 index 00000000000..47981787e38 --- /dev/null +++ b/.ci/scripts/wheel/test_cuda_linux.py @@ -0,0 +1,340 @@ +#!/usr/bin/env python +# Copyright (c) Meta Platforms, Inc. and affiliates. +# All rights reserved. +# +# This source code is licensed under the BSD-style license found in the +# LICENSE file in the root directory of this source tree. + +"""Smoke test for a CUDA wheel row. + +Runs the checks a GPU wheel needs, then the packaging, backend, and C++ SDK checks a CPU wheel +gets. The extra CUDA checks exist because a GPU wheel can install cleanly, import cleanly, and +still be unusable: + + the CUDA libraries can be absent while the wheel is still named as a CUDA build + the runtime dependency can be undeclared, so a user has nothing to resolve it from + the loader path can point at the build machine's toolkit, which no user has + the device code can cover no GPU the row claims, which only appears when a model runs + +The build machines for these rows have no GPU, so this does not execute a model. What runs a +model on real hardware before a publication is a separate release-time step that a person owns +today, not an automated job wired into these workflows. +""" + +import os +import pathlib +import platform +import subprocess +import tempfile +from pathlib import Path + +import test_base +import test_cpp_sdk +import test_shared_libraries +from examples.models import Backend, Model + + +def _package_dir() -> Path: + import executorch + + return Path(executorch.__path__[0]) + + +def test_cuda_libraries_are_shipped() -> None: + """The row is named for CUDA, so the CUDA libraries have to be in it.""" + lib_dir = _package_dir() / "lib" + shipped = {path.name for path in lib_dir.iterdir()} if lib_dir.is_dir() else set() + expected = { + "libexecutorch_backend_cuda.so", + "libexecutorch_extension_cuda.so", + } + missing = sorted(expected - shipped) + assert not missing, ( + f"this is a CUDA row but {missing} are not in the wheel, so it would install as a " + f"CUDA build with no CUDA delegate. Shipped: {sorted(shipped)}" + ) + print(f"✓ the CUDA libraries ship ({len(expected)} of them)") + + +def test_cuda_runtime_is_declared() -> None: + """The wheel links the CUDA runtime without bundling it, so it must declare it. + + Without this a user installs the wheel and has nothing to resolve libcudart from, which + surfaces as a loader error at the first import rather than as a resolution failure at + install time. + """ + import importlib.metadata as metadata + + requirements = metadata.requires("executorch") or [] + cuda = [ + requirement + for requirement in requirements + if "nvidia" in requirement.lower() or "cuda" in requirement.lower() + ] + assert cuda, ( + "this is a CUDA row but the wheel declares no CUDA runtime dependency, so nothing " + "would install the libraries its delegate links" + ) + print(f"✓ the CUDA runtime is declared ({len(cuda)} requirements)") + + +def test_cuda_libraries_resolve_relatively() -> None: + """Each CUDA library must reach its runtime through a relative path. + + An absolute toolkit path names the machine that built the wheel. It resolves there and + nowhere else, so the wheel would work only on a builder. + + Every shipped library that links the CUDA runtime is inspected, wherever it lives. Naming + only the two in lib/ skipped libaoti_cuda_shims.so, which sits under backends/cuda/, links + cudart and curand, and carries the device code, so an absolute toolkit path on the library + that matters most shipped green. + """ + readelf = test_shared_libraries._tool("readelf") + assert readelf is not None, "readelf is required to inspect the wheel" + + package_dir = _package_dir() + libraries = sorted(test_shared_libraries._shipped_shared_objects(package_dir)) + # Without this the loop below finds nothing on a wheel that ships no CUDA library and + # reports a pass, which is the same as having no check at all. + assert libraries, f"no shared libraries found under {package_dir}" + + linked_to_cuda = [] + for library in libraries: + output = subprocess.run( + [readelf, "-d", str(library)], capture_output=True, text=True, check=True + ).stdout + if any("NEEDED" in line and "libcud" in line for line in output.splitlines()): + linked_to_cuda.append((library, output)) + + assert linked_to_cuda, ( + "no shipped library links the CUDA runtime, so this check inspected nothing. A CUDA " + "row must ship the libraries it is named for." + ) + for library, output in linked_to_cuda: + name = library.relative_to(package_dir) + entries: list[str] = [] + for line in output.splitlines(): + if "RPATH" in line or "RUNPATH" in line: + entries += line.split("[", 1)[1].rstrip("]").strip().split(":") + relative = [ + entry + for entry in entries + if entry.startswith("$ORIGIN") and "nvidia" in entry + ] + assert relative, ( + f"{name} links the CUDA runtime but has no relative path to the CUDA wheels " + f"installed beside it, so it can only resolve where the builder had a toolkit: " + f"{entries}" + ) + print(f"✓ {name} resolves the CUDA runtime relatively ({relative[0]})") + + +def _row_architectures() -> list[str]: + """The architectures this row claims, from the same script the build uses. + + A refusal from that script is a fault, not an absence. It returns non-zero when a CUDA row reaches it + with no version, which is precisely the case that would otherwise build device code for whatever GPU the + builder happens to have, so swallowing it here would hide the one failure this check exists to catch. + + EXECUTORCH_BUILD_CUDA is passed through because that is how the build invokes the script, and the + refusal is conditional on it. Without it the script returned an empty list on a CUDA row that had lost + its version, this check reported nothing to do, and the assertion below could never fire. + """ + script = pathlib.Path(__file__).parent / "cuda_arch_list.sh" + assert script.is_file(), f"the architecture script is missing at {script}" + result = subprocess.run( + ["bash", "-c", f"source {script}; executorch_cuda_arch_list"], + capture_output=True, + text=True, + check=False, + env={**os.environ, "EXECUTORCH_BUILD_CUDA": "1"}, + ) + assert result.returncode == 0, ( + f"the architecture script refused this row with exit {result.returncode}, so the build had no list " + f"to compile against: {result.stderr.strip()[:300]}" + ) + # "8.0 9.0" describes sm_80 and sm_90. + return ["sm_" + value.replace(".", "") for value in result.stdout.split()] + + +def test_device_code_covers_the_row() -> None: + """Every GPU the row claims must have device code in the shipped libraries. + + A row that promises a GPU it did not compile for produces a wheel that installs and then dies + at the first kernel launch, which is the worst failure to publish. + """ + expected = _row_architectures() + if not expected: + print("- this row claims no GPU architectures, nothing to check") + return + + cuobjdump = test_shared_libraries._tool("cuobjdump") + if cuobjdump is None: + raise AssertionError( + "cuobjdump is required to check device code, and this is a CUDA row. Without it a " + "wheel missing code for a claimed GPU would ship unnoticed." + ) + + # Searched across every shipped library rather than a named one. The kernels are compiled + # into their own library, not into the delegate, and which library holds them is an internal + # detail. What the row promises is that the wheel covers those GPUs. + present: set[str] = set() + inspected = [] + with_device_code: dict = {} + for library in sorted(_package_dir().rglob("*.so")): + listed = subprocess.run( + [cuobjdump, "--list-elf", str(library)], + capture_output=True, + text=True, + check=False, + ).stdout + found = { + token + for token in listed.replace(".", " ").split() + if token.startswith("sm_") + } + if found: + inspected.append(f"{library.name} ({', '.join(sorted(found))})") + present |= found + with_device_code[library.name] = found + + assert inspected, ( + "no shipped library contains any GPU device code, so this wheel cannot run a model on any " + f"GPU, while the row claims {expected}" + ) + missing = sorted(set(expected) - present) + assert not missing, ( + f"the row claims {expected} but the wheel carries no device code for {missing}. " + f"Found: {inspected}. A user with one of those GPUs would install this wheel and fail at " + "the first kernel launch." + ) + # Every library that carries device code has to cover the row on its own. Unioning + # across libraries let a library with kernels cover only part of the row while an + # unrelated object supplied the rest, so on a GPU the first one did not compile for + # there was no executable kernel even though the union looked complete. + short = sorted(set(expected)) + for library in sorted(with_device_code): + library_missing = sorted(set(expected) - with_device_code[library]) + assert not library_missing, ( + f"{library} carries GPU device code but none for {library_missing}, while the row " + f"claims {short}. Checking the union across libraries hid this: another shipped " + "object supplied those architectures, and on such a GPU this library would have no " + "executable kernel." + ) + print(f"✓ device code covers the row in every library that has any: {inspected}") + + +def test_portable_device_code_is_present() -> None: + """The newest architecture must also ship in its portable form. + + The build appends "+PTX" for the top architecture so a GPU newer than any in the row can + still run, by having the driver compile that portable form at load time. Without it such a + GPU gets no usable code at all. + + Checked with --list-ptx rather than --list-elf. --list-elf prints byte-identical output for + a library built with or without the portable form, so it cannot see this. --list-ptx prints + an entry only for the library that has it. The entry is named for the target architecture, + "sm_90.ptx" rather than "compute_90.ptx", which is what the real tool prints. + """ + expected = _row_architectures() + if not expected: + print("- this row claims no GPU architectures, nothing to check") + return + + cuobjdump = test_shared_libraries._tool("cuobjdump") + assert ( + cuobjdump is not None + ), "cuobjdump is required to check the portable device code, and this is a CUDA row." + + # The newest architecture in the row, which is the one the build makes portable. + newest = max(expected, key=lambda name: int(name.removeprefix("sm_"))) + + found_in = [] + for library in sorted(_package_dir().rglob("*.so")): + listed = subprocess.run( + [cuobjdump, "--list-ptx", str(library)], + capture_output=True, + text=True, + check=False, + ).stdout + if newest in listed.replace(".", " ").split(): + found_in.append(library.name) + + assert found_in, ( + f"no shipped library carries portable device code for {newest}, the newest architecture " + f"in this row ({sorted(expected)}). A GPU newer than {newest} would install this wheel and " + "find no code it can run. The build appends the portable form for exactly this case, so " + "either it was dropped or the spelling in the architecture list is wrong." + ) + print(f"✓ portable device code for {newest} ships in {', '.join(found_in)}") + + +def test_the_delegate_registers() -> None: + """The delegate has to appear in the runtime's backend list, not merely be present as a file. + + Registration happens in a static initializer, which a normal link discards because nothing in the + program references it. Keeping it alive needs a linker option, and a wheel whose delegate ships but + does not register would load a delegated program and fail with an unregistered backend. That is the + failure this whole layout is most able to introduce, so it is worth asserting rather than assuming. + + Needs no GPU: registration is a link-time property, checked by importing. + """ + from executorch.extension.pybindings.portable_lib import ( + _get_registered_backend_names, + ) + + registered = _get_registered_backend_names() + assert "CudaBackend" in registered, ( + f"the wheel ships the CUDA delegate but CudaBackend is not registered: {registered}. " + "The library is present and its static initializer did not run, which means the option " + "that keeps it on the link line stopped working." + ) + print(f"✓ the delegate registers: CudaBackend among {len(registered)} backend(s)") + + +if __name__ == "__main__": + assert platform.system() == "Linux", "the CUDA rows are Linux only" + + test_cuda_libraries_are_shipped() + test_cuda_runtime_is_declared() + test_cuda_libraries_resolve_relatively() + test_device_code_covers_the_row() + test_portable_device_code_is_present() + test_the_delegate_registers() + + # The backend registrations a CPU Linux row asserts also apply here: the CUDA build enables + # OpenVINO on every Linux architecture and downloads the QNN SDK on x86_64, so a CUDA wheel + # carries both backends and needs both to register. + from executorch.extension.pybindings.portable_lib import ( + _get_registered_backend_names, + ) + + registered = _get_registered_backend_names() + if platform.machine() in ("x86_64", "amd64"): + assert ( + "QnnBackend" in registered + ), f"QnnBackend not found in registered backends: {registered}" + print("✓ QnnBackend is registered") + assert ( + "OpenvinoBackend" in registered + ), f"OpenvinoBackend not found in registered backends: {registered}" + print("✓ OpenvinoBackend is registered") + + test_base.test_cmsis_nn_install() + + # The packaging and linking checks a CPU wheel is held to still apply: one owner per + # component, no build-tree paths, and a C++ application able to link what the wheel + # ships. + with tempfile.TemporaryDirectory() as work_dir: + test_shared_libraries.run_tests(Path(work_dir)) + with tempfile.TemporaryDirectory() as work_dir: + test_cpp_sdk.run_tests(Path(work_dir)) + + test_base.run_tests( + model_tests=[ + test_base.ModelTest( + model=Model.Mv3, + backend=Backend.XnnpackQuantizationDelegation, + ), + ] + ) diff --git a/.ci/scripts/wheel/test_shared_libraries.py b/.ci/scripts/wheel/test_shared_libraries.py index 4c976be18c2..dbb889bf59c 100644 --- a/.ci/scripts/wheel/test_shared_libraries.py +++ b/.ci/scripts/wheel/test_shared_libraries.py @@ -433,6 +433,26 @@ def _wheel_cuda_train() -> str: _REQUIRED_ON_A_CUDA_WHEEL = "cuda-wheel-only" +# The exact dependency names packaging declares per CUDA train, mirroring +# _CUDA_RUNTIME_PACKAGES in setup.py. Listed here rather than imported because setup.py +# runs a build when imported, and duplicated deliberately so a rename on the packaging +# side has to be made here too rather than silently agreeing with itself. +_EXPECTED_CUDA_PACKAGES = { + "12": ( + "nvidia-cuda-runtime-cu12", + "nvidia-cublas-cu12", + "nvidia-curand-cu12", + "nvidia-cuda-nvrtc-cu12", + ), + "13": ( + "nvidia-cuda-runtime", + "nvidia-cublas", + "nvidia-curand", + "nvidia-cuda-nvrtc", + ), +} + + # Each component the wheel ships as its own library, the symbols that identify it, # and the library that must own them. `required` says whether the owner has to be # present: the optimized kernels are optional, because a wheel built without them @@ -1618,17 +1638,34 @@ def test_model_matches_eager_pytorch(work_dir: Path) -> None: def test_declared_dependencies_match_the_wheel_tag() -> None: - """A CPU wheel must not declare the CUDA runtime, and a CUDA wheel must declare it. + """A CPU wheel must not declare the CUDA runtime, and a CUDA wheel must declare its own train. The tag is what a user resolves against, so a mismatch is a promise the wheel cannot keep in either direction: a CPU wheel that pulls the CUDA packages costs a user hundreds of megabytes it never loads, and a CUDA wheel that declares nothing leaves the runtime unresolvable. + Declaring the wrong train is the quiet case, and the reason this checks the names rather than + only their presence. The CUDA 12 packages are published with a "-cu12" suffix and the CUDA 13 + ones without, so a cu130 wheel that asked for the cu12 packages would install a runtime its + libraries cannot load, while looking correctly specified. + This is metadata only, so no library check can see it. A CPU wheel that wrongly declared the CUDA runtime passed every other check in this file. """ requirements = importlib.metadata.requires("executorch") or [] - cuda = sorted(r.split()[0] for r in requirements if r.lower().startswith("nvidia")) + + # Split off any environment marker AND any version specifier. The name, the specifier + # and the marker can arrive as one token, so taking the first whitespace-separated + # word left "nvidia-cuda-runtime-cu12==12.6.77" as the name and made a correctly + # specified wheel fail the moment any CUDA dependency gained a pin. + def distribution_name(requirement: str) -> str: + return re.split(r"[\s;\[<>=!~(]", requirement.strip(), maxsplit=1)[0] + + cuda = sorted( + name + for name in (distribution_name(r) for r in requirements) + if name.lower().startswith("nvidia") + ) # The local version segment of the installed version states what the wheel was built for. version = importlib.metadata.version("executorch") @@ -1640,7 +1677,35 @@ def test_declared_dependencies_match_the_wheel_tag() -> None: f"version {version} says this is a CUDA wheel, but it declares no CUDA runtime " "packages, so nothing resolves the runtime it links" ) - print(f"✓ this CUDA wheel declares the runtime ({len(cuda)} packages)") + # Compared as sets in both directions rather than as a name suffix: for CUDA 13 the + # expected suffix is the empty string and every name ends with that, so a suffix test + # accepted a name from any train whose spelling happened not to be one of the two + # literals it also excluded. Measured: a cu130 wheel declaring nvidia-cuda-runtime-cu11 + # passed. The reverse check catches the other side of the same defect: a wheel that + # declares one package and omits the others still cannot load, and one-direction only + # would accept it. + train = local[len("cu") : len("cu") + 2] + expected = set(_EXPECTED_CUDA_PACKAGES.get(train, ())) + assert expected, ( + f"version {version} names CUDA train {train}, which this check has no expected " + f"package list for. Add it beside the packaging list it mirrors." + ) + actual = set(cuda) + wrong = sorted(actual - expected) + missing = sorted(expected - actual) + assert not wrong, ( + f"version {version} is a CUDA {train} wheel, but it declares {wrong}, which belong to " + f"another CUDA train. Expected only {sorted(expected)}. A user would install a runtime " + "this wheel's libraries cannot load." + ) + assert not missing, ( + f"version {version} is a CUDA {train} wheel, but it does not declare {missing} " + f"(expected {sorted(expected)}). A user installing this wheel would end up without part " + "of the CUDA runtime the wheel's libraries need." + ) + print( + f"✓ this CUDA {train} wheel declares its own runtime ({len(cuda)} packages)" + ) else: assert not cuda, ( f"version {version} is not a CUDA wheel, yet it declares {cuda}. A user installing it " diff --git a/.github/scripts/filter_cuda_matrix.py b/.github/scripts/filter_cuda_matrix.py new file mode 100644 index 00000000000..385ea9385cb --- /dev/null +++ b/.github/scripts/filter_cuda_matrix.py @@ -0,0 +1,238 @@ +#!/usr/bin/env python3 +# Copyright (c) Meta Platforms, Inc. and affiliates. +# All rights reserved. +# +# This source code is licensed under the BSD-style license found in the +# LICENSE file in the root directory of this source tree. + +"""Narrow the generated build matrix to the rows a GPU wheel can honestly support. + +The shared matrix generator emits every CUDA version and Python version it knows about. +Building all of them would publish wheels for combinations nothing can verify, and a GPU +wheel that installs and then cannot run is worse than one that does not exist: the failure +appears when a model runs, and it looks like a model problem rather than a packaging one. + +A row is kept only when both of these hold: + + a GPU exists that the row's device code covers + a PyTorch build is published for that CUDA version and architecture + +Running a real model before release is a separate gate, on hardware that has the matching +GPU, so a row can be published for a CUDA version no machine here can execute. + +The values below are the current answers to those questions. They are written out rather +than derived because each one is an external fact that can change independently. +""" + +import argparse +import json +import sys +from typing import Any, Dict, List + +# Python versions that are deliberately NOT published, with the reason, so a row naming one +# is rejected for a stated cause rather than for merely being absent from the supported list. +# 3.14 is excluded because the current CPU wheel rows already fail on it for an unrelated +# reason in the example requirements, so a GPU row would inherit a known-broken build. The +# free-threaded builds are excluded because the CUDA dependencies are not published for them. +# +# This is documentation, not the gate. The gate is SUPPORTED_PYTHON_VERSIONS below: anything +# not on that list is rejected whether or not it appears here. +DISABLED_PYTHON_VERSIONS: List[str] = ["3.13t", "3.14", "3.14t", "3.15", "3.15t"] + +# CUDA versions to publish. +# +# Chosen so that every consumer row can find a matching wheel rather than by what is +# convenient to verify. A delegate built against one of these has to be able to depend on an +# ExecuTorch wheel for the same CUDA version, and a missing version means that consumer has +# nothing to depend on: +# +# cu126 the floor, and what Jetson devices are limited to +# cu130 the generator's stable choice, and the default for accelerator consumers +# cu132 the newest, which consumers building against a current TensorRT need +# +# cu132 is included even though no machine here can execute it, because omitting it would +# leave a published consumer row with no ExecuTorch wheel to pair with. The packaging +# properties are checked on every row; executing a model is a release-gate step on hardware +# that has the matching GPU. +SUPPORTED_CUDA_VERSIONS: List[str] = ["cu126", "cu130", "cu132"] + +# Python versions to publish, stated rather than derived for the same reason the CUDA +# versions are. Deriving them from the rows that survived the filter made the release +# guard below unable to notice a python that disappeared from every supported train: with +# nothing left to compare, a release quietly published nine wheels instead of twelve. +# Keep in step with the python-versions list in the CUDA wheel workflows. +SUPPORTED_PYTHON_VERSIONS: List[str] = ["3.10", "3.11", "3.12", "3.13"] + +# The single row built for a pull request. A full matrix on every push would cost hours for +# little signal, and this pair is the one with a machine that can run a model on it. +PR_PYTHON_VERSION: str = "3.12" +PR_CUDA_VERSION: str = "cu130" + +# Jetson devices are their own row: a JetPack image, one Python version, and one CUDA +# version. Kept empty on purpose today, so no Jetson row is emitted. +# +# The generic aarch64 CUDA 12.6 wheel does compile sm_87 device code for one embedded +# module, so the wheel itself is not the blocker. What is: published PyTorch stopped +# shipping sm_87 device code after 2.8.0, so a Jetson row today would produce a wheel +# whose PyTorch dependency cannot execute on the device. Populate this when that +# changes. +# +# Because both lists are empty, asking for the JetPack rows can only produce an empty result. +# No workflow asks, and the request is rejected up front with that reason rather than left to +# surface as the generic "the filter produced no rows" message, which reads as a broken +# matrix rather than as a row that is deliberately not built yet. +JETPACK_PYTHON_VERSIONS: List[str] = [] +JETPACK_CUDA_VERSIONS: List[str] = [] +JETPACK_CONTAINER_IMAGE: str = "nvcr.io/nvidia/l4t-jetpack:r36.4.0" + + +def keep(item: Dict[str, Any], is_jetpack: bool) -> bool: + """Whether this row should be built, adjusting its container image where needed.""" + # An allowlist, the same shape as the CUDA test below. Testing only the disabled list + # let any python not on it through: passing a 3.9 row returned success and emitted it, + # and the only thing preventing that today is both workflows happening to pin the list + # they pass in. + if item["python_version"] not in SUPPORTED_PYTHON_VERSIONS: + return False + + if is_jetpack: + if ( + item["python_version"] in JETPACK_PYTHON_VERSIONS + and item["desired_cuda"] in JETPACK_CUDA_VERSIONS + ): + item["container_image"] = JETPACK_CONTAINER_IMAGE + return True + return False + + if item["desired_cuda"] not in SUPPORTED_CUDA_VERSIONS: + return False + + return True + + +def _version_rank(cuda: str) -> int: + """Where a CUDA version sits in the supported list, or -1 when it is not supported at all.""" + try: + return SUPPORTED_CUDA_VERSIONS.index(cuda) + except ValueError: + return -1 + + +def only_pull_request_row(items: List[Dict[str, Any]]) -> List[Dict[str, Any]]: + """One representative row, so a pull request does not build the whole matrix. + + Chosen by preference rather than exact match, so a request that does not appear in the + generated matrix degrades to the closest supported combination instead of falling off the + end. + """ + if not items: + return [] + + # Looked up once, and tolerantly: a PR_CUDA_VERSION that falls off SUPPORTED_CUDA_VERSIONS used to + # raise here and break every pull request while releases kept working, which is the wrong way round + # for a constant that only chooses which single row to build. + wanted = _version_rank(PR_CUDA_VERSION) + + def rank(item: Dict[str, Any]) -> tuple: + # Closeness peaks at the requested version, then falls off, and it outranks the python match. + # Ranking python first picked a wheel for a CUDA version nothing on hand can execute whenever the + # generator skewed the two axes, and the point of building one row is to get signal from it. + offered = _version_rank(item["desired_cuda"]) + # Negative above the requested version, so a newer one never outranks an older one a machine here + # can actually run. + closeness = offered if offered <= wanted else wanted - offered + return (closeness, item["python_version"] == PR_PYTHON_VERSION) + + return [max(items, key=rank)] + + +def main(argv: List[str]) -> None: + parser = argparse.ArgumentParser() + parser.add_argument("--matrix", required=True, help="the generated matrix, as JSON") + parser.add_argument( + "--jetpack", default="false", help="build the Jetson row instead" + ) + parser.add_argument("--limit-pr-builds", default="false", help="build one row only") + args = parser.parse_args(argv) + + try: + matrix = json.loads(args.matrix) + except json.JSONDecodeError as error: + print(f"could not parse the matrix: {error}", file=sys.stderr) + sys.exit(1) + + is_jetpack = args.jetpack.lower() == "true" + if is_jetpack and not (JETPACK_PYTHON_VERSIONS and JETPACK_CUDA_VERSIONS): + # Rejected here rather than allowed to fall through to an empty result, so the reason + # is the actual one. Nothing passes this flag today. + print( + "the JetPack rows are not published yet: JETPACK_PYTHON_VERSIONS and " + "JETPACK_CUDA_VERSIONS are empty because published PyTorch carries no device code " + "for that GPU architecture, so any wheel built here could not run on the device. " + "Populate both lists to enable this row.", + file=sys.stderr, + ) + sys.exit(1) + items = [item for item in matrix.get("include", []) if keep(item, is_jetpack)] + + if args.limit_pr_builds.lower() == "true" and items: + items = only_pull_request_row(items) + elif items and not is_jetpack: + # A release has to publish every combination this policy advertises. Comparing the result against + # what the generator offered cannot catch anything, because both sides apply the same conditions, so + # the difference is empty by construction and the check never fires. The policy's own list is the + # thing to compare against: a CUDA version the generator stopped offering otherwise disappears from + # the release silently, and a missing job is a green check for a wheel that was never built. + # + # The generic rows only. A JetPack release advertises the single pair its own lists name rather than + # every supported CUDA version, so checking it against this list would fail a correct release. + # + # Both axes come from this policy's own lists, not from the matrix. Reading the generator's python + # axis pulled in rows this policy never builds, and deriving it from the rows that survived went + # blind to a python that disappeared from every supported train. The generator lives in another + # repository and its axes move independently of what this policy promises to publish. + built = {(item["python_version"], item["desired_cuda"]) for item in items} + # A train that produced no row at all is missing for every python, so reporting it per python + # would read as a python problem. Named on its own instead, and first, because the per-pair + # report below would otherwise bury it. + absent_trains = sorted( + set(SUPPORTED_CUDA_VERSIONS) - {cuda for _, cuda in built} + ) + if absent_trains: + print( + f"this policy publishes {SUPPORTED_CUDA_VERSIONS}, but the generator offered no row " + f"this filter could keep for {absent_trains}, so a release would publish no wheel for " + "that CUDA version at all", + file=sys.stderr, + ) + sys.exit(1) + missing = sorted( + f"{python}/{cuda}" + for python in SUPPORTED_PYTHON_VERSIONS + for cuda in SUPPORTED_CUDA_VERSIONS + if (python, cuda) not in built + ) + if missing: + print( + f"this policy publishes {SUPPORTED_CUDA_VERSIONS} for each of " + f"{SUPPORTED_PYTHON_VERSIONS}, but {len(missing)} combination(s) produced no row, so a " + f"release would publish no wheel for them: {missing}", + file=sys.stderr, + ) + sys.exit(1) + + # Fail loudly on an empty result. A silently empty matrix produces a workflow with no + # build job, which shows up as a green check for a build that never happened. + if not items: + print( + "the filter produced no rows to build, so nothing would be verified. " + f"jetpack={is_jetpack}, supported CUDA={SUPPORTED_CUDA_VERSIONS}", + file=sys.stderr, + ) + sys.exit(1) + + print(json.dumps({"include": items})) + + +if __name__ == "__main__": + main(sys.argv[1:]) diff --git a/.github/workflows/build-wheels-cuda-aarch64-linux.yml b/.github/workflows/build-wheels-cuda-aarch64-linux.yml new file mode 100644 index 00000000000..6e8ffbfe981 --- /dev/null +++ b/.github/workflows/build-wheels-cuda-aarch64-linux.yml @@ -0,0 +1,100 @@ +# From https://github.com/pytorch/test-infra/wiki/Using-Nova-Reusable-Build-Workflows +name: Build Aarch64 Linux CUDA Wheels + +on: + pull_request: + paths: + - .ci/**/* + - .github/scripts/filter_cuda_matrix.py + - .github/workflows/build-wheels-cuda-aarch64-linux.yml + - '**/CMakeLists.txt' + - backends/cuda/**/* + - examples/**/* + - extension/cuda/**/* + - install_requirements.py + - install_utils.py + - pyproject.toml + - setup.py + - tools/cmake/**/* + push: + branches: + - nightly + - release/* + tags: + # NOTE: Binary build pipelines should only get triggered on release candidate builds + # Release candidate tags look like: v1.11.0-rc1 + - v[0-9]+.[0-9]+.[0-9]+-rc[0-9]+ + - ciflow/binaries/* + workflow_dispatch: + +jobs: + generate-matrix: + uses: pytorch/test-infra/.github/workflows/generate_binary_build_matrix.yml@main + with: + package-type: wheel + os: linux-aarch64 + test-infra-repository: pytorch/test-infra + test-infra-ref: main + with-cuda: enable + with-cpu: disable + with-rocm: disable + python-versions: '["3.10", "3.11", "3.12", "3.13"]' + + # The generator emits every CUDA version it knows about. Publishing all of them would ship + # wheels for combinations nothing can verify, so this keeps only the rows with a GPU to run + # them on. The script fails rather than emitting an empty matrix, because a workflow with no + # build job reads as a pass. + filter-matrix: + needs: generate-matrix + runs-on: ubuntu-latest + outputs: + matrix: ${{ steps.filter.outputs.matrix }} + steps: + - uses: actions/setup-python@v6 + with: + python-version: '3.12' + - uses: actions/checkout@v4 + - name: Filter the matrix + id: filter + run: | + set -eou pipefail + MATRIX_BLOB=${{ toJSON(needs.generate-matrix.outputs.matrix) }} + LIMIT_PR=${{ github.event_name == 'pull_request' && 'true' || 'false' }} + MATRIX_BLOB="$(python3 .github/scripts/filter_cuda_matrix.py \ + --matrix "${MATRIX_BLOB}" --limit-pr-builds "${LIMIT_PR}")" + echo "${MATRIX_BLOB}" + echo "matrix=${MATRIX_BLOB}" >> "${GITHUB_OUTPUT}" + + build: + needs: filter-matrix + permissions: + id-token: write + contents: read + strategy: + fail-fast: false + matrix: + include: + - repository: pytorch/executorch + pre-script: .ci/scripts/wheel/pre_build_script.sh + post-script: .ci/scripts/wheel/post_build_script.sh + smoke-test-script: .ci/scripts/wheel/test_cuda_linux.py + package-name: executorch + name: ${{ matrix.repository }} + uses: pytorch/test-infra/.github/workflows/build_wheels_linux.yml@main + with: + repository: ${{ matrix.repository }} + ref: "" + test-infra-repository: pytorch/test-infra + test-infra-ref: main + build-matrix: ${{ needs.filter-matrix.outputs.matrix }} + submodules: recursive + env-var-script: .ci/scripts/wheel/envvar_cuda_linux.sh + pre-script: ${{ matrix.pre-script }} + post-script: ${{ matrix.post-script }} + package-name: ${{ matrix.package-name }} + smoke-test-script: ${{ matrix.smoke-test-script }} + trigger-event: ${{ github.event_name }} + # Required for aarch64. Without it the shared build workflow prepares an x86_64 job + # and skips the aarch64 conda install, so the first build step fails on a missing + # conda. + architecture: aarch64 diff --git a/.github/workflows/build-wheels-cuda-linux.yml b/.github/workflows/build-wheels-cuda-linux.yml new file mode 100644 index 00000000000..7a59e07545e --- /dev/null +++ b/.github/workflows/build-wheels-cuda-linux.yml @@ -0,0 +1,96 @@ +# From https://github.com/pytorch/test-infra/wiki/Using-Nova-Reusable-Build-Workflows +name: Build Linux CUDA Wheels + +on: + pull_request: + paths: + - .ci/**/* + - .github/scripts/filter_cuda_matrix.py + - .github/workflows/build-wheels-cuda-linux.yml + - '**/CMakeLists.txt' + - backends/cuda/**/* + - examples/**/* + - extension/cuda/**/* + - install_requirements.py + - install_utils.py + - pyproject.toml + - setup.py + - tools/cmake/**/* + push: + branches: + - nightly + - release/* + tags: + # NOTE: Binary build pipelines should only get triggered on release candidate builds + # Release candidate tags look like: v1.11.0-rc1 + - v[0-9]+.[0-9]+.[0-9]+-rc[0-9]+ + - ciflow/binaries/* + workflow_dispatch: + +jobs: + generate-matrix: + uses: pytorch/test-infra/.github/workflows/generate_binary_build_matrix.yml@main + with: + package-type: wheel + os: linux + test-infra-repository: pytorch/test-infra + test-infra-ref: main + with-cuda: enable + with-cpu: disable + with-rocm: disable + python-versions: '["3.10", "3.11", "3.12", "3.13"]' + + # The generator emits every CUDA version it knows about. Publishing all of them would ship + # wheels for combinations nothing can verify, so this keeps only the rows with a GPU to run + # them on. The script fails rather than emitting an empty matrix, because a workflow with no + # build job reads as a pass. + filter-matrix: + needs: generate-matrix + runs-on: ubuntu-latest + outputs: + matrix: ${{ steps.filter.outputs.matrix }} + steps: + - uses: actions/setup-python@v6 + with: + python-version: '3.12' + - uses: actions/checkout@v4 + - name: Filter the matrix + id: filter + run: | + set -eou pipefail + MATRIX_BLOB=${{ toJSON(needs.generate-matrix.outputs.matrix) }} + LIMIT_PR=${{ github.event_name == 'pull_request' && 'true' || 'false' }} + MATRIX_BLOB="$(python3 .github/scripts/filter_cuda_matrix.py \ + --matrix "${MATRIX_BLOB}" --limit-pr-builds "${LIMIT_PR}")" + echo "${MATRIX_BLOB}" + echo "matrix=${MATRIX_BLOB}" >> "${GITHUB_OUTPUT}" + + build: + needs: filter-matrix + permissions: + id-token: write + contents: read + strategy: + fail-fast: false + matrix: + include: + - repository: pytorch/executorch + pre-script: .ci/scripts/wheel/pre_build_script.sh + post-script: .ci/scripts/wheel/post_build_script.sh + smoke-test-script: .ci/scripts/wheel/test_cuda_linux.py + package-name: executorch + name: ${{ matrix.repository }} + uses: pytorch/test-infra/.github/workflows/build_wheels_linux.yml@main + with: + repository: ${{ matrix.repository }} + ref: "" + test-infra-repository: pytorch/test-infra + test-infra-ref: main + build-matrix: ${{ needs.filter-matrix.outputs.matrix }} + submodules: recursive + env-var-script: .ci/scripts/wheel/envvar_cuda_linux.sh + pre-script: ${{ matrix.pre-script }} + post-script: ${{ matrix.post-script }} + package-name: ${{ matrix.package-name }} + smoke-test-script: ${{ matrix.smoke-test-script }} + trigger-event: ${{ github.event_name }} diff --git a/backends/cuda/CMakeLists.txt b/backends/cuda/CMakeLists.txt index 05f238401a4..07daef607e1 100644 --- a/backends/cuda/CMakeLists.txt +++ b/backends/cuda/CMakeLists.txt @@ -42,6 +42,86 @@ if(NOT CMAKE_CUDA_COMPILER) check_language(CUDA) endif() +# Take the architectures from the release row when it names them, before the +# language is enabled, since CMake fixes them at that point. Without this the +# build uses CMake's default, which on some devices is older than the intrinsics +# these sources use, and the compile fails with an undefined identifier that +# looks like a source problem. +# +# TORCH_CUDA_ARCH_LIST is the variable the surrounding build environment already +# sets, in PyTorch's dotted form. CMake wants bare integers, so "9.0" becomes +# 90. A "+PTX" suffix asks for the portable form in addition to the compiled +# one, which is what PyTorch means by it, so it adds the -virtual kind rather +# than replacing the -real one. +if(NOT DEFINED CMAKE_CUDA_ARCHITECTURES AND DEFINED ENV{TORCH_CUDA_ARCH_LIST}) + set(_cuda_arch_list "") + string(STRIP "$ENV{TORCH_CUDA_ARCH_LIST}" _cuda_arch_request) + string(TOLOWER "${_cuda_arch_request}" _cuda_arch_request_lower) + # CMake understands these three itself, and they cannot be combined with a + # version list, so they pass straight through as the whole value. Dropping + # them left the variable unset and the compile fell back to CMake's own + # default, measured as 52 under CUDA 12.8 and 75 under 13.0, which is the + # outcome this block exists to prevent. + if(_cuda_arch_request_lower MATCHES "^(native|all|all-major)$") + set(CMAKE_CUDA_ARCHITECTURES ${_cuda_arch_request_lower}) + message( + STATUS + "CUDA architectures from TORCH_CUDA_ARCH_LIST: ${CMAKE_CUDA_ARCHITECTURES}" + ) + else() + string(REPLACE " " ";" _cuda_arch_items "${_cuda_arch_request}") + foreach(_cuda_arch_item IN LISTS _cuda_arch_items) + if(_cuda_arch_item STREQUAL "") + continue() + endif() + set(_cuda_arch_ptx OFF) + if(_cuda_arch_item MATCHES "\\+PTX$") + set(_cuda_arch_ptx ON) + string(REPLACE "+PTX" "" _cuda_arch_item "${_cuda_arch_item}") + endif() + string(REPLACE "." "" _cuda_arch_number "${_cuda_arch_item}") + # A trailing letter selects an architecture-specific feature set, as in + # "10.0a", which CMake accepts and this build passes through. + if(_cuda_arch_number MATCHES "^[0-9]+[a-z]?$") + list(APPEND _cuda_arch_list "${_cuda_arch_number}-real") + if(_cuda_arch_ptx) + list(APPEND _cuda_arch_list "${_cuda_arch_number}-virtual") + endif() + else() + # Said out loud, because dropping an entry silently is how the whole list + # ends up empty and the compile falls back to CMake's default, which is + # the failure this block exists to prevent. Named GPU families PyTorch + # accepts, such as "Hopper", land here. + message( + WARNING + "Ignoring \"${_cuda_arch_item}\" from TORCH_CUDA_ARCH_LIST: expected a " + "version such as 9.0, 9.0+PTX or 10.0a." + ) + endif() + endforeach() + if(_cuda_arch_list) + list(REMOVE_DUPLICATES _cuda_arch_list) + set(CMAKE_CUDA_ARCHITECTURES ${_cuda_arch_list}) + message( + STATUS + "CUDA architectures from TORCH_CUDA_ARCH_LIST: ${CMAKE_CUDA_ARCHITECTURES}" + ) + else() + # Fail here rather than let CMAKE_CUDA_ARCHITECTURES fall back to the + # compiler default, which is the failure this block exists to prevent. + # Reached when the variable held only named GPU families PyTorch accepts, + # such as "Hopper", or was empty, both of which the parser above passes + # over. + message( + FATAL_ERROR + "TORCH_CUDA_ARCH_LIST was set (\"$ENV{TORCH_CUDA_ARCH_LIST}\") but " + "contained no version this parser recognises. Use versions such as " + "9.0, 9.0+PTX or 10.0a rather than named GPU families like \"Hopper\"." + ) + endif() + endif() +endif() + if(CMAKE_CUDA_COMPILER) enable_language(CUDA) endif() diff --git a/install_requirements.py b/install_requirements.py index 1aedcf6f0f8..101ff18ea68 100644 --- a/install_requirements.py +++ b/install_requirements.py @@ -7,6 +7,7 @@ import argparse import os +import platform import subprocess import sys @@ -45,7 +46,15 @@ def install_requirements(use_pytorch_nightly): # Determine the appropriate PyTorch URL based on CUDA delegate status torch_url = determine_torch_url(TORCH_URL_BASE) - torchao_url = determine_torch_url(TORCHAO_URL_BASE) + # torchao's CUDA channel publishes x86_64 only, so asking for a CUDA build makes the pin + # unsatisfiable on aarch64. Only that case is special-cased: falling back everywhere would + # change which torchao a CPU x86_64 install resolves, and the CUDA build is genuinely wanted + # where it exists. Nothing in the wheel links or bundles torchao; it is a quantization + # workflow dependency of the examples and tests. + if platform.machine().lower() in ("aarch64", "arm64"): + torchao_url = TORCHAO_URL_BASE + else: + torchao_url = determine_torch_url(TORCHAO_URL_BASE) # pip packages needed by exir. TORCH_PACKAGE = [