Skip to content

Commit 1bc3075

Browse files
shoumikhinshoumikhin
committed
Ship a prebuilt C++ SDK in the ExecuTorch Linux wheel
## Why this is needed Today, using the ExecuTorch runtime from a C++ program means building ExecuTorch from source: clone the repo, sync submodules, and run a CMake build before you can compile and link your own runner. The pip wheel only ships the Python runtime module (`_portable_lib`) plus a small set of headers meant for authoring custom operators. There is no way to just `pip install executorch` and link a standalone C++ application against the runtime. This is friction for anyone whose deployment path is C++ (the common case for on-device inference) and who already has the wheel installed for export. The runtime is compiled during the wheel build and then discarded. This change ships the runtime as a linkable shared library, its public headers, and a CMake package config inside the Linux wheel, so a C++ program can link the ExecuTorch runtime with no source checkout and no separate build. ## What is inside Added to the Linux wheel (nothing removed; other platforms unchanged): - `executorch/lib/libexecutorch.so` (SONAME-versioned, with the standard `libexecutorch.so -> .so.1 -> .so.<version>` chain): the consolidated shared runtime. It bundles the runtime core plus the common runtime extensions (module, tensor, data_loader, flat_tensor, named_data_map). - `executorch/include/executorch/extension/...`: the public headers for the Module, Tensor, DataLoader, FlatTensor (.ptd reader), NamedDataMap, and header-only MallocMemoryAllocator APIs. Runtime/Program/backend headers were already shipped and are reused. - `executorch/share/cmake/executorch-config.cmake`: a CMake package config that exposes an `executorch::runtime` imported target (plus convenience aliases `executorch::core`, `executorch::extension_*`). - `executorch/utils/cmake_prefix_path`: a small helper (mirrors `torch.utils.cmake_prefix_path`) so CMake can find the config in one line. ## Why a shared library (not static archives) The runtime is shipped shared on purpose. ExecuTorch keeps a single process-global backend/kernel registry. Shipping the runtime as one shared `libexecutorch.so` lets a separately distributed backend or delegate shared library register into that one registry: the backend `.so` is built without its own copy of the runtime (its `register_backend` reference is undefined and resolves against `libexecutorch.so` at load), and its static-init registration runs when the `.so` is loaded (via whole-archive for a C++ app, or an explicit import/dlopen for Python, which is how ExecuTorch already ships the QNN backend today). Static archives would give each consumer its own private registry, which cannot support loading multiple independently distributed backends into one runtime. The set is intentionally libtorch-free and excludes the general CPU operator/kernel libraries, because a delegate supplies its own compute. It is also Linux only: the `.so` naming and SONAME symlink chain are Unix specific, so the Windows and macOS wheels are byte-identical to before. ## How to use it ```bash pip install executorch ``` ```cmake find_package(executorch CONFIG REQUIRED) add_executable(my_runner main.cpp) target_link_libraries(my_runner PRIVATE executorch::runtime) ``` ```bash cmake -S . -B build \ -DCMAKE_PREFIX_PATH="$(python -c 'import executorch.utils as u; print(u.cmake_prefix_path)')" cmake --build build ``` Existing consumers that use `find_package(executorch)` to link the Python `_portable_lib` for custom-op extensions keep working unchanged. The new C++ SDK availability is reported separately via `EXECUTORCH_SDK_FOUND`, so the legacy `EXECUTORCH_FOUND` / `EXECUTORCH_LIBRARIES` contract is preserved. ## Test plan Verified on Linux x86_64: - Built the wheel with `python setup.py bdist_wheel` and confirmed it contains `libexecutorch.so` with its SONAME symlink chain, the CMake config, the `utils` helper, and the new extension headers, and that headers with no shipped implementation are excluded. - Installed the wheel into a clean virtual environment and built a small standalone C++ runner against it with `find_package(executorch)` and `executorch.utils.cmake_prefix_path`. The runner compiled, linked, ran, and initialized the runtime. - Built a separate "coreless" backend shared library (no bundled runtime, `register_backend` left undefined) and confirmed that loading it against the installed `libexecutorch.so` registers the backend into the runtime's registry (`get_backend_class` goes from not-found to found). This is the mechanism that lets independently distributed backends coalesce into one runtime. - Confirmed the runner and the runtime link no libtorch/libc10 (via `ldd`). - Confirmed the Windows and macOS wheel code paths add nothing new, so those wheels are unaffected. - Lint and format pass (flake8, ufmt, cmake-format). Known limitation: the wheel-build step stores the SONAME symlinks as plain file copies rather than symlinks. Linking and loading still work because the SONAME target is present as a real file; a follow-up can preserve them as true symlinks to save space. ## CI Adds a PR job (test-cpp-sdk-wheel-linux in pull.yml, calling .ci/scripts/test_cpp_sdk_wheel.sh) that builds the wheel, installs it into a clean venv, and uses find_package(executorch) to build a C++ consumer linking executorch::runtime, then loads a coreless backend shared library and asserts it registers into libexecutorch.so. This gives the feature direct coverage: before, no PR job built the full wheel or linked the C++ SDK.
1 parent 0b13b6a commit 1bc3075

5 files changed

Lines changed: 543 additions & 21 deletions

File tree

‎.ci/scripts/test_cpp_sdk_wheel.sh‎

Lines changed: 226 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,226 @@
1+
#!/bin/bash
2+
# Copyright (c) Meta Platforms, Inc. and affiliates.
3+
# All rights reserved.
4+
#
5+
# This source code is licensed under the BSD-style license found in the
6+
# LICENSE file in the root directory of this source tree.
7+
8+
# Builds the Linux wheel with the prebuilt C++ SDK, installs it into a clean
9+
# environment, and builds a standalone C++ program against it via
10+
# find_package(executorch). This is the signal that the shipped
11+
# libexecutorch.so, headers, and CMake package config actually let a C++
12+
# consumer link the ExecuTorch runtime with no source checkout, and that a
13+
# separately built "coreless" backend shared library can register into the one
14+
# runtime registry (the mechanism coalesced multi-backend execution relies on).
15+
16+
set -euxo pipefail
17+
18+
PYTHON_EXECUTABLE="${PYTHON_EXECUTABLE:-python}"
19+
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
20+
BUILD_VENV="${REPO_ROOT}/.venv-sdk-build"
21+
TEST_VENV="${REPO_ROOT}/.venv-sdk-test"
22+
WORK_DIR="${REPO_ROOT}/cpp_sdk_consumer"
23+
24+
rm -rf "${BUILD_VENV}" "${TEST_VENV}" "${WORK_DIR}" "${REPO_ROOT}/dist" \
25+
"${REPO_ROOT}/pip-out"
26+
27+
# ---------------------------------------------------------------------------
28+
# Build the wheel.
29+
# ---------------------------------------------------------------------------
30+
"${PYTHON_EXECUTABLE}" -m venv "${BUILD_VENV}"
31+
# shellcheck source=/dev/null
32+
source "${BUILD_VENV}/bin/activate"
33+
python -m pip install --upgrade pip
34+
python -m pip install \
35+
"cmake>=3.24,<4.0.0" \
36+
"numpy>=2.0.0" \
37+
packaging \
38+
pyyaml \
39+
setuptools \
40+
wheel \
41+
zstd \
42+
certifi \
43+
torch \
44+
torchvision \
45+
--index-url https://download.pytorch.org/whl/cpu \
46+
--extra-index-url https://pypi.org/simple
47+
48+
(
49+
cd "${REPO_ROOT}"
50+
python setup.py bdist_wheel
51+
)
52+
53+
WHEEL_FILE="$(find "${REPO_ROOT}/dist" -maxdepth 1 -name 'executorch-*.whl' | head -1)"
54+
test -n "${WHEEL_FILE}"
55+
56+
# ---------------------------------------------------------------------------
57+
# Verify the SDK payload is present in the wheel.
58+
# ---------------------------------------------------------------------------
59+
python - "${WHEEL_FILE}" <<'PY'
60+
import sys
61+
import zipfile
62+
63+
wheel_file = sys.argv[1]
64+
with zipfile.ZipFile(wheel_file) as wheel:
65+
names = set(wheel.namelist())
66+
67+
required = [
68+
"executorch/lib/libexecutorch.so",
69+
"executorch/share/cmake/executorch-config.cmake",
70+
"executorch/utils/__init__.py",
71+
"executorch/include/executorch/runtime/executor/program.h",
72+
"executorch/include/executorch/extension/module/module.h",
73+
]
74+
missing = [name for name in required if name not in names]
75+
if missing:
76+
raise AssertionError(f"{wheel_file} is missing SDK files: {missing}")
77+
78+
# Headers whose implementation is not shipped must not be advertised.
79+
forbidden = [
80+
"executorch/include/executorch/extension/module/bundled_module.h",
81+
"executorch/include/executorch/extension/flat_tensor/serialize/serialize.h",
82+
]
83+
present = [name for name in forbidden if name in names]
84+
if present:
85+
raise AssertionError(f"{wheel_file} advertises unshipped APIs: {present}")
86+
87+
print("SDK payload OK")
88+
PY
89+
90+
deactivate
91+
92+
# ---------------------------------------------------------------------------
93+
# Install into a clean environment and link a C++ consumer against it.
94+
# ---------------------------------------------------------------------------
95+
"${PYTHON_EXECUTABLE}" -m venv "${TEST_VENV}"
96+
# shellcheck source=/dev/null
97+
source "${TEST_VENV}/bin/activate"
98+
python -m pip install --upgrade pip
99+
python -m pip install "cmake>=3.24,<4.0.0"
100+
# --no-deps: the C++ SDK link test does not need torch, proving the runtime is
101+
# linkable standalone. (A plain install pulls declared deps; not needed here.)
102+
python -m pip install --no-deps "${WHEEL_FILE}"
103+
104+
CMAKE_PREFIX_PATH="$(python -c 'import executorch.utils as u; print(u.cmake_prefix_path)')"
105+
export CMAKE_PREFIX_PATH
106+
107+
mkdir -p "${WORK_DIR}"
108+
cd "${WORK_DIR}"
109+
110+
cat > main.cpp <<'CPP'
111+
#include <cstdio>
112+
#include <executorch/runtime/backend/interface.h>
113+
#include <executorch/runtime/platform/runtime.h>
114+
115+
using executorch::runtime::get_backend_class;
116+
using executorch::runtime::get_num_registered_backends;
117+
using executorch::runtime::runtime_init;
118+
119+
int main() {
120+
runtime_init();
121+
printf("registered backends: %zu\n", get_num_registered_backends());
122+
// The delegate-only SDK ships no backends, so this must be null. What matters
123+
// is that the symbol links and resolves from libexecutorch.so.
124+
printf("stock backend lookup resolves: %d\n",
125+
get_backend_class("NonexistentBackend") == nullptr);
126+
printf("PASS: linked executorch::runtime from the wheel\n");
127+
return 0;
128+
}
129+
CPP
130+
131+
cat > CMakeLists.txt <<'CMAKE'
132+
cmake_minimum_required(VERSION 3.24)
133+
project(executorch_cpp_sdk_consumer LANGUAGES CXX)
134+
set(CMAKE_CXX_STANDARD 17)
135+
find_package(executorch CONFIG REQUIRED)
136+
if(NOT EXECUTORCH_SDK_FOUND)
137+
message(FATAL_ERROR "EXECUTORCH_SDK_FOUND is false; the wheel C++ SDK is missing")
138+
endif()
139+
add_executable(consumer main.cpp)
140+
target_link_libraries(consumer PRIVATE executorch::runtime)
141+
CMAKE
142+
143+
cmake -S . -B build
144+
cmake --build build
145+
./build/consumer
146+
147+
# The runtime and the consumer must not pull in libtorch.
148+
if ldd ./build/consumer | grep -Eiq "libtorch|libc10"; then
149+
echo "ERROR: consumer unexpectedly links libtorch/libc10" >&2
150+
exit 1
151+
fi
152+
153+
# ---------------------------------------------------------------------------
154+
# Prove cross-shared-object registration: a "coreless" backend .so (no bundled
155+
# runtime, register_backend undefined) registers into the runtime inside
156+
# libexecutorch.so when loaded. This is the mechanism coalesced multi-backend
157+
# execution depends on.
158+
# ---------------------------------------------------------------------------
159+
cat > mybackend.cpp <<'CPP'
160+
#include <executorch/runtime/backend/interface.h>
161+
using namespace executorch::runtime;
162+
namespace {
163+
struct MyBackend final : public BackendInterface {
164+
bool is_available() const override { return true; }
165+
Result<DelegateHandle*> init(BackendInitContext&, FreeableBuffer*,
166+
ArrayRef<CompileSpec>) const override {
167+
return nullptr;
168+
}
169+
Error execute(BackendExecutionContext&, DelegateHandle*,
170+
Span<EValue*>) const override {
171+
return Error::Ok;
172+
}
173+
void destroy(DelegateHandle*) const override {}
174+
};
175+
MyBackend g_backend;
176+
Backend g_id{"MyTestBackend", &g_backend};
177+
static auto g_registered = register_backend(g_id);
178+
} // namespace
179+
CPP
180+
181+
cat >> CMakeLists.txt <<'CMAKE'
182+
# Coreless: undefined ExecuTorch symbols resolve from libexecutorch.so at load.
183+
add_library(mybackend SHARED mybackend.cpp)
184+
target_link_options(mybackend PRIVATE "LINKER:--unresolved-symbols=ignore-all")
185+
target_include_directories(mybackend PRIVATE
186+
$<TARGET_PROPERTY:executorch::runtime,INTERFACE_INCLUDE_DIRECTORIES>)
187+
target_compile_definitions(mybackend PRIVATE C10_USING_CUSTOM_GENERATED_MACROS)
188+
189+
add_executable(reg_consumer reg_main.cpp)
190+
target_link_libraries(reg_consumer PRIVATE executorch::runtime ${CMAKE_DL_LIBS})
191+
CMAKE
192+
193+
cat > reg_main.cpp <<'CPP'
194+
#include <cstdio>
195+
#include <dlfcn.h>
196+
#include <executorch/runtime/backend/interface.h>
197+
#include <executorch/runtime/platform/runtime.h>
198+
199+
using executorch::runtime::get_backend_class;
200+
using executorch::runtime::runtime_init;
201+
202+
int main(int argc, char** argv) {
203+
runtime_init();
204+
if (get_backend_class("MyTestBackend") != nullptr) {
205+
fprintf(stderr, "backend registered before load\n");
206+
return 1;
207+
}
208+
void* handle = dlopen(argv[1], RTLD_NOW | RTLD_GLOBAL);
209+
if (handle == nullptr) {
210+
fprintf(stderr, "dlopen failed: %s\n", dlerror());
211+
return 1;
212+
}
213+
if (get_backend_class("MyTestBackend") == nullptr) {
214+
fprintf(stderr, "backend NOT registered after load\n");
215+
return 1;
216+
}
217+
printf("PASS: coreless backend .so registered into libexecutorch.so\n");
218+
return 0;
219+
}
220+
CPP
221+
222+
cmake -S . -B build
223+
cmake --build build
224+
./build/reg_consumer "$(find build -name 'libmybackend.so' | head -1)"
225+
226+
echo "ALL C++ SDK WHEEL CHECKS PASSED"

‎.github/workflows/pull.yml‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,29 @@ jobs:
116116
# Build and test ExecuTorch with the add model on portable backend.
117117
PYTHON_EXECUTABLE=python bash .ci/scripts/test_model.sh "add" "${BUILD_TOOL}" "portable"
118118
119+
test-cpp-sdk-wheel-linux:
120+
name: test-cpp-sdk-wheel-linux
121+
uses: pytorch/test-infra/.github/workflows/linux_job_v2.yml@main
122+
permissions:
123+
id-token: write
124+
contents: read
125+
strategy:
126+
fail-fast: false
127+
with:
128+
runner: linux.2xlarge
129+
docker-image: ci-image:executorch-ubuntu-22.04-gcc11
130+
submodules: 'recursive'
131+
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
132+
timeout: 90
133+
script: |
134+
# The generic Linux job chooses to use base env, not the one setup by the image
135+
CONDA_ENV=$(conda env list --json | jq -r ".envs | .[-1]")
136+
conda activate "${CONDA_ENV}"
137+
138+
# Build the wheel, install it clean, and link a C++ consumer against the
139+
# shipped libexecutorch.so via find_package(executorch).
140+
PYTHON_EXECUTABLE=python bash .ci/scripts/test_cpp_sdk_wheel.sh
141+
119142
test-models-linux-basic:
120143
name: test-models-linux-basic
121144
uses: pytorch/test-infra/.github/workflows/linux_job_v2.yml@main

0 commit comments

Comments
 (0)