Ship a prebuilt C++ SDK in the ExecuTorch Linux wheel

shoumikhin · shoumikhin · commit 86c35bdd7808 · 2026-07-29T13:34:14.000-07:00
## Why this is needed

Today, using the ExecuTorch runtime from a C++ program means building
ExecuTorch from source: clone the repo, sync submodules, and run a CMake
build before you can compile and link your own runner. The pip wheel only
ships the Python runtime module (`_portable_lib`) plus a small set of
headers meant for authoring custom operators. There is no way to just
`pip install executorch` and link a standalone C++ application against the
runtime.

This is friction for anyone whose deployment path is C++ (the common case
for on-device inference) and who already has the wheel installed for
export. The runtime is compiled during the wheel build and then discarded.

This change ships the runtime as a linkable shared library, its public
headers, and a CMake package config inside the Linux wheel, so a C++
program can link the ExecuTorch runtime with no source checkout and no
separate build.

## What is inside

Added to the Linux wheel (nothing removed; other platforms unchanged):

- `executorch/lib/libexecutorch.so` (SONAME-versioned, with the standard
  `libexecutorch.so -&gt; .so.1 -&gt; .so.&lt;version&gt;` chain): the consolidated
  shared runtime. It bundles the runtime core plus the common runtime
  extensions (module, tensor, data_loader, flat_tensor, named_data_map).
- `executorch/include/executorch/extension/...`: the public headers for the
  Module, Tensor, DataLoader, FlatTensor (.ptd reader), NamedDataMap, and
  header-only MallocMemoryAllocator APIs. Runtime/Program/backend headers
  were already shipped and are reused.
- `executorch/share/cmake/executorch-config.cmake`: a CMake package config
  that exposes an `executorch::runtime` imported target (plus convenience
  aliases `executorch::core`, `executorch::extension_*`).
- `executorch/utils/cmake_prefix_path`: a small helper (mirrors
  `torch.utils.cmake_prefix_path`) so CMake can find the config in one line.

## Why a shared library (not static archives)

The runtime is shipped shared on purpose. ExecuTorch keeps a single
process-global backend/kernel registry. Shipping the runtime as one shared
`libexecutorch.so` lets a separately distributed backend or delegate shared
library register into that one registry: the backend `.so` is built without
its own copy of the runtime (its `register_backend` reference is undefined
and resolves against `libexecutorch.so` at load), and its static-init
registration runs when the `.so` is loaded (via whole-archive for a C++ app,
or an explicit import/dlopen for Python, which is how ExecuTorch already
ships the QNN backend today). Static archives would give each consumer its
own private registry, which cannot support loading multiple independently
distributed backends into one runtime.

The set is intentionally libtorch-free and excludes the general CPU
operator/kernel libraries, because a delegate supplies its own compute. It
is also Linux only: the `.so` naming and SONAME symlink chain are Unix
specific, so the Windows and macOS wheels are byte-identical to before.

## How to use it

```bash
pip install executorch
```

```cmake
find_package(executorch CONFIG REQUIRED)

add_executable(my_runner main.cpp)
target_link_libraries(my_runner PRIVATE executorch::runtime)
```

```bash
cmake -S . -B build \
  -DCMAKE_PREFIX_PATH="$(python -c 'import executorch.utils as u; print(u.cmake_prefix_path)')"
cmake --build build
```

Existing consumers that use `find_package(executorch)` to link the Python
`_portable_lib` for custom-op extensions keep working unchanged. The new
C++ SDK availability is reported separately via `EXECUTORCH_SDK_FOUND`, so
the legacy `EXECUTORCH_FOUND` / `EXECUTORCH_LIBRARIES` contract is
preserved.

## Test plan

Verified on Linux x86_64:

- Built the wheel with `python setup.py bdist_wheel` and confirmed it
  contains `libexecutorch.so` with its SONAME symlink chain, the CMake
  config, the `utils` helper, and the new extension headers, and that
  headers with no shipped implementation are excluded.
- Installed the wheel into a clean virtual environment and built a small
  standalone C++ runner against it with `find_package(executorch)` and
  `executorch.utils.cmake_prefix_path`. The runner compiled, linked, ran,
  and initialized the runtime.
- Built a separate "coreless" backend shared library (no bundled runtime,
  `register_backend` left undefined) and confirmed that loading it against
  the installed `libexecutorch.so` registers the backend into the runtime's
  registry (`get_backend_class` goes from not-found to found). This is the
  mechanism that lets independently distributed backends coalesce into one
  runtime.
- Confirmed the runner and the runtime link no libtorch/libc10 (via `ldd`).
- Confirmed the Windows and macOS wheel code paths add nothing new, so those
  wheels are unaffected.
- Lint and format pass (flake8, ufmt, cmake-format).

Known limitation: the wheel-build step stores the SONAME symlinks as plain
file copies rather than symlinks. Linking and loading still work because the
SONAME target is present as a real file; a follow-up can preserve them as
true symlinks to save space.
diff --git a/setup.py b/setup.py
@@ -320,6 +320,27 @@ def get_dynamic_lib_name(name: str) -> str:
         return f"lib{name}.so"
 
 
+def _read_soname(lib_path: Path) -> Optional[str]:
+    """Return the ELF SONAME of a shared library, or None if unavailable.
+
+    Used to recreate the SONAME symlink for a versioned .so in the wheel. Best
+    effort: any failure (non-ELF, no readelf, non-Linux) returns None.
+    """
+    try:
+        out = subprocess.run(
+            ["readelf", "-d", os.fspath(lib_path)],
+            capture_output=True,
+            text=True,
+            check=True,
+        ).stdout
+    except Exception:
+        return None
+    for line in out.splitlines():
+        if "SONAME" in line and "[" in line and "]" in line:
+            return line[line.index("[") + 1 : line.index("]")]
+    return None
+
+
 def get_executable_name(name: str) -> str:
     if _is_windows():
         return name + ".exe"
@@ -508,6 +529,43 @@ def inplace_dir(self, installer: "InstallerBuildExt") -> Path:
         return Path(package_dir)
 
 
+class BuiltSharedLib(BuiltFile):
+    """Installs a SONAME-versioned shared library plus its symlink chain.
+
+    A normal ``BuiltFile`` copies one file. A shared library like
+    ``libexecutorch.so.1.4.0`` also needs the loader-visible SONAME symlink
+    (``libexecutorch.so.1``) and the developer symlink (``libexecutorch.so``),
+    or a consumer that links ``-lexecutorch`` fails at runtime because the
+    SONAME recorded in dependents cannot be found. This recreates that chain in
+    the wheel, matching a standard ``cmake --install`` layout.
+    """
+
+    def __init__(self, src_dir: str, src_name: str, dst: str):
+        # src_name is the base library name (e.g. "executorch"); the real file
+        # is libexecutorch.so.<version>, resolved by glob in src_path().
+        super().__init__(
+            src_dir=src_dir,
+            src_name=f"lib{src_name}.so.*",
+            dst=dst,
+            dependent_cmake_flags=[],
+        )
+
+    def src_path(self, installer: "InstallerBuildExt") -> Path:
+        # The glob matches the versioned real file and any symlinks; pick the
+        # regular file (the real library), not the symlinks.
+        build_dir = self._get_build_dir(installer)
+        pattern = self.src.replace("%CMAKE_CACHE_DIR%/", "")
+        matches = [
+            p for p in build_dir.glob(pattern) if p.is_file() and not p.is_symlink()
+        ]
+        if len(matches) != 1:
+            raise ValueError(
+                f"Expecting exactly 1 real shared library matching {self.src} "
+                f"in {build_dir}, found {matches}."
+            )
+        return matches[0]
+
+
 class BuiltExtension(_BaseExtension):
     """An extension that installs a python extension that was built by cmake."""
 
@@ -663,6 +721,31 @@ def build_extension(self, ext: _BaseExtension) -> None:
         # Copy the file.
         self.copy_file(os.fspath(src_file), os.fspath(dst_file))
 
+        # For a SONAME-versioned shared library, also recreate the symlink chain
+        # (libexecutorch.so -> libexecutorch.so.<major> -> libexecutorch.so.<ver>)
+        # so `-lexecutorch` links and the SONAME resolves at load time.
+        if isinstance(ext, BuiltSharedLib):
+            self._create_soname_symlinks(src_file, dst_file)
+
+    def _create_soname_symlinks(self, src_file: Path, dst_file: Path) -> None:
+        real_name = dst_file.name  # e.g. libexecutorch.so.1.4.0
+        link_dir = dst_file.parent
+        # SONAME (e.g. libexecutorch.so.1) read from the built library; fall back
+        # to none if unreadable. The dev symlink drops all version suffixes.
+        links = set()
+        soname = _read_soname(src_file)
+        if soname and soname != real_name:
+            links.add(soname)
+        dev_name = real_name.split(".so")[0] + ".so"
+        if dev_name != real_name:
+            links.add(dev_name)
+        for link_name in links:
+            link_path = link_dir / link_name
+            if link_path.exists() or link_path.is_symlink():
+                link_path.unlink()
+            # Relative link so the wheel is relocatable.
+            os.symlink(real_name, os.fspath(link_path))
+
         # Ensure that the destination file is writable, even if the source was
         # not. build_py does this by passing preserve_mode=False to copy_file,
         # but that would clobber the X bit on any executables. TODO(dbort): This
@@ -749,6 +832,38 @@ def run(self):
                     src_to_dst.append(
                         (str(src), os.path.join("include/executorch", str(src)))
                     )
+            # Delegate-only C++ SDK headers: the Program/Module/Tensor/DataLoader/
+            # .ptd APIs a standalone C++ runner needs, so it can link the prebuilt
+            # runtime without an ExecuTorch source tree. These are listed
+            # explicitly (not rglob) so we only advertise APIs whose implementation
+            # archive is actually shipped in executorch/lib/. Excluded on purpose:
+            # bundled_module.h (needs the separate bundled_module archive),
+            # flat_tensor/serialize/serialize.h (serializer .cpp not shipped),
+            # file_descriptor_data_loader.h (impl not in the shipped archive), and
+            # cpu_caching_malloc_allocator.h (needs a memory_allocator archive we
+            # do not ship). flat_tensor_header.h IS shipped: it is the .ptd reader.
+            # Linux only, matching the SDK archives below; keeps Windows/macOS
+            # wheels unchanged.
+            sdk_headers = (
+                [
+                    "extension/module/module.h",
+                    "extension/data_loader/buffer_data_loader.h",
+                    "extension/data_loader/file_data_loader.h",
+                    "extension/data_loader/mmap_data_loader.h",
+                    "extension/data_loader/mman.h",
+                    "extension/data_loader/mman_windows.h",
+                    "extension/data_loader/shared_ptr_data_loader.h",
+                    "extension/flat_tensor/flat_tensor_data_map.h",
+                    "extension/flat_tensor/serialize/flat_tensor_header.h",
+                    "extension/named_data_map/merged_data_map.h",
+                    "extension/memory_allocator/malloc_memory_allocator.h",
+                    "extension/memory_allocator/memory_allocator_utils.h",
+                ]
+                if sys.platform == "linux"
+                else []
+            )
+            for src in sdk_headers:
+                src_to_dst.append((src, os.path.join("include/executorch", src)))
         for src, dst in src_to_dst:
             dst = os.path.join(dst_root, dst)
 
@@ -900,6 +1015,12 @@ def run(self):  # noqa C901
         ):
             cmake_configuration_args += ["-DEXECUTORCH_BUILD_OPENVINO=ON"]
 
+        # Build the consolidated shared runtime libexecutorch.so so the wheel can
+        # ship a linkable C++ SDK (see the executorch_shared build target and the
+        # packaging step). Linux only; the SDK is not shipped on Windows/macOS.
+        if not minimal_build and sys.platform == "linux":
+            cmake_configuration_args += ["-DEXECUTORCH_BUILD_SHARED=ON"]
+
         with Buck2EnvironmentFixer():
             # Generate the cmake cache from scratch to ensure that the cache state
             # is predictable.
@@ -954,6 +1075,16 @@ def run(self):  # noqa C901
             # list explicitly rather than relying on each flag being OFF.
             cmake_build_args += ["--target", "flatbuffers_ep"]
         else:
+            # Delegate-only C++ SDK: ship the consolidated shared runtime
+            # libexecutorch.so (the executorch_shared target), which bundles the
+            # runtime core plus the common extensions (module, tensor,
+            # data_loader, flat_tensor, named_data_map). Shared is required so a
+            # separately distributed backend/delegate .so can register into the
+            # one process-global registry inside libexecutorch.so. Build it
+            # explicitly, Linux only, so packaging below always finds it.
+            if sys.platform == "linux":
+                cmake_build_args += ["--target", "executorch_shared"]
+
             if cmake_cache.is_enabled("EXECUTORCH_BUILD_PYBIND"):
                 cmake_build_args += ["--target", "portable_lib"]
                 cmake_build_args += ["--target", "data_loader"]
@@ -1125,6 +1256,28 @@ def run(self):  # noqa C901
                     modpath="executorch.backends.qualcomm.python.PyQnnManagerAdaptor",
                     dependent_cmake_flags=["EXECUTORCH_BUILD_QNN"],
                 ),
+                # Delegate-only C++ SDK: the consolidated shared runtime
+                # libexecutorch.so (core + module/tensor/data_loader/flat_tensor/
+                # named_data_map), plus its SONAME symlink chain. Shipping it
+                # shared (not static archives) lets a separately distributed
+                # backend/delegate .so register into the one process-global
+                # registry inside libexecutorch.so, which is what coalesced
+                # multi-backend .pte execution requires. Paired with
+                # executorch-config.cmake (executorch::runtime target) and
+                # executorch.utils.cmake_prefix_path. Linux only: the .so naming
+                # and symlink chain are Unix specific, so Windows/macOS wheels are
+                # unchanged.
+                *(
+                    [
+                        BuiltSharedLib(
+                            src_dir="%CMAKE_CACHE_DIR%/",
+                            src_name="executorch",
+                            dst="executorch/lib/",
+                        )
+                    ]
+                    if sys.platform == "linux"
+                    else []
+                ),
             ]
         ),
     ],
diff --git a/src/executorch/utils/__init__.py b/src/executorch/utils/__init__.py
@@ -0,0 +1,23 @@
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+"""Utilities for locating ExecuTorch's packaged assets.
+
+``cmake_prefix_path`` points at the directory that contains the installed
+ExecuTorch CMake package config, so a C++ project can discover it with:
+
+    cmake -DCMAKE_PREFIX_PATH="$(python -c 'import executorch.utils as u; print(u.cmake_prefix_path)')"
+"""
+
+import os as _os
+
+# Mirror torch.utils.cmake_prefix_path: <package_root>/share/cmake. This file
+# lives at <package_root>/utils/__init__.py, so go up one level.
+cmake_prefix_path = _os.path.join(
+    _os.path.dirname(_os.path.dirname(__file__)), "share", "cmake"
+)
+
+__all__ = ["cmake_prefix_path"]
diff --git a/tools/cmake/executorch-wheel-config.cmake b/tools/cmake/executorch-wheel-config.cmake
@@ -19,10 +19,24 @@
 # EXECUTORCH_INCLUDE_DIRS -- The include directories for ExecuTorch
 # EXECUTORCH_LIBRARIES    -- Libraries to link against
 #
-cmake_minimum_required(VERSION 3.19)
+# In addition to the legacy variables above, this config defines namespaced
+# imported targets for the prebuilt delegate-only C++ SDK when the corresponding
+# static libraries are shipped in the wheel (see the "C++ SDK targets" section
+# below):
+#
+# executorch::core                 -- executorch_core (runtime, no ops)
+# executorch::runtime              -- executorch (adds primitive ops)
+# executorch::extension_data_loader executorch::extension_flat_tensor
+# executorch::extension_named_data_map executorch::extension_tensor
+# executorch::extension_module
+#
+cmake_minimum_required(VERSION 3.24)
 
-# Find prebuilt _portable_lib.<EXT_SUFFIX>.so. This file should be installed
-# under <site-packages>/executorch/share/cmake
+# ---------------------------------------------------------------------------
+# Legacy: discover the CPython _portable_lib extension for custom-op authors.
+# This keeps `find_package(executorch)` working for prebuilt custom-op
+# extensions that link the Python runtime module, unchanged from before.
+# ---------------------------------------------------------------------------
 
 # Find python
 if(DEFINED ENV{CONDA_DEFAULT_ENV} AND NOT $ENV{CONDA_DEFAULT_ENV} STREQUAL
@@ -43,36 +57,119 @@ execute_process(
   OUTPUT_STRIP_TRAILING_WHITESPACE
 )
 
+set(EXECUTORCH_INCLUDE_DIRS
+    "${CMAKE_CURRENT_LIST_DIR}/../../include"
+    "${CMAKE_CURRENT_LIST_DIR}/../../include/executorch/runtime/core/portable_type/c10"
+)
+set(EXECUTORCH_LIBRARIES)
+set(EXECUTORCH_FOUND OFF)
+
+# Only discover the portable Python module when we could read EXT_SUFFIX;
+# probing with an empty suffix would match a wrong/generic file. A missing
+# suffix is not fatal because a pure-C++ consumer (see the C++ SDK section
+# below) does not need the Python extension at all.
 if(SYSCONFIG_RESULT EQUAL 0)
   message(STATUS "Sysconfig extension suffix: ${EXT_SUFFIX}")
+  find_library(
+    _portable_lib_LIBRARY
+    NAMES _portable_lib${EXT_SUFFIX}
+    PATHS "${CMAKE_CURRENT_LIST_DIR}/../../extension/pybindings/"
+  )
 else()
   message(
-    FATAL_ERROR
-      "Failed to retrieve sysconfig config var EXT_SUFFIX: ${SYSCONFIG_ERROR}"
+    WARNING
+      "Failed to retrieve sysconfig config var EXT_SUFFIX: ${SYSCONFIG_ERROR}. "
+      "The _portable_lib Python runtime target will not be available; the C++ "
+      "SDK targets (executorch::*) are unaffected."
   )
 endif()
 
-find_library(
-  _portable_lib_LIBRARY
-  NAMES _portable_lib${EXT_SUFFIX}
-  PATHS "${CMAKE_CURRENT_LIST_DIR}/../../extension/pybindings/"
-)
-
-set(EXECUTORCH_LIBRARIES)
-set(EXECUTORCH_FOUND OFF)
 if(_portable_lib_LIBRARY)
   set(EXECUTORCH_FOUND ON)
   message(
     STATUS "ExecuTorch portable library is found at ${_portable_lib_LIBRARY}"
   )
   list(APPEND EXECUTORCH_LIBRARIES _portable_lib)
-  add_library(_portable_lib STATIC IMPORTED)
-  set(EXECUTORCH_INCLUDE_DIRS ${CMAKE_CURRENT_LIST_DIR}/../../include)
-  # PyTorch requires C++20, so pybindings must be compiled with C++20.
-  set_target_properties(
-    _portable_lib
-    PROPERTIES IMPORTED_LOCATION "${_portable_lib_LIBRARY}"
-               INTERFACE_INCLUDE_DIRECTORIES "${EXECUTORCH_INCLUDE_DIRS}"
-               CXX_STANDARD 20
+  if(NOT TARGET _portable_lib)
+    add_library(_portable_lib STATIC IMPORTED)
+    # PyTorch requires C++20, so pybindings must be compiled with C++20.
+    set_target_properties(
+      _portable_lib
+      PROPERTIES IMPORTED_LOCATION "${_portable_lib_LIBRARY}"
+                 INTERFACE_INCLUDE_DIRECTORIES "${EXECUTORCH_INCLUDE_DIRS}"
+                 CXX_STANDARD 20
+    )
+  endif()
+endif()
+
+# ---------------------------------------------------------------------------
+# C++ SDK targets (delegate-only). Defined only when the prebuilt static
+# archives are present in the wheel (they are shipped alongside this config
+# under ../../lib). This lets a C++ application link the ExecuTorch runtime and
+# the common runtime extensions without an ExecuTorch source checkout:
+#
+# find_package(executorch REQUIRED) target_link_libraries(app PRIVATE
+# executorch::runtime executorch::extension_module executorch::extension_tensor)
+#
+# The set is intentionally libtorch-free and excludes CPU operator/kernel
+# libraries; delegates (e.g. TensorRT, CUDA) supply their own compute. If your
+# model needs portable CPU operators, link a kernel library in addition.
+# ---------------------------------------------------------------------------
+
+get_filename_component(
+  _executorch_sdk_root "${CMAKE_CURRENT_LIST_DIR}/../.." ABSOLUTE
+)
+set(_executorch_sdk_libdir "${_executorch_sdk_root}/lib")
+
+# EXECUTORCH_SDK_FOUND is separate from EXECUTORCH_FOUND on purpose: the legacy
+# EXECUTORCH_FOUND / EXECUTORCH_LIBRARIES contract describes the _portable_lib
+# Python runtime for custom-op authors. Overloading it here would let existing
+# `if(EXECUTORCH_FOUND) link(${EXECUTORCH_LIBRARIES})` code enter its branch
+# with an empty library list. C++ SDK consumers should check the imported target
+# (e.g. `if(TARGET executorch::runtime)`) or EXECUTORCH_SDK_FOUND.
+#
+# The C++ SDK ships one shared library, libexecutorch.so, which bundles the
+# runtime core plus the common runtime extensions (module, tensor, data_loader,
+# flat_tensor, named_data_map). Shared (not static archives) is required so that
+# a separately distributed backend/delegate shared library can register into the
+# one process-global registry that lives in libexecutorch.so. A backend .so is
+# built "coreless" (its register_backend reference is undefined and resolves
+# against libexecutorch.so at load), then force-loaded so its static-init
+# registration runs.
+set(EXECUTORCH_SDK_FOUND OFF)
+find_library(
+  _executorch_shared_LIBRARY
+  NAMES executorch
+  PATHS "${_executorch_sdk_libdir}"
+  NO_DEFAULT_PATH
+)
+if(_executorch_shared_LIBRARY)
+  set(EXECUTORCH_SDK_FOUND ON)
+  if(NOT TARGET executorch::runtime)
+    add_library(executorch::runtime SHARED IMPORTED)
+    set_target_properties(
+      executorch::runtime
+      PROPERTIES IMPORTED_LOCATION "${_executorch_shared_LIBRARY}"
+                 INTERFACE_INCLUDE_DIRECTORIES "${EXECUTORCH_INCLUDE_DIRS}"
+                 INTERFACE_COMPILE_FEATURES cxx_std_17
+                 INTERFACE_COMPILE_DEFINITIONS
+                 "C10_USING_CUSTOM_GENERATED_MACROS"
+    )
+  endif()
+
+  # Convenience aliases. libexecutorch.so already contains the core and these
+  # extensions, so all names resolve to the one shared library. Provided so
+  # consumer CMake can name what it uses without depending on the bundling
+  # layout.
+  foreach(_alias core extension_module extension_tensor extension_data_loader
+                 extension_flat_tensor extension_named_data_map
   )
+    if(NOT TARGET executorch::${_alias})
+      add_library(executorch::${_alias} INTERFACE IMPORTED)
+      set_property(
+        TARGET executorch::${_alias} PROPERTY INTERFACE_LINK_LIBRARIES
+                                              executorch::runtime
+      )
+    endif()
+  endforeach()
 endif()
