[RFC] First-class support and reference implementation for out-of-tree backends

[joozhan]

🚀 The feature, motivation and pitch

The feature, motivation and pitch

We are integrating a new hardware accelerator with ExecuTorch and would like to maintain the integration in an independent repository, for example:

vendor-executorch-backend

The repository would contain the complete vendor-specific integration, including:

• The Python AOT components, such as the Partitioner, backend preprocessing, graph passes, support checks, and optional quantization components.
• The C++ runtime backend implementation and registration.
• Integration with the vendor compiler and runtime SDK.
• Build scripts, packaging, tests, examples, and release management.

Our goal is to avoid:

• Maintaining a fork of ExecuTorch.
• Modifying the ExecuTorch source tree.
• Requiring all vendor-specific code to be upstreamed into executorch/backends/.
• Coupling the vendor backend release cycle directly to the ExecuTorch repository release cycle.

The currently documented backend integration workflow appears to focus on in-tree backends:

• Backend sources are placed under executorch/backends/<delegate_name>/.
• The backend is added to ExecuTorch's top-level CMake build.
• An EXECUTORCH_BUILD_<DELEGATE_NAME> option is added.
• Python AOT components are installed as part of the top-level ExecuTorch package.

It is currently unclear whether a fully out-of-tree backend is an intended and officially supported integration model, or whether it only happens to be technically possible through the existing interfaces.

This issue is primarily intended to clarify the architectural direction and the support contract for independently maintained hardware backends.

Desired integration model

We would like to maintain a repository with a structure similar to:

vendor-executorch-backend/
├── python/
│   └── vendor_executorch_backend/
│       ├── partitioner.py
│       ├── backend.py
│       ├── quantizer.py
│       └── passes/
├── runtime/
│   ├── vendor_backend.cpp
│   ├── vendor_backend.h
│   └── CMakeLists.txt
├── examples/
├── tests/
├── pyproject.toml
└── README.md

The package would depend on a released version of ExecuTorch instead of being included in the ExecuTorch source tree.

Expected Python/AOT user experience

Ideally, users could install both packages independently:

pip install executorch vendor-executorch-backend

They could then use the external partitioner in essentially the same way as an in-tree backend:

from executorch.exir import to_edge_transform_and_lower
from vendor_executorch_backend import VendorPartitioner

program = to_edge_transform_and_lower(
    exported_program,
    partitioner=[VendorPartitioner()],
).to_executorch()

An explicit import of the vendor package is reasonable. However, after the package is imported, we would like the export and lowering workflow to behave consistently with an in-tree backend.

Expected runtime user experience

The application would link both the ExecuTorch runtime and the separately built vendor backend library.

Conceptually, the CMake integration could look like:

find_package(ExecuTorch CONFIG REQUIRED)
find_package(VendorExecuTorchBackend CONFIG REQUIRED)

target_link_libraries(
    application
    PRIVATE
    <executorch-runtime-target>
    VendorExecuTorchBackend::Backend
)

The vendor library could either register the backend automatically or expose an explicit registration function called by the application.

After registration, loading and executing the generated .pte file should ideally use the same ExecuTorch runtime APIs as an in-tree backend.

Questions for the ExecuTorch maintainers

Architecture and intended support

1. Was independently maintained, out-of-tree hardware backend integration considered as part of the ExecuTorch backend and delegate architecture?

2. Is a fully out-of-tree backend an officially supported use case today?

3. Is out-of-tree backend integration intended to become a first-class extension model, or are production backends generally expected to be added to the ExecuTorch main repository?

4. Which parts of an external backend can currently remain completely outside the ExecuTorch source tree?

   • Python Partitioner
   • Backend preprocessing
   • Quantizer and graph passes
   • C++ runtime backend
   • Backend registration
   • Platform packaging and build integration

Existing examples

5. Is there an existing open-source repository that demonstrates a complete out-of-tree backend, including both:

   • Python AOT partitioning and preprocessing
   • C++ runtime implementation, linking, and registration?

6. The executorch-examples repository is described as containing out-of-tree demos. Is any example there intended to serve as a reference architecture for an independently distributed hardware backend?

7. If no complete example currently exists, would the community consider providing or accepting a minimal reference repository such as:

executorch-out-of-tree-backend-example

First-class user experience

8. Should an out-of-tree backend be usable through the same export and lowering APIs as an in-tree backend, with the only difference being an additional package installation and import?

9. Is there, or is there a plan for, an official Python discovery or registration mechanism for external backend packages?

For example, could a package register its backend through an explicit API or Python package entry point, rather than depending on internal subclass discovery behavior?

10. What is the recommended C++ registration model for an external backend?

• Static initialization
• An explicit registration function
• Whole-archive linking
• Dynamic loading
• Another mechanism

11. Should an application using an out-of-tree backend be able to use the standard ExecuTorch runtime APIs without backend-specific changes after the backend has been linked and registered?

12. Are selective builds expected to work consistently with externally registered backends?

Compatibility expectations

13. Which backend-facing Python and C++ interfaces are considered public and stable for external implementations?

In particular:

• Partitioner
• DelegationSpec
• Backend preprocessing interfaces
• CompileSpec
• PreprocessResult
• Runtime backend interfaces
• register_backend
• Backend identifiers stored in .pte files

14. Is source compatibility across ExecuTorch releases expected for these interfaces?

15. Is any binary ABI compatibility expected for separately compiled C++ backend libraries, or should vendors rebuild their runtime backend for every supported ExecuTorch version?

16. What versioning model is recommended for an external backend package?

For example:

vendor-executorch-backend 1.x
requires executorch >=1.3,<1.4

17. What compatibility guarantees exist between:

• The ExecuTorch version used during AOT export.
• The external backend package used during AOT export.
• The ExecuTorch runtime version on the target device.
• The external vendor backend runtime library.
• The backend-specific processed binary stored in the .pte file.

18. Is there a recommended mechanism for detecting an incompatible backend/runtime combination and producing a clear error?

Packaging and platform integration

19. What is the recommended distribution model for an out-of-tree backend on:

• Linux
• Android
• iOS
• Windows
• Embedded or bare-metal systems

20. Are installed ExecuTorch headers and exported CMake targets intended to be sufficient for building an external backend without checking out the ExecuTorch source repository?

21. Is there a recommended approach for distributing an external backend together with the ExecuTorch Python wheel, Android libraries, iOS frameworks, or embedded runtime builds?

Requested documentation or reference implementation

A minimal official example would be very useful if it covered the following end-to-end workflow:

1. Implement a simple external Python Partitioner.
2. Implement external backend preprocessing.
3. Generate a .pte file containing a delegated subgraph.
4. Implement the C++ runtime backend in a separate repository.
5. Register the backend without modifying ExecuTorch.
6. Build an application using an installed ExecuTorch SDK or package.
7. Load and execute the .pte through the standard runtime API.
8. Demonstrate version checks and compatibility handling.
9. Demonstrate how static-linker dead stripping should be handled.
10. Document the differences, if any, between in-tree and out-of-tree backends.

This example would help hardware vendors avoid relying on internal APIs or accidentally depending on implementation details that are not intended to be stable.

Relationship to PyTorch PrivateUse1

PyTorch's PrivateUse1 is a useful analogy for the ownership and distribution model, but this issue is not proposing that ExecuTorch copy its dispatcher or device implementation.

The desired similarity is primarily that:

• A hardware vendor can maintain its integration in an independent repository.
• Users can install that integration separately.
• The external implementation uses documented and stable extension interfaces.
• The user-facing workflow is close to that of a backend maintained in the main repository.
• Compatibility expectations are explicitly defined.

ExecuTorch may require a different design because its backend integration spans AOT compilation, graph partitioning, serialization, runtime registration, static linking, and platform packaging.

Related issue

This appears related to #4891, which raised concerns about the lack of clarity around external backends.

The purpose of this issue is to ask more specifically whether an independently distributed backend is an intended first-class integration model and, if so, what reference implementation, user experience, and compatibility contract backend vendors should follow.

We would be interested in contributing documentation or a minimal reference implementation if this direction is aligned with the ExecuTorch architecture and roadmap.

Alternatives

No response

Additional context

No response

RFC (Optional)

No response

cc @cccclai

[psiddh]

cc @JacobSzwejbka @digantdesai