[EP ABI] Add CreateCustomOpDomains() API for plugin EP to register custom ops #27050
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ection before model load
Contributor
Author
|
This PR is identical to #26759, as for some reasons i can't find the unresolved comment in the conversation and resolve it, it's blocking that PR to be merged. |
adrianlizarraga
approved these changes
Jan 18, 2026
alex-spacemit
pushed a commit
to spacemit-com/onnxruntime
that referenced
this pull request
Jan 20, 2026
…stom ops (microsoft#27050) ### Description The newly added two APIs, `CreateCustomOpDomains()` and `GetNumCustomOpDomains`, are used when running inference on a model that contains EP-specific custom operations. Workflow: 1. The EP implements these functions to supply a list of `OrtCustomOpDomain` instances. 2. The application either 1) calls `SessionOptionsAppendExecutionProvider_V2()` with an `OrtEpDevice` containing the plugin EP's factory or 2) enables auto ep selection. 3. Then ORT either 1) `SessionOptionsAppendExecutionProvider_V2()` appends the provided OrtCustomOpDomains to the session options or 2) registers the OrtCustomOpDomains from the selected EP devices. As a result, any session created from these session options will have these custom op domains registered in ORT, ensuring that the custom ops are properly recognized and validated when the model is loaded. Plugin EPs can provide two types of custom ops: 1. A full OrtCustomOp with a concrete kernel implementation - This Example EP demonstrates this approach. - In GetCapability(), it calls EpGraphSupportInfo_AddSingleNode() to inform ORT that the custom node should NOT be fused or compiled. Instead, ORT should invoke the custom node's Compute() function at runtime. 2. A "placeholder" OrtCustomOp with an empty kernel implementation - A compile-based Plugin EP can supply an OrtCustomOp whose CustomKernel::Compute() does nothing. The purpose is to satisfy model validation during model loading by registering the custom op as a valid operator in the session. - In GetCapability(), the EP should call EpGraphSupportInfo_AddNodesToFuse() to notify ORT that this custom node should be fused and compiled by the EP. - In Compile(), the EP executes its compiled bits to perform inference for the fused custom node. ### Motivation and Context Currently, the provider-bridge TRT RTX EP and TRT EP supports registering custom op domain list in session option so that it can run model contains TRT specific custom ops. This PR adds the same feature for plugin EP.
edgchen1
reviewed
Jan 20, 2026
| ORT_API_RETURN_IF_STATUS_NOT_OK(GetCustomOpDomainsFromEpDevice(*ep_device, domains)); | ||
|
|
||
| for (auto domain : domains) { | ||
| if (ShouldAddDomain(domain, options->custom_op_domains_)) { |
Contributor
There was a problem hiding this comment.
do we want to check the domains in all_ep_custom_op_domains here too?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
The newly added two APIs,
CreateCustomOpDomains()andGetNumCustomOpDomains, are used when running inference on a model that contains EP-specific custom operations.Workflow:
OrtCustomOpDomaininstances.SessionOptionsAppendExecutionProvider_V2()with anOrtEpDevicecontainingthe plugin EP's factory or 2) enables auto ep selection.
SessionOptionsAppendExecutionProvider_V2()appends the provided OrtCustomOpDomains to thesession options or 2) registers the OrtCustomOpDomains from the selected EP devices.
As a result, any session created from these session options will have these custom op domains registered
in ORT, ensuring that the custom ops are properly recognized and validated when the model is loaded.
Plugin EPs can provide two types of custom ops:
A full OrtCustomOp with a concrete kernel implementation
that the custom node should NOT be fused or compiled. Instead, ORT should invoke
the custom node's Compute() function at runtime.
A "placeholder" OrtCustomOp with an empty kernel implementation
does nothing. The purpose is to satisfy model validation during model loading by
registering the custom op as a valid operator in the session.
notify ORT that this custom node should be fused and compiled by the EP.
the fused custom node.
Motivation and Context
Currently, the provider-bridge TRT RTX EP and TRT EP supports registering custom op domain list in session option so
that it can run model contains TRT specific custom ops.
This PR adds the same feature for plugin EP.