-
Notifications
You must be signed in to change notification settings - Fork 3.6k
[EP ABI] Add OrtKernelInfo APIs for kernel-based plugin EPs #26803
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 1 commit
Commits
Show all changes
53 commits
Select commit
Hold shift + click to select a range
4f3d004
[EP ABI] Add weight pre-packing support to kernel-based plugin EPs
adrianlizarraga e954021
Add comment about sharing of prepacked weights (cpu ep only)
adrianlizarraga fb2998b
Update Mul kernel to pre-pack input b
adrianlizarraga 5e64f79
Apply suggestions from code review
adrianlizarraga 9b1c6a2
Add comments regarding prepack allocator lifetime
adrianlizarraga c638a1a
Merge branch 'adrianl/plugin-ep-kernel-prepack' of github.com:microso…
adrianlizarraga 717ed4a
Added support for sharing pre-packed weights for cpu-accessible alloc…
adrianlizarraga bd8f6f0
Define what should happen if OrtKernelImpl::SetSharedPrePackedWeight(…
adrianlizarraga 5c94ec4
Add some missing OrtKernelInfo APIs for plugin EPs
adrianlizarraga fc1fd16
Merge branch 'main' into adrianl/plugin-ep-kernel-prepack
adrianlizarraga 8b3f56c
Clean up some exception handling
adrianlizarraga 23503a1
Refactor example kernel classes (no inheritance)
adrianlizarraga 7f37ffb
Merge branch 'main' into adrianl/plugin-ep-kernel-prepack
adrianlizarraga 26eca56
Correct use of output param
adrianlizarraga 7af257b
Add more edge-case handling for PrePack() call
adrianlizarraga 515062e
API version checks
adrianlizarraga 347ce4f
Use correct SAL annotation for array parameters
adrianlizarraga 906187d
Clean up some includes
adrianlizarraga 56074f6
Merge branch 'main' into adrianl/KernelPluginEp_KernelInfoApis
adrianlizarraga 1611fc3
Update onnxruntime/core/session/plugin_ep/ep_kernel_registration.cc
adrianlizarraga cb75d5a
Rename ExecuteIfCustomOpsApiEnabled to ExecuteifKernelApiEnabled
adrianlizarraga 30ca590
Remove OrtAllocator parameter from SharedPrePackedWeightCache_StoreWe…
adrianlizarraga a5342b9
Clarify what happens when SharedPrePackedWeightCache_StoreWeightData …
adrianlizarraga c42d7ac
Merge branch 'main' into adrianl/KernelPluginEp_KernelInfoApis
adrianlizarraga 3323330
Move some kernel info apis to OrtApi; Fix unrelated memory leak for K…
adrianlizarraga 51bc731
Merge branch 'main' into adrianl/plugin-ep-kernel-prepack
adrianlizarraga edf3f2c
Review comments
adrianlizarraga e94c0aa
C++ API
adrianlizarraga c8eb3c9
Improve doc for c++ api convenience class
adrianlizarraga 98e3d13
Add buffer_sizes as a parameter to OrtKernelImpl::SetSharedWeightData
adrianlizarraga c61ae41
Add comment to implementation of OrtKernelImpl::SetSharedPrePackedWeight
adrianlizarraga 02d75d2
Do not prescribe what the kernel impl should return for a situation t…
adrianlizarraga 0a84eda
Update include/onnxruntime/core/session/onnxruntime_ep_c_api.h
adrianlizarraga 5f80f9d
Adjust comments
adrianlizarraga c60472d
Tweak comment again
adrianlizarraga 12b8394
Merge main and fix conflicts
adrianlizarraga 52453f2
Update example kernel EP to get the OrtEp from OrtKernelInfo and add …
adrianlizarraga 79eadd8
Address copilot review comments
adrianlizarraga 1dc98b2
Address copilot review comment: update error message when APIs for Or…
adrianlizarraga 441c9e2
Add comments to clarify ownership scenarios
adrianlizarraga 5cf9600
Merge branch 'adrianl/plugin-ep-kernel-prepack' into adrianl/KernelPl…
adrianlizarraga ec0183e
Merge main and fix conflicts
adrianlizarraga 7e89261
Add support for Add kernel to test kernel-based EP. This used as moti…
adrianlizarraga 1ddc9fb
Rename variable from 'mul_kernel' to 'binary_op_kernel'
adrianlizarraga 9e6b06e
Address review comments and fix compiler warnings on Linux
adrianlizarraga 1d496a2
Move KernelContext_GetAllocator memory leak fix to a different PR
adrianlizarraga 6378bf2
Simplify binary op type check
adrianlizarraga eadc029
Merge branch 'main' into adrianl/KernelPluginEp_KernelInfoApis
adrianlizarraga cad57e7
Review comments + add new API to get op domain
adrianlizarraga 4fb2be9
Check for NULL size arg in API call. Update comment
adrianlizarraga ebe58ed
Update SAL annotations for older KernelInfo_* API functions
adrianlizarraga 3968ec3
Update example BinaryOp class to support Sub instead of Add
adrianlizarraga df45a2f
Update comment for KernelInfo_GetNodeName
adrianlizarraga File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.