-
Notifications
You must be signed in to change notification settings - Fork 2.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WIP] Out-Tree EP feature #21450
base: main
Are you sure you want to change the base?
[WIP] Out-Tree EP feature #21450
Commits on Jul 17, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 0e6a80c - Browse repository at this point
Copy the full SHA 0e6a80cView commit details
Commits on Jul 23, 2024
-
Configuration menu - View commit details
-
Copy full SHA for c30a639 - Browse repository at this point
Copy the full SHA c30a639View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7bfe57e - Browse repository at this point
Copy the full SHA 7bfe57eView commit details
Commits on Jul 26, 2024
-
Test Relu with compile based EP, build work, runtime error of loading…
… EP as graph API is not exported by ORT. Need to put these graph API into ortapi structure
Configuration menu - View commit details
-
Copy full SHA for 8e7d28d - Browse repository at this point
Copy the full SHA 8e7d28dView commit details
Commits on Jul 29, 2024
-
prototype works with hardcode node_compute_info's index in ExecutionP…
…roviderAdapter::Compile()
Configuration menu - View commit details
-
Copy full SHA for 808bfc3 - Browse repository at this point
Copy the full SHA 808bfc3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 49e396c - Browse repository at this point
Copy the full SHA 49e396cView commit details
Commits on Jul 31, 2024
-
Configuration menu - View commit details
-
Copy full SHA for e790105 - Browse repository at this point
Copy the full SHA e790105View commit details
Commits on Aug 1, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 92f529d - Browse repository at this point
Copy the full SHA 92f529dView commit details
Commits on Aug 5, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 3d83ed1 - Browse repository at this point
Copy the full SHA 3d83ed1View commit details
Commits on Aug 6, 2024
-
Configuration menu - View commit details
-
Copy full SHA for e29499a - Browse repository at this point
Copy the full SHA e29499aView commit details
Commits on Aug 7, 2024
-
Configuration menu - View commit details
-
Copy full SHA for f3678c4 - Browse repository at this point
Copy the full SHA f3678c4View commit details -
Configuration menu - View commit details
-
Copy full SHA for ac5ae0a - Browse repository at this point
Copy the full SHA ac5ae0aView commit details
Commits on Aug 12, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 0cc78e8 - Browse repository at this point
Copy the full SHA 0cc78e8View commit details
Commits on Aug 13, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 740a687 - Browse repository at this point
Copy the full SHA 740a687View commit details
Commits on Aug 15, 2024
-
Configuration menu - View commit details
-
Copy full SHA for dad6397 - Browse repository at this point
Copy the full SHA dad6397View commit details
Commits on Aug 16, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 94e9cf7 - Browse repository at this point
Copy the full SHA 94e9cf7View commit details
Commits on Aug 20, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 8698517 - Browse repository at this point
Copy the full SHA 8698517View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3d5d2bf - Browse repository at this point
Copy the full SHA 3d5d2bfView commit details
Commits on Aug 22, 2024
-
Add extra parameters in OrtExecutionProvider to avoid capture variabl…
…es in Compile function's lambda
Configuration menu - View commit details
-
Copy full SHA for 1f10c28 - Browse repository at this point
Copy the full SHA 1f10c28View commit details
Commits on Aug 23, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 5e46d0f - Browse repository at this point
Copy the full SHA 5e46d0fView commit details
Commits on Aug 24, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 85c168d - Browse repository at this point
Copy the full SHA 85c168dView commit details
Commits on Aug 26, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 7bdb36a - Browse repository at this point
Copy the full SHA 7bdb36aView commit details
Commits on Aug 27, 2024
-
add outOfTree tensorrt ep.1 (#21830)
add GetCapability for tensorrt ep
Configuration menu - View commit details
-
Copy full SHA for 7d915b7 - Browse repository at this point
Copy the full SHA 7d915b7View commit details
Commits on Aug 28, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 4aea94b - Browse repository at this point
Copy the full SHA 4aea94bView commit details
Commits on Aug 29, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 865a17f - Browse repository at this point
Copy the full SHA 865a17fView commit details -
Add simple CUDA allocators for TRT EP (#21901)
Only implemented two simple CUDA allocators (without BFC) for now. Note: several TODO.
Configuration menu - View commit details
-
Copy full SHA for 2811541 - Browse repository at this point
Copy the full SHA 2811541View commit details -
Configuration menu - View commit details
-
Copy full SHA for c97b19f - Browse repository at this point
Copy the full SHA c97b19fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 36f97b5 - Browse repository at this point
Copy the full SHA 36f97b5View commit details
Commits on Aug 31, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 2fc7aac - Browse repository at this point
Copy the full SHA 2fc7aacView commit details
Commits on Sep 2, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 4ad6993 - Browse repository at this point
Copy the full SHA 4ad6993View commit details
Commits on Sep 4, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 53c736f - Browse repository at this point
Copy the full SHA 53c736fView commit details
Commits on Sep 5, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 5fcb972 - Browse repository at this point
Copy the full SHA 5fcb972View commit details
Commits on Sep 6, 2024
-
Configuration menu - View commit details
-
Copy full SHA for c3bb437 - Browse repository at this point
Copy the full SHA c3bb437View commit details -
Merge branch 'leca/outOfTreeEP' of https://github.com/microsoft/onnxr…
…untime into leca/outOfTreeEP
Configuration menu - View commit details
-
Copy full SHA for d1c657c - Browse repository at this point
Copy the full SHA d1c657cView commit details -
resnet18-v1-7 works for TRT EP, with next_nodes_list assignment comme…
…nted out in GetSupportedList()
Configuration menu - View commit details
-
Copy full SHA for 3efac97 - Browse repository at this point
Copy the full SHA 3efac97View commit details
Commits on Sep 9, 2024
-
test cases for decoder and fast_rcnn, delete dynamic_cast in ShouldPo…
…stPoneRegisterResourceFor()
Configuration menu - View commit details
-
Copy full SHA for 766fec9 - Browse repository at this point
Copy the full SHA 766fec9View commit details
Commits on Sep 11, 2024
-
add tensorrt home in CMakeLists, add trt and CUDA ep for test, change…
… trt type to tensorrtEp
Configuration menu - View commit details
-
Copy full SHA for ea2465c - Browse repository at this point
Copy the full SHA ea2465cView commit details
Commits on Sep 18, 2024
-
[WIP, DONT REVIEW] add initializer to graph proto (#22085)
### Description <!-- Describe your changes. --> ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> --------- Co-authored-by: guyang3532 <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 76a9305 - Browse repository at this point
Copy the full SHA 76a9305View commit details
Commits on Sep 19, 2024
-
use parameter ExecutionOrder::PRIORITY_BASED for GraphViewerToProto()…
… to make graph partition the same, and improve test
Configuration menu - View commit details
-
Copy full SHA for 330cdb6 - Browse repository at this point
Copy the full SHA 330cdb6View commit details
Commits on Sep 23, 2024
-
can create session with out tree trt ep now. Error:Name:'tensorrtEp_T…
…RTKernel_graph_torch-jit-export10497453988321570186_10_10' Status Message: TensorRT EP failed to create engine from network
Configuration menu - View commit details
-
Copy full SHA for 6fd50f0 - Browse repository at this point
Copy the full SHA 6fd50f0View commit details -
make trt_node_name_with_precision_ from string to map, to capture the…
… corresponding value for compute function
Configuration menu - View commit details
-
Copy full SHA for 681585f - Browse repository at this point
Copy the full SHA 681585fView commit details
Commits on Sep 24, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 7db20cb - Browse repository at this point
Copy the full SHA 7db20cbView commit details
Commits on Sep 25, 2024
-
Configuration menu - View commit details
-
Copy full SHA for ff782e0 - Browse repository at this point
Copy the full SHA ff782e0View commit details
Commits on Sep 26, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 1d7b2df - Browse repository at this point
Copy the full SHA 1d7b2dfView commit details -
Configuration menu - View commit details
-
Copy full SHA for a407944 - Browse repository at this point
Copy the full SHA a407944View commit details
Commits on Oct 2, 2024
-
new test control_flow, error: ErrorMessage:Failed to find kernel for …
…MemcpyFromHost(1) (node:'Memcpy' ep:'tensorrtEp'). Kernel not found
Configuration menu - View commit details
-
Copy full SHA for f871b25 - Browse repository at this point
Copy the full SHA f871b25View commit details
Commits on Oct 3, 2024
-
Configuration menu - View commit details
-
Copy full SHA for e84f00c - Browse repository at this point
Copy the full SHA e84f00cView commit details
Commits on Oct 7, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 5b2de22 - Browse repository at this point
Copy the full SHA 5b2de22View commit details
Commits on Oct 14, 2024
-
Configuration menu - View commit details
-
Copy full SHA for b1f8e2a - Browse repository at this point
Copy the full SHA b1f8e2aView commit details
Commits on Oct 15, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 7acaaab - Browse repository at this point
Copy the full SHA 7acaaabView commit details
Commits on Oct 17, 2024
-
Configuration menu - View commit details
-
Copy full SHA for d150a03 - Browse repository at this point
Copy the full SHA d150a03View commit details
Commits on Oct 18, 2024
-
Configuration menu - View commit details
-
Copy full SHA for da5b6eb - Browse repository at this point
Copy the full SHA da5b6ebView commit details -
Configuration menu - View commit details
-
Copy full SHA for d280e59 - Browse repository at this point
Copy the full SHA d280e59View commit details
Commits on Oct 19, 2024
-
Configuration menu - View commit details
-
Copy full SHA for cbe98e7 - Browse repository at this point
Copy the full SHA cbe98e7View commit details
Commits on Oct 21, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 1529059 - Browse repository at this point
Copy the full SHA 1529059View commit details
Commits on Oct 24, 2024
-
Configuration menu - View commit details
-
Copy full SHA for fa549f8 - Browse repository at this point
Copy the full SHA fa549f8View commit details -
Configuration menu - View commit details
-
Copy full SHA for a28ad38 - Browse repository at this point
Copy the full SHA a28ad38View commit details
Commits on Oct 26, 2024
-
Configuration menu - View commit details
-
Copy full SHA for aa49805 - Browse repository at this point
Copy the full SHA aa49805View commit details
Commits on Oct 31, 2024
-
Configuration menu - View commit details
-
Copy full SHA for bc65613 - Browse repository at this point
Copy the full SHA bc65613View commit details
Commits on Nov 1, 2024
-
Configuration menu - View commit details
-
Copy full SHA for a1a3eea - Browse repository at this point
Copy the full SHA a1a3eeaView commit details -
Configuration menu - View commit details
-
Copy full SHA for 0fe5f01 - Browse repository at this point
Copy the full SHA 0fe5f01View commit details -
Add some utility files for plugin ep to include and compile. - provider option map -> `provider_option`.h - provider option parser -> `provider_option_utils`.h - some macro define, classes and functions from include/onnxruntime/core/common
Configuration menu - View commit details
-
Copy full SHA for 6bae1b9 - Browse repository at this point
Copy the full SHA 6bae1b9View commit details
Commits on Nov 2, 2024
-
Configuration menu - View commit details
-
Copy full SHA for ab75d98 - Browse repository at this point
Copy the full SHA ab75d98View commit details -
Merge branch 'leca/outOfTreeEP' of https://github.com/microsoft/onnxr…
…untime into leca/outOfTreeEP
Configuration menu - View commit details
-
Copy full SHA for c5510f2 - Browse repository at this point
Copy the full SHA c5510f2View commit details -
Add unit test for TRT EP plugin (#22548)
Hook TRT EP plugin to run the existing unit test in CI - Migrate from `onnxruntime/test/providers/tensorrt/tensorrt_basic_test.cc` - Replace internal APIs with new EP APIs - Add unit test in `onnxruntime_shared_lib_test` (which links against onnxruntime dll) - Build ORT with `--test_tensorrt_ep_plugin` to run `onnxruntime_shared_lib_test` Note: The unit test doesn't cover all the cases since current TRT EP plugin hasn't added all the features yet, will update later.
Configuration menu - View commit details
-
Copy full SHA for 08e3f20 - Browse repository at this point
Copy the full SHA 08e3f20View commit details
Commits on Nov 5, 2024
-
Configuration menu - View commit details
-
Copy full SHA for b0b3123 - Browse repository at this point
Copy the full SHA b0b3123View commit details
Commits on Nov 6, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 9dbb0b1 - Browse repository at this point
Copy the full SHA 9dbb0b1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5a59803 - Browse repository at this point
Copy the full SHA 5a59803View commit details -
Configuration menu - View commit details
-
Copy full SHA for 999e7fd - Browse repository at this point
Copy the full SHA 999e7fdView commit details -
Configuration menu - View commit details
-
Copy full SHA for 084f735 - Browse repository at this point
Copy the full SHA 084f735View commit details
Commits on Nov 7, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 2b1cfdf - Browse repository at this point
Copy the full SHA 2b1cfdfView commit details
Commits on Nov 13, 2024
-
Add OrtGraphApis::OrtNode_GetAttributeStrWithSize to handle case wher…
…e attribute might contain null character (#22769) When running EP Context model, EP might call `OrtGraphApis::OrtNode_GetAttributeStr` to get the string-based content of the attribute. However, the API returns the c_str() of the string, and it's possible that the cache context contains null character, so the string might be cut off and caller ends up getting the wrong string. Add a new OrtGraphApis::OrtNode_GetAttributeStrWithSize to return const char* pointer and string size.
Configuration menu - View commit details
-
Copy full SHA for e337d8f - Browse repository at this point
Copy the full SHA e337d8fView commit details -
Make EP plugin be able to create and update EP Context graph (#22740)
This PR support several features: - Add new graph API to create and update EP Context graph, and dump EP Context model. 1. OrtGraph_CreateOrUpdateEpCtxGraph 2. OrtGraph_DumpOnnxModel 3. OrtGraph_ReleaseGraph - Add new graph API to dump onnx model - The APIs provided by this PR can dump EP Context model when the whole model can be run by one EP, the APIs also aim to support the case where the whole model is partitioned into multiple EP's subgraphs. (Note: i haven't fully tested the partitioning case, please help review it) - Modify TRT EP plugin to use those APIs.
Configuration menu - View commit details
-
Copy full SHA for afe92e1 - Browse repository at this point
Copy the full SHA afe92e1View commit details
Commits on Nov 14, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 63f8774 - Browse repository at this point
Copy the full SHA 63f8774View commit details
Commits on Nov 16, 2024
-
Configuration menu - View commit details
-
Copy full SHA for bf359a1 - Browse repository at this point
Copy the full SHA bf359a1View commit details
Commits on Nov 20, 2024
-
[TensorRT EP Plugin] Add cuda::Impl_Cast (#22908)
TRT 8 doesn't support INT64 and DOUBLE data type. TRT 10 doesn't support DOUBLE data type. Therefore, TRT EP internally needs to convert INT64 to INT32, and DOUBLE to FLOAT, which needs the cuda::Impl_Cast function. The implementation is copied from CUDA EP.
Configuration menu - View commit details
-
Copy full SHA for c267ea5 - Browse repository at this point
Copy the full SHA c267ea5View commit details
Commits on Nov 22, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 72afdc4 - Browse repository at this point
Copy the full SHA 72afdc4View commit details
Commits on Dec 3, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 6822206 - Browse repository at this point
Copy the full SHA 6822206View commit details -
Configuration menu - View commit details
-
Copy full SHA for c8ddc73 - Browse repository at this point
Copy the full SHA c8ddc73View commit details