Repository navigation
[Go] Update to the current C API and add plugin execution providers - #33282
Open
Danny On The Air (dannyota) wants to merge 7 commits into
Open
Danny On The Air (dannyota) wants to merge 7 commits into
Danny On The Air (dannyota) wants to merge 7 commits into
Conversation
The test binary exited zero without running any test when ORT_LIB_PATH was unset and no fallback library existed, so a missing runtime could look like a passing suite. TestMain now fails with a setup diagnostic in that case, every child mode must initialize successfully, and the profiling test no longer writes to a fixed Unix path. TestMain sets ORT_RUNNING_UNIT_TESTS before initializing so the runtime's telemetry does not leave files in the package directory. Subprocess tests cover an unset runtime, an explicit missing path, an invalid library and a successful setup. Signed-off-by: Danny <dannyontheair@outlook.com>
The vendored headers had drifted to API 29 while the canonical headers are at API 32, and unrelated C API changes had to hand-edit the copies. The three headers are now exact copies of include/onnxruntime/core/session, and the binding requests the API version that matches the loaded runtime: it reads the runtime version string, asks for min(runtime minor, 32) and falls back down to API 17. Requesting a version the runtime does not have makes ONNX Runtime print to stderr, so the binding no longer probes from the top. Every wrapper now checks the negotiated version before reading a function pointer and returns a typed error when the runtime is too old. Adds the error codes 12 to 15 and moves the API 27 option getter checks into the shim. Signed-off-by: Danny <dannyontheair@outlook.com>
Execution providers now ship as plugin libraries (for example the CUDA and WebGPU plugin EPs), but Go callers could only append providers by name. This adds RegisterExecutionProviderLibrary and UnregisterExecutionProviderLibrary, device discovery through GetEpDevices with immutable Go snapshots of the device and hardware metadata, explicit selection with SessionOptionsAppendExecutionProvider_V2, and the native selection policies. Unregistering a library fails while any session or session options still reference its devices, because the native objects borrow from the library. All plugin calls require API 22 and return a typed error on older runtimes. Signed-off-by: Danny <dannyontheair@outlook.com>
Adds SetDeterministicCompute, explicit I/O binding synchronization and SessionOptionsSetWeightlessSourceModelBuffer (API 29), which lets a weightless compiled model take its initializers from memory. The binding copies the buffer into C memory and keeps it alive for every options object and session that uses it. Two native paths drop the buffer on the runtimes released so far: CloneSessionOptions does not copy it, and automatic EP selection builds fresh options without it. The binding restores the buffer on every clone, and rejects combining a buffer with a selection policy, so explicit device selection is the supported path. Session constructors work on a private options snapshot so the public options lock is not held while native code runs. The orttest build tag adds native checks that read the buffer back from a clone and from inside a test provider's CreateEp. Signed-off-by: Danny <dannyontheair@outlook.com>
SessionOptionsSetEpContextDataReadFunc (API 31) lets an application supply named external EPContext data from memory instead of the file system. The Go reader receives the data name and returns the bytes, bounded by a caller-set limit; the binding copies them into memory from the allocator ONNX Runtime passes in and translates errors and panics into an OrtStatus. The reader stays alive for every options clone and session that can call it, and session creation does not hold the options lock while the reader runs, so the reader may call options getters or Clone. Tests drive the callback from native code, including concurrent calls, calls from a C thread, re-entry from both constructors and a bounded subprocess timeout, and run under cgocheck2 as well as the race detector. Signed-off-by: Danny <dannyontheair@outlook.com>
Adds integration tests that load a real plugin library, select its device explicitly and run a float32 Mul model, then check the session profile to prove the nodes ran on the plugin rather than falling back to CPU. The tests are opt-in through ORT_TEST_WEBGPU_EP_PATH and ORT_TEST_PLUGIN_EP_PATH, skip when the variable is unset, and fail when it is set but the plugin cannot be loaded or used. The tests prove execution on the plugin EP, not which adapter the plugin chose. Verified with the WebGPU plugin EP 0.4.0 and ONNX Runtime 1.31.0 on an Intel Arc 130V/140V GPU, with the Intel Vulkan driver loaded and no software Vulkan driver. The example_plugin_ep test needs a runtime built from the same source and belongs in source-build CI. Signed-off-by: Danny <dannyontheair@outlook.com>
This was referenced Oct 11, 2026
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Copilot started reviewing on behalf of
Danny On The Air (dannyota)
October 11, 2026 15:08
View session
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Compiler command parsing breaks valid quoted CC configurations in two new test fixtures.
2 open findings
What changed in this PR
Updates the Go binding to C API 32, adding plugin EP support and newer runtime features.
Changes:
- Adds API negotiation and typed feature gating.
- Adds plugin EP discovery, selection, and lifecycle handling.
- Adds weightless models, EPContext callbacks, synchronization, and tests.
| File | Description |
|---|---|
go/testdata/gen_models.py |
Generates plugin test model. |
go/testdata/mul_f32.onnx |
Generated Mul fixture. |
go/onnxruntime/session.go |
Snapshots options and transfers resources. |
go/onnxruntime/ort.go |
Negotiates API and tracks lifecycle. |
go/onnxruntime/ort_test.go |
Tests initialization and feature gates. |
go/onnxruntime/options.go |
Clones resources and adds deterministic mode. |
go/onnxruntime/options_test.go |
Tests updated options behavior. |
go/onnxruntime/onnxruntime_error_code.h |
Synchronizes error codes. |
go/onnxruntime/onnxruntime_ep_c_api.h |
Synchronizes EP API declarations. |
go/onnxruntime/onnxruntime_c_api.h |
Synchronizes C API 32 declarations. |
go/onnxruntime/iobinding.go |
Adds synchronization methods. |
go/onnxruntime/iobinding_test.go |
Tests I/O synchronization. |
go/onnxruntime/errors.go |
Exposes new error codes. |
go/onnxruntime/ep.go |
Implements plugin EP APIs. |
go/onnxruntime/ep_test.go |
Tests EP selection and lifetimes. |
go/onnxruntime/ep_integration_test.go |
Adds real-plugin integration coverage. |
go/onnxruntime/ep_context.go |
Implements buffers and reader callbacks. |
go/onnxruntime/ep_context_testbridge.go |
Adds tagged native test bridge. |
go/onnxruntime/ep_context_test.go |
Tests resource and callback lifetimes. |
go/onnxruntime/ep_context_bridge_test.go |
Tests native callback behavior. |
go/onnxruntime/doc.go |
Documents compatibility and lifecycle rules. |
go/onnxruntime/cshim.h |
Declares new shim APIs. |
go/onnxruntime/cshim.c |
Implements negotiation and API shims. |
🧠 Review effort: Balanced
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Two test fixtures split the output of `go env CC` with strings.Fields, which breaks a compiler path that contains spaces or quoted flags. They now use a test-only parser that follows cmd/internal/quoted.Split, and the orttest bridge receives the parsed command from its test callers. Signed-off-by: Danny <dannyontheair@outlook.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Brings the Go binding up to the current C API and adds plugin execution provider support.
Version handling
include/onnxruntime/core/session/(API 32).Initrequests the API version that matches the loaded runtime and falls back to API 17. It does not probe from the top, because every rejectedGetApicall prints to stderr.New APIs
RegisterExecutionProviderLibrary,UnregisterExecutionProviderLibraryGetEPDevices,EPDevice,HardwareDeviceSessionOptions.AppendExecutionProviderV2,SessionOptions.SetEPSelectionPolicySessionOptions.SetWeightlessSourceModelBufferSessionOptions.SetEPContextDataReader,ClearEPContextDataReaderSetDeterministicCompute,IOBinding.SynchronizeInputs/SynchronizeOutputs,GetBuildInfo, error codes 12 to 15Workaround for #33281: cloned options and EP selection policies drop the weightless buffer. The binding restores the buffer on every clone and rejects a selection policy combined with a buffer, so explicit device selection is the supported path.
Test harness: the tests used to exit 0 without running anything when no runtime was set. They now fail.
The change is split into six commits that each build and pass on their own:
Motivation and Context
This PR touches 23 files: three are exact header copies, one is a generated test model and two are test-only bridges (
-tags=orttest). Header sync, version negotiation and plugin and callback lifetimes share code and tests, so I kept them in one PR. Release-archive CI, module tags and the README will be separate PRs.Testing (Linux x64)
go test -racepasses against released ORT 1.27.0, 1.28.3, 1.29.1, 1.30.0 and 1.31.0, negotiating API 27 to 31 with nothing written to stderr.-tags=orttestsuite passes with-raceon all five, and withGOEXPERIMENT=cgocheck2on 1.31.0.WebGpuExecutionProvider.gofmt,go vetandgolangci-lintare clean.Not yet covered: the
example_plugin_epand EPContext integration tests, which need a runtime built from the same source; macOS, Windows and Linux arm64.