-
Notifications
You must be signed in to change notification settings - Fork 3.7k
Add type definitions, registration, utilities for INT2/UINT2 support #26824
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
29ae9be to
d83952d
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can commit the suggested changes from lintrunner.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds foundational support for 2-bit integer types (INT2/UINT2) to ONNX Runtime, following the ONNX 1.20 specification. It introduces type definitions, registration, and utilities but does not yet include CPU operators (which are planned for follow-up PRs).
Changes:
- Introduces
Int2x4andUInt2x4types for packing 4 two-bit values into a single byte - Registers INT2/UINT2 types throughout the runtime's type system and provider bridge
- Adds comprehensive unit tests for the new 2-bit types
Reviewed changes
Copilot reviewed 22 out of 22 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| include/onnxruntime/core/framework/int2.h | Core implementation of Int2x4/UInt2x4 classes with pack/unpack utilities |
| include/onnxruntime/core/session/onnxruntime_c_api.h | Adds UINT2/INT2 enum values to public C API |
| onnxruntime/core/framework/data_types.cc | Registers INT2/UINT2 tensor types and adds them to type lists |
| onnxruntime/core/framework/element_type_lists.h | Adds AllIRv13 type list including Int2x4/UInt2x4 |
| onnxruntime/core/framework/tensorprotoutils.cc | Implements tensor proto serialization/deserialization for INT2/UINT2 |
| onnxruntime/core/providers/shared_library/* | Adds provider bridge support for INT2/UINT2 types |
| onnxruntime/test/framework/int2_test.cc | Comprehensive unit tests for Int2x4/UInt2x4 functionality |
| onnxruntime/test/util/compare_ortvalue.cc | Adds comparison logic for INT2/UINT2 test outputs |
| onnxruntime/test/unittest_util/checkers.cc | Adds tensor checking utilities for INT2/UINT2 |
| onnxruntime/test/onnx/tensorprotoutils.cc | Adds test utilities for unpacking INT2/UINT2 tensors |
| Various other files | Updates type dispatch macros and utilities to include INT2/UINT2 |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| ORT_CXX_API_THROW("p_data == nullptr, but size != 0", OrtErrorCode::ORT_INVALID_ARGUMENT); \ | ||
| } \ | ||
| if (ONNX_NAMESPACE::ONNX_TYPE != tensor.data_type()) { \ | ||
| ORT_CXX_API_THROW("TensorProto data type is not INT2", OrtErrorCode::ORT_INVALID_ARGUMENT); \ |
Copilot
AI
Jan 13, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The error message is imprecise. It should specify whether the expected type is INT2 or UINT2 to match the specific type being unpacked. Consider making this error message generic (e.g., 'TensorProto data type does not match expected type') or using a parameter to specify the exact type.
| UInt2x4, | ||
| Int2x4>; | ||
|
|
||
| // TODO: This needs upgrade to some newer version ,buit it has been |
Copilot
AI
Jan 13, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Corrected spelling of 'buit' to 'but'.
| // TODO: This needs upgrade to some newer version ,buit it has been | |
| // TODO: This needs upgrade to some newer version ,but it has been |
814dce8 to
5d82d3c
Compare
Description
This is first PR towards adding 2 bit support to ORT
I will create follow up PRs to
Motivation and Context