Skip to content

Update exchange API to be capsule following new convention #180

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
tqchen wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Update exchange API to be capsule following new convention #180

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
5eeaaae
Update exchange API to be capsule following new convention
tqchen Dec 5, 2025
1f18a9e
[DOCS] Update the exchange api spec
tqchen Dec 12, 2025
bd96c5f
Update docs/source/python_spec.rst
tqchen Dec 24, 2025
4d25c02
Update docs/source/python_spec.rst
tqchen Dec 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions docs/source/python_spec.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,42 @@ refer to `github.com/dmlc/dlpack <https://github.com/dmlc/dlpack>`_.
guaranteed to be in a certain order or not.


DLPack C Exchange API
~~~~~~~~~~~~~~~~~~~~~

Starting with DLPack 1.3, a new C Exchange API is introduced to enable faster
data exchange than the Python ``__dlpack__`` API at the C extension level.
Producer array frameworks must provide a ``__dlpack_c_exchange_api__``
attribute on the array type.
The attribute should be a ``PyCapsule`` with name ``"dlpack_exchange_api"``.
The consumer can query whether this attribute exists and use it at the C extension level.
Notably, consumer frameworks can always start implementing by only using the Python ``__dlpack__`` API,
and then upgrade to the C Exchange API later when faster data exchange is needed.

.. code-block:: C

PyObject *api_obj = type(tensor_obj).__dlpack_c_exchange_api__; // as C code.
Copy link
Collaborator

@leofang leofang Dec 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am confused. Is this valid C code? Shouldn't we use PyObject_HasAttrString or alike to query if the Python attribute exists with this type?

Copy link
Member Author

@tqchen tqchen Dec 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not valid C code, but as a pesudo cython like code to query the types, happy to change to C code here

MyDLPackExchangeAPI *api = PyCapsule_GetPointer(api_obj, "dlpack_exchange_api");
if (api == NULL && PyErr_Occurred()) { goto handle_error; }


.. note:: Implementation of the C Exchange API

Producer framework should implement the C Exchange API in a static way either
through Cython, Python C extensions, or Python binding mechanism. Importantly,
because the DLPack C exchange API operates at the C extension level, we need
direct interaction between the array framework PyObject* and DLPack,
as a result it is harder to implement the C Exchange API through ctypes (because
ctypes releases GIL by default and sometimes in non-free-threading environment,
GIL is needed to interact with the Python C API).
Copy link
Collaborator

@leofang leofang Dec 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dalcinl or @seberg to comment. I am aware that this is to capture the meeting discussion on why ctypes does not work for our purposes, but I am a bit confused by the mention of GIL.


A reference implementations of the C Exchange API in frameworks:


* PyTorch: `C++ <https://github.com/pytorch/pytorch/blob/main/torch/csrc/Module.cpp#L692>`__
Copy link
Collaborator

@leofang leofang Dec 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's link to a particular commit (copy permalink) in case the upstream drifts away.

* Paddle: `C++ <https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/pybind/pybind.cc#L856>`__
Copy link
Collaborator

@leofang leofang Dec 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto



Reference Implementations
~~~~~~~~~~~~~~~~~~~~~~~~~

Expand All @@ -198,3 +234,4 @@ ctypes, cffi, etc:
* mpi4py: `Cython <https://github.com/mpi4py/mpi4py/blob/master/src/mpi4py/MPI.src/asdlpack.pxi>`_
* Paddle: `C++ <https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/framework/tensor_util.cc#L901-L951>`__, `Python wrapper using Python C API <https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/pybind/pybind.cc#L1263-L1280>`__
* Hidet: `ctypes <https://github.com/hidet-org/hidet/blob/main/python/hidet/graph/impl/dlpack.py>`__

23 changes: 12 additions & 11 deletions include/dlpack/dlpack.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
#define DLPACK_MAJOR_VERSION 1

/*! \brief The current minor version of dlpack */
#define DLPACK_MINOR_VERSION 2
#define DLPACK_MINOR_VERSION 3

/*! \brief DLPACK_DLL prefix for windows */
#ifdef _WIN32
Expand Down Expand Up @@ -375,7 +375,7 @@ typedef struct DLManagedTensorVersioned {
} DLManagedTensorVersioned;

//----------------------------------------------------------------------
// DLPack `__c_dlpack_exchange_api__` fast exchange protocol definitions
// DLPack `__dlpack_c_exchange_api__` fast exchange protocol definitions
//----------------------------------------------------------------------
/*!
* \brief Request a producer library to create a new tensor.
Expand All @@ -391,7 +391,7 @@ typedef struct DLManagedTensorVersioned {
* \param error_ctx Context for `SetError`.
* \param SetError The function to set the error.
* \return The owning DLManagedTensorVersioned* or NULL on failure.
* SetError is called exactly when NULL is returned (the implementor
* SetError is called exactly when NULL is returned (the implementer
* must ensure this).
* \note - As a C function, must not thrown C++ exceptions.
* - Error propagation via SetError to avoid any direct need
Expand Down Expand Up @@ -432,11 +432,11 @@ typedef int (*DLPackManagedTensorFromPyObjectNoSync)( //
* \brief Exports a PyObject* Tensor/NDArray to a provided DLTensor.
*
* This function provides a faster interface for temporary, non-owning, exchange.
* The producer (implementor) still owns the memory of data, strides, shape.
* The producer (implementer) still owns the memory of data, strides, shape.
* The liveness of the DLTensor and the data it views is only guaranteed until
* control is returned.
*
* This function currently assumes that the producer (implementor) can fill
* This function currently assumes that the producer (implementer) can fill
* in the DLTensor shape and strides without the need for temporary allocations.
*
* This function does not perform any stream synchronization. The consumer should query
Expand Down Expand Up @@ -488,7 +488,7 @@ typedef int (*DLPackCurrentWorkStream)( //
* \brief Imports a DLManagedTensorVersioned to a PyObject* Tensor/NDArray.
*
* Convert an owning DLManagedTensorVersioned* to the Python tensor of the
* producer (implementor) library with the correct type.
* producer (implementer) library with the correct type.
*
* This function does not perform any stream synchronization.
*
Expand Down Expand Up @@ -532,16 +532,17 @@ typedef struct DLPackExchangeAPIHeader {
* \brief Framework-specific function pointers table for DLPack exchange.
*
* Additionally to `__dlpack__()` we define a C function table sharable by
* Python implementations via `__c_dlpack_exchange_api__`.
* This attribute must be set on the type as a Python integer compatible
* with `PyLong_FromVoidPtr`/`PyLong_AsVoidPtr`.
*
* Python implementations via `__dlpack_c_exchange_api__`.
* This attribute must be set on the type as a Python PyCapsule
* with name "dlpack_exchange_api".
*
* A consumer library may use a pattern such as:
*
* \code
*
* PyObject *api_obj = type(tensor_obj).__c_dlpack_exchange_api__; // as C-code
* MyDLPackExchangeAPI *api = PyLong_AsVoidPtr(api_obj);
* PyObject *api_obj = type(tensor_obj).__dlpack_c_exchange_api__; // as C-code
Copy link
Collaborator

@leofang leofang Dec 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto, now that I take a closer look, this seems odd to me

* MyDLPackExchangeAPI *api = PyCapsule_GetPointer(api_obj, "dlpack_exchange_api");
* if (api == NULL && PyErr_Occurred()) { goto handle_error; }
*
* \endcode
Expand Down