Skip to content

Commit

Permalink
add IntoPyObjectExt trait (#4708)
Browse files Browse the repository at this point in the history 
* add `IntoPyObjectExt` trait

* adjust method names, more docs & usage internally

* more uses of `IntoPyObjectExt`

* guide docs

* newsfragment

* fixup doctest

* Update guide/src/conversions/traits.md

Co-authored-by: Icxolu <[email protected]>

---------

Co-authored-by: Icxolu <[email protected]>
  • Loading branch information
@davidhewitt @Icxolu
davidhewitt and Icxolu authored Nov 19, 2024
1 parent e37e985 commit 4be4759
Show file tree
Hide file tree
Showing 21 changed files with 218 additions and 333 deletions.
82 changes: 45 additions & 37 deletions guide/src/conversions/traits.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -490,9 +490,11 @@ If the input is neither a string nor an integer, the error message will be:
- the function signature must be `fn(&Bound<PyAny>) -> PyResult<T>` where `T` is the Rust type of the argument.

### `IntoPyObject`
This trait defines the to-python conversion for a Rust type. All types in PyO3 implement this trait,
The ['IntoPyObject'] trait defines the to-python conversion for a Rust type. All types in PyO3 implement this trait,
as does a `#[pyclass]` which doesn't use `extends`.

This trait defines a single method, `into_pyobject()`, which returns a [`Result`] with `Ok` and `Err` types depending on the input value. For convenience, there is a companion [`IntoPyObjectExt`] trait which adds methods such as `into_py_any()` which converts the `Ok` and `Err` types to commonly used types (in the case of `into_py_any()`, `Py<PyAny>` and `PyErr` respectively).

Occasionally you may choose to implement this for custom types which are mapped to Python types
_without_ having a unique python type.

Expand All @@ -510,7 +512,7 @@ into `PyTuple` with the fields in declaration order.

// structs convert into `PyDict` with field names as keys
#[derive(IntoPyObject)]
struct Struct {
struct Struct {
count: usize,
obj: Py<PyAny>,
}
Expand All @@ -532,11 +534,11 @@ forward the implementation to the inner type.

// newtype tuple structs are implicitly `transparent`
#[derive(IntoPyObject)]
struct TransparentTuple(PyObject);
struct TransparentTuple(PyObject);

#[derive(IntoPyObject)]
#[pyo3(transparent)]
struct TransparentStruct<'py> {
struct TransparentStruct<'py> {
inner: Bound<'py, PyAny>, // `'py` lifetime will be used as the Python lifetime
}
```
Expand Down Expand Up @@ -582,7 +584,7 @@ impl<'py> IntoPyObject<'py> for MyPyObjectWrapper {
}
}

// equivalent to former `ToPyObject` implementations
// equivalent to former `ToPyObject` implementations
impl<'a, 'py> IntoPyObject<'py> for &'a MyPyObjectWrapper {
type Target = PyAny;
type Output = Borrowed<'a, 'py, Self::Target>; // `Borrowed` can be used to optimized reference counting
Expand All @@ -594,38 +596,6 @@ impl<'a, 'py> IntoPyObject<'py> for &'a MyPyObjectWrapper {
}
```

### `IntoPy<T>`

<div class="warning">

⚠️ Warning: API update in progress 🛠️

PyO3 0.23 has introduced `IntoPyObject` as the new trait for to-python conversions. While `#[pymethods]` and `#[pyfunction]` contain a compatibility layer to allow `IntoPy<PyObject>` as a return type, all Python API have been migrated to use `IntoPyObject`. To migrate implement `IntoPyObject` for your type.
</div>


This trait defines the to-python conversion for a Rust type. It is usually implemented as
`IntoPy<PyObject>`, which is the trait needed for returning a value from `#[pyfunction]` and
`#[pymethods]`.

All types in PyO3 implement this trait, as does a `#[pyclass]` which doesn't use `extends`.

Occasionally you may choose to implement this for custom types which are mapped to Python types
_without_ having a unique python type.

```rust
use pyo3::prelude::*;
# #[allow(dead_code)]
struct MyPyObjectWrapper(PyObject);

#[allow(deprecated)]
impl IntoPy<PyObject> for MyPyObjectWrapper {
fn into_py(self, py: Python<'_>) -> PyObject {
self.0
}
}
```

#### `BoundObject` for conversions that may be `Bound` or `Borrowed`

`IntoPyObject::into_py_object` returns either `Bound` or `Borrowed` depending on the implementation for a concrete type. For example, the `IntoPyObject` implementation for `u32` produces a `Bound<'py, PyInt>` and the `bool` implementation produces a `Borrowed<'py, 'py, PyBool>`:
Expand Down Expand Up @@ -672,6 +642,8 @@ where
the_vec.iter()
.map(|x| {
Ok(
// Note: the below is equivalent to `x.into_py_any()`
// from the `IntoPyObjectExt` trait
x.into_pyobject(py)
.map_err(Into::into)?
.into_any()
Expand All @@ -693,6 +665,38 @@ let vec_of_pyobjs: Vec<Py<PyAny>> = Python::with_gil(|py| {

In the example above we used `BoundObject::into_any` and `BoundObject::unbind` to manipulate the python types and smart pointers into the result type we wanted to produce from the function.

### `IntoPy<T>`

<div class="warning">

⚠️ Warning: API update in progress 🛠️

PyO3 0.23 has introduced `IntoPyObject` as the new trait for to-python conversions. While `#[pymethods]` and `#[pyfunction]` contain a compatibility layer to allow `IntoPy<PyObject>` as a return type, all Python API have been migrated to use `IntoPyObject`. To migrate implement `IntoPyObject` for your type.
</div>


This trait defines the to-python conversion for a Rust type. It is usually implemented as
`IntoPy<PyObject>`, which is the trait needed for returning a value from `#[pyfunction]` and
`#[pymethods]`.

All types in PyO3 implement this trait, as does a `#[pyclass]` which doesn't use `extends`.

Occasionally you may choose to implement this for custom types which are mapped to Python types
_without_ having a unique python type.

```rust
use pyo3::prelude::*;
# #[allow(dead_code)]
struct MyPyObjectWrapper(PyObject);

#[allow(deprecated)]
impl IntoPy<PyObject> for MyPyObjectWrapper {
fn into_py(self, py: Python<'_>) -> PyObject {
self.0
}
}
```

### The `ToPyObject` trait

<div class="warning">
Expand All @@ -710,8 +714,12 @@ same purpose, except that it consumes `self`.
[`IntoPy`]: {{#PYO3_DOCS_URL}}/pyo3/conversion/trait.IntoPy.html
[`FromPyObject`]: {{#PYO3_DOCS_URL}}/pyo3/conversion/trait.FromPyObject.html
[`ToPyObject`]: {{#PYO3_DOCS_URL}}/pyo3/conversion/trait.ToPyObject.html
[`IntoPyObject`]: {{#PYO3_DOCS_URL}}/pyo3/conversion/trait.IntoPyObject.html
[`IntoPyObjectExt`]: {{#PYO3_DOCS_URL}}/pyo3/conversion/trait.IntoPyObjectExt.html
[`PyObject`]: {{#PYO3_DOCS_URL}}/pyo3/type.PyObject.html

[`PyRef`]: {{#PYO3_DOCS_URL}}/pyo3/pycell/struct.PyRef.html
[`PyRefMut`]: {{#PYO3_DOCS_URL}}/pyo3/pycell/struct.PyRefMut.html
[`BoundObject`]: {{#PYO3_DOCS_URL}}/pyo3/instance/trait.BoundObject.html

[`Result`]: https://doc.rust-lang.org/stable/std/result/enum.Result.html
1 change: 1 addition & 0 deletions newsfragments/4708.added.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add `IntoPyObjectExt` trait.
35 changes: 7 additions & 28 deletions pyo3-macros-backend/src/pyclass.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1367,10 +1367,7 @@ fn impl_complex_enum_tuple_variant_getitem(
.map(|i| {
let field_access = format_ident!("_{}", i);
quote! { #i =>
#pyo3_path::IntoPyObject::into_pyobject(#variant_cls::#field_access(slf)?, py)
.map(#pyo3_path::BoundObject::into_any)
.map(#pyo3_path::BoundObject::unbind)
.map_err(::std::convert::Into::into)
#pyo3_path::IntoPyObjectExt::into_py_any(#variant_cls::#field_access(slf)?, py)
}
})
.collect();
Expand Down Expand Up @@ -1852,16 +1849,10 @@ fn pyclass_richcmp_arms(
.map(|span| {
quote_spanned! { span =>
#pyo3_path::pyclass::CompareOp::Eq => {
#pyo3_path::IntoPyObject::into_pyobject(self_val == other, py)
.map(#pyo3_path::BoundObject::into_any)
.map(#pyo3_path::BoundObject::unbind)
.map_err(::std::convert::Into::into)
#pyo3_path::IntoPyObjectExt::into_py_any(self_val == other, py)
},
#pyo3_path::pyclass::CompareOp::Ne => {
#pyo3_path::IntoPyObject::into_pyobject(self_val != other, py)
.map(#pyo3_path::BoundObject::into_any)
.map(#pyo3_path::BoundObject::unbind)
.map_err(::std::convert::Into::into)
#pyo3_path::IntoPyObjectExt::into_py_any(self_val != other, py)
},
}
})
Expand All @@ -1876,28 +1867,16 @@ fn pyclass_richcmp_arms(
.map(|ord| {
quote_spanned! { ord.span() =>
#pyo3_path::pyclass::CompareOp::Gt => {
#pyo3_path::IntoPyObject::into_pyobject(self_val > other, py)
.map(#pyo3_path::BoundObject::into_any)
.map(#pyo3_path::BoundObject::unbind)
.map_err(::std::convert::Into::into)
#pyo3_path::IntoPyObjectExt::into_py_any(self_val > other, py)
},
#pyo3_path::pyclass::CompareOp::Lt => {
#pyo3_path::IntoPyObject::into_pyobject(self_val < other, py)
.map(#pyo3_path::BoundObject::into_any)
.map(#pyo3_path::BoundObject::unbind)
.map_err(::std::convert::Into::into)
#pyo3_path::IntoPyObjectExt::into_py_any(self_val < other, py)
},
#pyo3_path::pyclass::CompareOp::Le => {
#pyo3_path::IntoPyObject::into_pyobject(self_val <= other, py)
.map(#pyo3_path::BoundObject::into_any)
.map(#pyo3_path::BoundObject::unbind)
.map_err(::std::convert::Into::into)
#pyo3_path::IntoPyObjectExt::into_py_any(self_val <= other, py)
},
#pyo3_path::pyclass::CompareOp::Ge => {
#pyo3_path::IntoPyObject::into_pyobject(self_val >= other, py)
.map(#pyo3_path::BoundObject::into_any)
.map(#pyo3_path::BoundObject::unbind)
.map_err(::std::convert::Into::into)
#pyo3_path::IntoPyObjectExt::into_py_any(self_val >= other, py)
},
}
})
Expand Down
5 changes: 1 addition & 4 deletions pyo3-macros-backend/src/pyimpl.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -213,10 +213,7 @@ pub fn gen_py_const(cls: &syn::Type, spec: &ConstSpec, ctx: &Ctx) -> MethodAndMe

let associated_method = quote! {
fn #wrapper_ident(py: #pyo3_path::Python<'_>) -> #pyo3_path::PyResult<#pyo3_path::PyObject> {
#pyo3_path::IntoPyObject::into_pyobject(#cls::#member, py)
.map(#pyo3_path::BoundObject::into_any)
.map(#pyo3_path::BoundObject::unbind)
.map_err(::std::convert::Into::into)
#pyo3_path::IntoPyObjectExt::into_py_any(#cls::#member, py)
}
};

Expand Down
77 changes: 65 additions & 12 deletions src/conversion.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -178,8 +178,23 @@ pub trait IntoPy<T>: Sized {

/// Defines a conversion from a Rust type to a Python object, which may fail.
///
/// This trait has `#[derive(IntoPyObject)]` to automatically implement it for simple types and
/// `#[derive(IntoPyObjectRef)]` to implement the same for references.
///
/// It functions similarly to std's [`TryInto`] trait, but requires a [GIL token](Python)
/// as an argument.
///
/// The [`into_pyobject`][IntoPyObject::into_pyobject] method is designed for maximum flexibility and efficiency; it
/// - allows for a concrete Python type to be returned (the [`Target`][IntoPyObject::Target] associated type)
/// - allows for the smart pointer containing the Python object to be either `Bound<'py, Self::Target>` or `Borrowed<'a, 'py, Self::Target>`
/// to avoid unnecessary reference counting overhead
/// - allows for a custom error type to be returned in the event of a conversion error to avoid
/// unnecessarily creating a Python exception
///
/// # See also
///
/// - The [`IntoPyObjectExt`] trait, which provides convenience methods for common usages of
/// `IntoPyObject` which erase type information and convert errors to `PyErr`.
#[cfg_attr(
diagnostic_namespace,
diagnostic::on_unimplemented(
Expand Down Expand Up @@ -227,12 +242,7 @@ pub trait IntoPyObject<'py>: Sized {
I: IntoIterator<Item = Self> + AsRef<[Self]>,
I::IntoIter: ExactSizeIterator<Item = Self>,
{
let mut iter = iter.into_iter().map(|e| {
e.into_pyobject(py)
.map(BoundObject::into_any)
.map(BoundObject::into_bound)
.map_err(Into::into)
});
let mut iter = iter.into_iter().map(|e| e.into_bound_py_any(py));
let list = crate::types::list::try_new_from_iter(py, &mut iter);
list.map(Bound::into_any)
}
Expand All @@ -250,12 +260,7 @@ pub trait IntoPyObject<'py>: Sized {
I: IntoIterator<Item = Self> + AsRef<[<Self as private::Reference>::BaseType]>,
I::IntoIter: ExactSizeIterator<Item = Self>,
{
let mut iter = iter.into_iter().map(|e| {
e.into_pyobject(py)
.map(BoundObject::into_any)
.map(BoundObject::into_bound)
.map_err(Into::into)
});
let mut iter = iter.into_iter().map(|e| e.into_bound_py_any(py));
let list = crate::types::list::try_new_from_iter(py, &mut iter);
list.map(Bound::into_any)
}
Expand Down Expand Up @@ -347,6 +352,54 @@ where
}
}

mod into_pyobject_ext {
pub trait Sealed {}
impl<'py, T> Sealed for T where T: super::IntoPyObject<'py> {}
}

/// Convenience methods for common usages of [`IntoPyObject`]. Every type that implements
/// [`IntoPyObject`] also implements this trait.
///
/// These methods:
/// - Drop type information from the output, returning a `PyAny` object.
/// - Always convert the `Error` type to `PyErr`, which may incur a performance penalty but it
/// more convenient in contexts where the `?` operator would produce a `PyErr` anyway.
pub trait IntoPyObjectExt<'py>: IntoPyObject<'py> + into_pyobject_ext::Sealed {
/// Converts `self` into an owned Python object, dropping type information.
#[inline]
fn into_bound_py_any(self, py: Python<'py>) -> PyResult<Bound<'py, PyAny>> {
match self.into_pyobject(py) {
Ok(obj) => Ok(obj.into_any().into_bound()),
Err(err) => Err(err.into()),
}
}

/// Converts `self` into an owned Python object, dropping type information and unbinding it
/// from the `'py` lifetime.
#[inline]
fn into_py_any(self, py: Python<'py>) -> PyResult<Py<PyAny>> {
match self.into_pyobject(py) {
Ok(obj) => Ok(obj.into_any().unbind()),
Err(err) => Err(err.into()),
}
}

/// Converts `self` into a Python object.
///
/// This is equivalent to calling [`into_pyobject`][IntoPyObject::into_pyobject] followed
/// with `.map_err(Into::into)` to convert the error type to [`PyErr`]. This is helpful
/// for generic code which wants to make use of the `?` operator.
#[inline]
fn into_pyobject_or_pyerr(self, py: Python<'py>) -> PyResult<Self::Output> {
match self.into_pyobject(py) {
Ok(obj) => Ok(obj),
Err(err) => Err(err.into()),
}
}
}

impl<'py, T> IntoPyObjectExt<'py> for T where T: IntoPyObject<'py> {}

/// Extract a type from a Python object.
///
///
Expand Down
28 changes: 6 additions & 22 deletions src/conversions/either.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,8 @@
#[cfg(feature = "experimental-inspect")]
use crate::inspect::types::TypeInfo;
use crate::{
conversion::IntoPyObject, exceptions::PyTypeError, types::any::PyAnyMethods, Bound,
BoundObject, FromPyObject, PyAny, PyErr, PyObject, PyResult, Python,
exceptions::PyTypeError, types::any::PyAnyMethods, Bound, FromPyObject, IntoPyObject,
IntoPyObjectExt, PyAny, PyErr, PyObject, PyResult, Python,
};
#[allow(deprecated)]
use crate::{IntoPy, ToPyObject};
Expand Down Expand Up @@ -82,16 +82,8 @@ where

fn into_pyobject(self, py: Python<'py>) -> Result<Self::Output, Self::Error> {
match self {
Either::Left(l) => l
.into_pyobject(py)
.map(BoundObject::into_any)
.map(BoundObject::into_bound)
.map_err(Into::into),
Either::Right(r) => r
.into_pyobject(py)
.map(BoundObject::into_any)
.map(BoundObject::into_bound)
.map_err(Into::into),
Either::Left(l) => l.into_bound_py_any(py),
Either::Right(r) => r.into_bound_py_any(py),
}
}
}
Expand All @@ -108,16 +100,8 @@ where

fn into_pyobject(self, py: Python<'py>) -> Result<Self::Output, Self::Error> {
match self {
Either::Left(l) => l
.into_pyobject(py)
.map(BoundObject::into_any)
.map(BoundObject::into_bound)
.map_err(Into::into),
Either::Right(r) => r
.into_pyobject(py)
.map(BoundObject::into_any)
.map(BoundObject::into_bound)
.map_err(Into::into),
Either::Left(l) => l.into_bound_py_any(py),
Either::Right(r) => r.into_bound_py_any(py),
}
}
}
Expand Down
7 changes: 2 additions & 5 deletions src/coroutine.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ use crate::{
exceptions::{PyAttributeError, PyRuntimeError, PyStopIteration},
panic::PanicException,
types::{string::PyStringMethods, PyIterator, PyString},
Bound, BoundObject, IntoPyObject, Py, PyAny, PyErr, PyObject, PyResult, Python,
Bound, IntoPyObject, IntoPyObjectExt, Py, PyAny, PyErr, PyObject, PyResult, Python,
};

pub(crate) mod cancel;
Expand Down Expand Up @@ -60,10 +60,7 @@ impl Coroutine {
let wrap = async move {
let obj = future.await.map_err(Into::into)?;
// SAFETY: GIL is acquired when future is polled (see `Coroutine::poll`)
obj.into_pyobject(unsafe { Python::assume_gil_acquired() })
.map(BoundObject::into_any)
.map(BoundObject::unbind)
.map_err(Into::into)
obj.into_py_any(unsafe { Python::assume_gil_acquired() })
};
Self {
name: name.map(Bound::unbind),
Expand Down
Loading

0 comments on commit 4be4759

Please sign in to comment.