How to work with the same reference of my cloned object

[cyrus937]

Hello,

I'm trying to execute asynchronous functions in methods using the pyo3_asyncio library, based mainly on this example issue #1632

I'd like to continue working with the reference of my object that has been cloned, as this object is modified in the asynchronous function.

An example of what I want to do is shown below.

Do you have any ideas on how I could do this?

Thanks in advance.

#[pymodule]
fn example(_py: Python<'_>, m: &PyModule) -> PyResult<()> {
    m.add_class::<MyStruct >()?;
    Ok(())
}

#[derive(Clone)]
#[pyclass]
struct MyStruct {
    a: u32,
}

impl MyStruct {
    async fn future(&mut self) -> PyResult<()> {
        add(self).await;
        Ok(())
    }
}

#[pymethods]
impl MyStruct {
    #[new]
    fn new(a: u32) -> MyStruct {
        MyStruct {a}
    }

    fn send_asyncio<'py>(&self, py: Python<'py>) -> PyResult<&'py PyAny> {
        pyo3_asyncio::tokio::future_into_py(py, self.clone().future())
    }

    fn display(&self) {
         println!("The value is : {}", self.a);
    }
}

async fn add(myStruct: &mut MyStruct )
{
     myStruct.a = myStruct.a + 1 ;
}

[davidhewitt]

To work with the Python representation of number you should use Py<Number> rather than &Number or &mut Number. It has try_borrow and try_borrow_mut methods to get to & and &mut references when you need them.

[cyrus937]

Please I don't quite understand your answer.

[davidhewitt]

Try to make your send_asyncio method like this:

    fn send_asyncio<'py>(slf: Py<Self>, py: Python<'py>) -> PyResult<&'py PyAny> {
        pyo3_asyncio::tokio::future_into_py(py, async move { slf.try_borrow_mut(py)?.future().await })
    }

[cyrus937]

When I try that, I got some error which is

*mut pyo3::Python<'static>` cannot be sent between threads safely within `[async block@src\lib.rs:122:49: 122:101]`, the trait `std::marker::Send` is not implemented for `*mut pyo3::Python<'static>` required because it appears within the type `(&GILGuard, NotSend)

[davidhewitt]

Right, so there's some complexity here. The future needs to be Send, so py can't be sent between threads. This leads to the error you are encountering.

The problem is that you need the Python GIL held to safely modify the data contained in the Python object. So ultimately you need to do this:

impl MyStruct {
    async fn future(slf: &Py<Self>) -> PyResult<()> {
        add(slf).await
    }
}

#[pymethods]
impl MyStruct {
    #[new]
    fn new(a: u32) -> MyStruct {
        MyStruct {a}
    }

    fn send_asyncio<'py>(slf: Py<Self>, py: Python<'py>) -> PyResult<&'py PyAny> {
        pyo3_asyncio::tokio::future_into_py(py, async move { Number::future(&slf).await })
    }

    fn display(&self) {
         println!("The value is : {}", self.a);
    }
}

async fn add(myStruct: &Py<MyStruct> ) -> PyResult<()>
{
     Python::with_gil(|py| {
        let myStruct = myStruct.try_borrow_mut()?;
        myStruct.a = myStruct.a + 1 ;
     });
    Ok(())
}

This is what I mean by you need to work with Py<Number>; in general if you want to pass around references to the Python data and modify it, you'll need to use Python::with_gil to make modifications with the GIL held without crossing .await points.

[cyrus937]

Thanks, but if in addition to that in my add function I'd like to call an asynchronous function, in my previous discussion (#3665) I was told that the gil won't allow me to do that.

For example, do this:

async fn add(myStruct: &Py<MyStruct> ) -> PyResult<()>
{
     Python::with_gil(|py| {
        let mut myStruct = myStruct.try_borrow_mut()?;
        myStruct.a = myStruct.a + 1 ;
        some_async_function(myStruct).await;
     });
    Ok(())
}

I was looking for solution and came across this #1632