Implement locked iteration for PyList

[ngoldbaum]

Fixes #4571.

Re-enables get_item_unchecked on the free-threaded build (with a new free-threaded-specific note about safety), adds locked_for_each, and implements a number of iterator methods for BouldListIterator on the free-threaded build to amortize synchronization overhead where possible.

Largely follows the implementation and tests from #4439, along with fixes similar to the ones I implemented for #4788.

[ngoldbaum]

I was chatting with @epilys on an IRC channel we both use and he had a suggestion to avoid the inner struct. I've included a commit from him implementing that.

[mejrs]

rather than just using this to split the borrow, maybe we should just have a macro that wraps

        crate::sync::with_critical_section(list, || {
            ...
        })

as well?

[ngoldbaum]

I tried to do this this afternoon and got stuck. I'm pretty novice at writing macros - do you know of any similar examples I could look at somewhere?

Here's what I tried, but this doesn't compile if I used it in e.g. the fold implementation: https://gist.github.com/ngoldbaum/13ac11629f042ae0bee84559e4e8bb31

[epilys]

@ngoldbaum was the error about identifiers not being found? If yes it's because Rust macros are hygienic, all referenced identifiers in a macro argument must be defined outside of the macro. What you can do is something like this:

/// Helper to manage mutable borrows below
macro_rules! split_borrow {
    ($instance:expr, $index:ident, $length:ident, $list:ident) => {
        let Self {
            ref mut $index,
            ref mut $length,
            ref $list,
        } = $instance;
    };
}

macro_rules! op_with_critical_section {
    ($instance:expr, $fn:expr) => {{
        split_borrow!($instance, index, length, list);
        crate::sync::with_critical_section(list, || $fn(index, length, list))
    }};
}

impl<'py> Iterator for BoundListIterator<'py> {
    type Item = Bound<'py, PyAny>;

    fn fold<B, F>(mut self, init: B, mut f: F) -> B
    where
        Self: Sized,
        F: FnMut(B, Self::Item) -> B,
    {
        op_with_critical_section!(self, |index: &mut Index, length: &mut Length, list| {
            let mut accum = init;
            while let Some(x) = unsafe { Self::next_unchecked(&mut *index, &mut *length, list) } {
                accum = f(accum, x);
            }
            accum
        })
    }
}

Though at this point I'm not sure the macro redirections makes it more readable anymore...

[davidhewitt]

Personally I'd drop the macros and suggest we have

impl<'py> BoundListIterator<'py> {
    fn with_critical_section<R>(&mut self, f: impl FnOnce(&mut Index, &mut Length, &Bound<'py, PyList>) -> R) -> R {
        let Self { index, length, list } = self;
        crate::sync::with_critical_section(list, || $fn(index, length, list))
}

[mejrs]

Is it really safe to call next_unchecked here (and elsewhere)? Can't the closure modify the list?

[ngoldbaum]

More thoughts about thread safety here:

• It is possible for another thread to try to acquire a critical section on the list, but we hold a critical section here so that thread will block until this thread exits the critical section.
• That means the index and length are correct going into f, despite us getting them without any synchronization in next_unchecked.
• 99% of the time, the critical section never getd released. It does get released if f creates a new innermost critical section on the list, but then only this thread can access the list still. It is possible that fold is getting called recursively, in which case this thread would create new innermost critical sections until the recursion terminates.

(did some edits of the text above to drop irrelevant references to the GIL)

[ngoldbaum]

@mejrs if a closure updates the length and the old length stored in the iterator is out-of-bounds, this step means the index.0 < length check below is False so the iterator returns None and the iteration terminates.

We hold a critical section so other threads can't modify the list between here and the next get_item_unchecked call.

Does that make sense? Are you worried about other scenarios?

In any case, I'll try to add a test where the closure modifies the list to see what happens....

[ngoldbaum]

See test_iter_fold_out_of_bounds added in the last commit.

[davidhewitt]

Thanks, this looks like it's needed before we can land #4810, additionally it'd be great to include this in 0.23.4. Sorry for the very slow review!

I think main suggestions from me is that we can simplify away the macros and also we should probably use the _unchecked paths in PyPy.

[davidhewitt]

Suggested change

	#[cfg(not(any(Py_LIMITED_API)))]
	#[cfg(not(Py_LIMITED_API))]

[davidhewitt]

Personally I'd drop the macros and suggest we have

impl<'py> BoundListIterator<'py> {
    fn with_critical_section<R>(&mut self, f: impl FnOnce(&mut Index, &mut Length, &Bound<'py, PyList>) -> R) -> R {
        let Self { index, length, list } = self;
        crate::sync::with_critical_section(list, || $fn(index, length, list))
}

[davidhewitt]

Especially if we use my suggestion from above to avoid macros on the critical sections, I think this can just be:

Suggested change

	split_borrow!(self, index, length, list);
	let Self { index, length, list } = self;