Labels: bug, safety, documentation, priority: critical
Summary
mohu's core design pillar is "Zero-copy Python integration via PyO3" combined with "Parallel operations by default via Rayon". This specific combination is one of the most well-known soundness hazards in the entire Rust/Python FFI ecosystem, and the README's own architecture description — Arrow-backed arrays shared between Python and Rust with zero-copy semantics and Rayon parallelism — is precisely the scenario where the PyO3 and Rayon maintainers explicitly warn contributors.
The specific unsoundness risk: if a PyReadonlyArray (or any buffer borrowing from a Python object's memory) is passed to a Rayon parallel iterator without holding the GIL across the entirety of the parallel region, another Python thread can trigger the garbage collector which may move or free the underlying buffer while Rayon workers are reading it. PyO3 requires either: (a) the GIL is held for the duration of any access to Python-owned memory, OR (b) the data is explicitly copied into Rust-owned memory before releasing the GIL. Zero-copy + parallel is only safe when the buffer is provably owned by Rust (e.g. a Box<[T]> you allocated yourself) for the duration of the parallel operation.
Since examples/ and any user-facing API surface are the first thing contributors will reference when implementing new ops, even a single example showing direct parallel iteration over a Python-borrowed buffer without these invariants documented would propagate unsound patterns across every GSSoC contributor adding new operations.
Why this matters
This is a Rust UB (undefined behavior) risk, not a performance or correctness edge case. Rust's safety guarantees do not extend across FFI boundaries — and PyO3 specifically documents (in its parallelism guide) that releasing the GIL while holding a borrow from Python-managed memory is unsafe and the programmer's full responsibility. For a project positioning itself as "the array library Python deserves" with explicit safety as a differentiator over C-based NumPy, shipping unsound FFI patterns in examples directly contradicts the core value proposition.
Proposed solution
- Add a
docs/SAFETY.md document explicitly describing the invariants contributors must uphold when implementing parallel ops: (1) copy-to-Rust-owned memory before py.allow_threads(), (2) never pass a PyReadonlyArray's raw data pointer into a rayon::par_iter without first materializing into a Vec<T> or an Arc-wrapped Arrow buffer with a Rust-controlled lifetime, (3) reference the PyO3 parallelism docs section explicitly.
- Audit the existing
crates/ and examples/ for any current instances of this pattern and add // SAFETY: <invariant> comments documenting why each unsafe block is sound.
- Add a Clippy lint (
#[deny(clippy::missing_safety_doc)]) to the workspace Cargo.toml and wire it into the CI pipeline to enforce safety documentation on all future unsafe blocks.
I'd like to be assigned this issue. I'd start with the docs/SAFETY.md and the Clippy CI lint (additive, zero behavior change) as the first PR, then audit existing unsafe blocks in the crates as a documented follow-up.
Labels: bug, safety, documentation, priority: critical
Summary
mohu's core design pillar is "Zero-copy Python integration via PyO3" combined with "Parallel operations by default via Rayon". This specific combination is one of the most well-known soundness hazards in the entire Rust/Python FFI ecosystem, and the README's own architecture description — Arrow-backed arrays shared between Python and Rust with zero-copy semantics and Rayon parallelism — is precisely the scenario where the PyO3 and Rayon maintainers explicitly warn contributors.
The specific unsoundness risk: if a
PyReadonlyArray(or any buffer borrowing from a Python object's memory) is passed to a Rayon parallel iterator without holding the GIL across the entirety of the parallel region, another Python thread can trigger the garbage collector which may move or free the underlying buffer while Rayon workers are reading it. PyO3 requires either: (a) the GIL is held for the duration of any access to Python-owned memory, OR (b) the data is explicitly copied into Rust-owned memory before releasing the GIL. Zero-copy + parallel is only safe when the buffer is provably owned by Rust (e.g. aBox<[T]>you allocated yourself) for the duration of the parallel operation.Since
examples/and any user-facing API surface are the first thing contributors will reference when implementing new ops, even a single example showing direct parallel iteration over a Python-borrowed buffer without these invariants documented would propagate unsound patterns across every GSSoC contributor adding new operations.Why this matters
This is a Rust UB (undefined behavior) risk, not a performance or correctness edge case. Rust's safety guarantees do not extend across FFI boundaries — and PyO3 specifically documents (in its parallelism guide) that releasing the GIL while holding a borrow from Python-managed memory is
unsafeand the programmer's full responsibility. For a project positioning itself as "the array library Python deserves" with explicit safety as a differentiator over C-based NumPy, shipping unsound FFI patterns in examples directly contradicts the core value proposition.Proposed solution
docs/SAFETY.mddocument explicitly describing the invariants contributors must uphold when implementing parallel ops: (1) copy-to-Rust-owned memory beforepy.allow_threads(), (2) never pass aPyReadonlyArray's raw data pointer into arayon::par_iterwithout first materializing into aVec<T>or an Arc-wrapped Arrow buffer with a Rust-controlled lifetime, (3) reference the PyO3 parallelism docs section explicitly.crates/andexamples/for any current instances of this pattern and add// SAFETY: <invariant>comments documenting why each unsafe block is sound.#[deny(clippy::missing_safety_doc)]) to the workspaceCargo.tomland wire it into the CI pipeline to enforce safety documentation on all futureunsafeblocks.I'd like to be assigned this issue. I'd start with the
docs/SAFETY.mdand the Clippy CI lint (additive, zero behavior change) as the first PR, then audit existingunsafeblocks in the crates as a documented follow-up.