Labels: architecture, enhancement, priority: high
Summary
mohu's documented design pillars are: Rayon parallelism, Arrow-native memory layout, and zero-copy Python interop. The current retrieval infrastructure (per vector_store.py committed at the repo root) uses FAISS — Facebook's C++ vector index library — as the vector search backend.
FAISS is the most widely adopted ANN library but it directly conflicts with mohu's stated design goals in three specific ways:
- Not Arrow-native: FAISS operates on contiguous float32 NumPy arrays with its own C++ memory layout. To use FAISS from mohu's Arrow-backed array infrastructure requires an explicit copy (Arrow column → NumPy contiguous → FAISS) at the index boundary, directly breaking the "zero-copy" invariant mohu is designed around.
- Single-threaded index construction by default: FAISS's Python bindings are single-threaded for index building (
IndexFlatL2, IndexIVFFlat) unless you explicitly set faiss.omp_set_num_threads(). This is the exact bottleneck mohu's Rayon parallel foundation is designed to avoid — but since FAISS runs in its own C++ thread pool (OpenMP), it doesn't integrate with Rayon and there's no clean way to make them cooperate efficiently.
vector_store.py at the repository root is a pure-Python file that doesn't use mohu's own array infrastructure at all — it imports NumPy and FAISS directly. This means the current "vector retrieval" code path bypasses mohu entirely, which both undermines the platform's thesis and creates contributor confusion about whether mohu arrays are actually used anywhere in the retrieval pipeline.
Why this matters
For contributors implementing new retrieval ops, vector_store.py is a visible, top-level file that implicitly signals "this is how you do vector search in this project" — and it models the opposite of mohu's stated design pattern.
Proposed solution
- Open a design discussion (GitHub Discussion preferred) on whether to: (a) replace FAISS with
usearch or hnswlib (both are more Rust-friendly and have Arrow-compatible buffer interfaces), (b) wrap FAISS more carefully with explicit Arrow→flat-copy boundaries that are clearly documented as the unavoidable FAISS interop layer, or (c) build a pure-Rust ANN index (e.g. using hora or contributing a Rayon-parallel HNSW implementation as a mohu crate).
- Move
vector_store.py into an examples/faiss_integration/ subdirectory with a README clearly labeling it as an external integration example, not part of mohu's core Arrow infrastructure — reducing contributor confusion about the intended data path.
- Add a
crates/mohu-retrieval/ crate stub (even if empty initially) to claim the namespace and signal where Arrow-native vector search belongs in the workspace architecture.
I'd like to be assigned this issue. I'd scope the first PR as the vector_store.py relocation + a minimal crates/mohu-retrieval/ crate stub with ARCHITECTURE.md documentation explaining the intended zero-copy retrieval design — purely additive, no behavior change.
Labels: architecture, enhancement, priority: high
Summary
mohu's documented design pillars are: Rayon parallelism, Arrow-native memory layout, and zero-copy Python interop. The current retrieval infrastructure (per
vector_store.pycommitted at the repo root) uses FAISS — Facebook's C++ vector index library — as the vector search backend.FAISS is the most widely adopted ANN library but it directly conflicts with mohu's stated design goals in three specific ways:
IndexFlatL2,IndexIVFFlat) unless you explicitly setfaiss.omp_set_num_threads(). This is the exact bottleneck mohu's Rayon parallel foundation is designed to avoid — but since FAISS runs in its own C++ thread pool (OpenMP), it doesn't integrate with Rayon and there's no clean way to make them cooperate efficiently.vector_store.pyat the repository root is a pure-Python file that doesn't use mohu's own array infrastructure at all — it imports NumPy and FAISS directly. This means the current "vector retrieval" code path bypasses mohu entirely, which both undermines the platform's thesis and creates contributor confusion about whether mohu arrays are actually used anywhere in the retrieval pipeline.Why this matters
For contributors implementing new retrieval ops,
vector_store.pyis a visible, top-level file that implicitly signals "this is how you do vector search in this project" — and it models the opposite of mohu's stated design pattern.Proposed solution
usearchorhnswlib(both are more Rust-friendly and have Arrow-compatible buffer interfaces), (b) wrap FAISS more carefully with explicit Arrow→flat-copy boundaries that are clearly documented as the unavoidable FAISS interop layer, or (c) build a pure-Rust ANN index (e.g. usinghoraor contributing a Rayon-parallel HNSW implementation as a mohu crate).vector_store.pyinto anexamples/faiss_integration/subdirectory with a README clearly labeling it as an external integration example, not part of mohu's core Arrow infrastructure — reducing contributor confusion about the intended data path.crates/mohu-retrieval/crate stub (even if empty initially) to claim the namespace and signal where Arrow-native vector search belongs in the workspace architecture.I'd like to be assigned this issue. I'd scope the first PR as the
vector_store.pyrelocation + a minimalcrates/mohu-retrieval/crate stub withARCHITECTURE.mddocumentation explaining the intended zero-copy retrieval design — purely additive, no behavior change.