docs(retrieval): add ARCHITECTURE.md and mohu-retrieval crate stub - #302
docs(retrieval): add ARCHITECTURE.md and mohu-retrieval crate stub#302aaniya22 wants to merge 1 commit into
Conversation
…ohu-org#286) - Add ARCHITECTURE.md documenting the retrieval design constraints: mohu's Rayon-parallel / Arrow-native / zero-copy pillars, why FAISS conflicts with them, and the three options under discussion (usearch/hnswlib, wrapped FAISS, or a native Rayon-parallel HNSW crate). - Add an empty crates/mohu-retrieval stub to reserve the namespace and give future contributors on this issue a clear home for the implementation. - No behavior change — purely additive. Note: issue mohu-org#286 also asks to relocate a root-level vector_store.py into examples/faiss_integration/. That file does not exist anywhere in this repo's history (confirmed via git log --all), so there is nothing to relocate. This PR covers the documentation and crate-stub portions only. Signed-off-by: Aaniya <aaniyaatomar@gmail.com>
PR Check Summary
CI will run: build, test, clippy, fmt, cargo-deny, DCO, semver. |
|
Warning Review limit reached
Next review available in: 39 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (4)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
What
Adds
ARCHITECTURE.mddocumenting the retrieval design constraints, and an emptycrates/mohu-retrievalstub to reserve the namespace. Addresses part of #286.Why
mohu's design pillars (Rayon parallelism by default, Arrow-native memory layout, zero-copy Python interop) conflict with FAISS, the most widely-adopted ANN library — it's not Arrow-native and is single-threaded by default. Before any retrieval backend gets implemented, it's worth documenting that conflict and the options under discussion, and giving contributors a clear crate to build toward instead of an ambiguous "where does this go?" question.
How
ARCHITECTURE.md: documents the pillar conflict, and the three options raised in architecture: FAISS chosen as sole vector index but is CPU-bound, single-threaded-by-default, and non-Arrow-native — conflicts with mohu's core "parallel by default, Arrow-native" design pillars #286 (Rust-native ANN library likeusearch/hnswlib, a carefully-wrapped FAISS with an explicit copy boundary, or a native Rayon-parallel HNSW implementation as a mohu crate). No decision is made here — this is scoping, not a design doc for an implementation.crates/mohu-retrieval: empty placeholder crate, registered in the workspace, to reserve the namespace.Important — scope note: #286 also asks to relocate a root-level
vector_store.pyintoexamples/faiss_integration/. I checked (git log --all -- "*vector_store.py*", plus a repo-wide grep forfaiss) and this file doesn't exist anywhere in this repo's history — there's no FAISS code, and nothing to relocate. This PR covers only the documentation and crate-stub portions of the issue. Happy to close out the rest of #286 once there's clarity on whether that file was expected to exist already or is part of unstarted future work.Checklist
cargo test --workspacepassescargo clippy --workspace -- -D warningspassescargo fmt --allappliedCHANGELOG.mdupdated (if user-facing change) — not user-facing, no behavior change