Skip to content

Docs/internals architecture #307

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sudha09-git wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Docs/internals architecture #307

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ce881cf
docs: add internals/architecture page
sudha09-git Jul 17, 2026
b6e2ac1
chore: ignore mdbook build output
sudha09-git Jul 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,4 @@ __pycache__/
dist/
*.egg-info/
.DS_Store
docs/book/
90 changes: 90 additions & 0 deletions docs/src/internals/architecture.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Architecture

mohu is organized as a Cargo workspace of layered crates. Each layer only depends on
the layers below it, which keeps compile times low and makes it possible to use
pieces of mohu (e.g. just the error types, or just the dtype system) without pulling
in the whole library.
Comment on lines +3 to +6

@coderabbitai coderabbitai Bot Jul 17, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
rg -n 'mohu-(error|dtype|buffer|array|core|simd|ufunc|index|ops|fft|random|special|stats|sparse|masked|io|testing)' \
  --glob 'Cargo.toml' --glob '!target/**'

Repository: mohu-org/mohu

Length of output: 151

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf 'Top-level Cargo.toml files:\n'
git ls-files '**/Cargo.toml'

printf '\nWorkspace members from root Cargo.toml:\n'
sed -n '1,220p' Cargo.toml

printf '\nDependency lines mentioning mohu crates across Cargo.toml files:\n'
rg -n 'mohu-' --glob 'Cargo.toml' --glob '!target/**' .

Repository: mohu-org/mohu

Length of output: 15259

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Show all Cargo.toml files and their first 200 lines, enough to inspect workspace and dependencies.
while IFS= read -r f; do
  echo "===== $f ====="
  sed -n '1,200p' "$f"
  echo
done < <(git ls-files '**/Cargo.toml')

Repository: mohu-org/mohu

Length of output: 203

🏁 Script executed:

#!/bin/bash
set -euo pipefail

find . -name Cargo.toml -not -path '*/target/*' -print | sort

Repository: mohu-org/mohu

Length of output: 690

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf 'architecture.md:\n'
cat -n docs/src/internals/architecture.md | sed -n '1,120p'

printf '\nCompute/dispatch crate dependencies:\n'
for f in \
  crates/mohu-simd/Cargo.toml \
  crates/mohu-ufunc/Cargo.toml \
  crates/mohu-index/Cargo.toml \
  crates/mohu-ops/Cargo.toml \
  crates/mohu-fft/Cargo.toml \
  crates/mohu-random/Cargo.toml \
  crates/mohu-special/Cargo.toml \
  crates/mohu-stats/Cargo.toml
do
  echo "===== $f ====="
  sed -n '1,120p' "$f"
  echo
done

Repository: mohu-org/mohu

Length of output: 11113

Reword the compute-layer description to match the manifest graph. The workspace layering is fine, but the compute crates do not all route through the dispatch layer: mohu-ops and mohu-stats depend on mohu-core, while mohu-random, mohu-special, and mohu-fft depend on foundation crates directly.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/src/internals/architecture.md` around lines 3 - 6, Update the
layered-crates description in the architecture documentation to match the
manifest dependency graph: state that mohu-ops and mohu-stats depend on
mohu-core, while mohu-random, mohu-special, and mohu-fft depend directly on
foundation crates rather than all routing through the dispatch layer. Preserve
the existing explanation of workspace layering and composable crate usage.


┌─────────────────────────────────────────────────────────────┐
│ I/O & tooling mohu-io · mohu-testing │
├─────────────────────────────────────────────────────────────┤
│ Data structures mohu-sparse · mohu-masked │
├─────────────────────────────────────────────────────────────┤
│ Compute mohu-ops · mohu-fft · mohu-random · │
│ mohu-special · mohu-stats │
├─────────────────────────────────────────────────────────────┤
│ Dispatch & protocol mohu-simd · mohu-ufunc · mohu-index │
├─────────────────────────────────────────────────────────────┤
│ Foundation mohu-error · mohu-dtype · mohu-buffer ·│
│ mohu-array · mohu-core (facade) │
└─────────────────────────────────────────────────────────────┘

## Foundation layer

- **`mohu-error`** has zero dependencies on the rest of the workspace and defines
`MohuError`, `MohuResult<T>`, error codes, and the `bail!`/`ensure!` macros used
everywhere else. Every other crate depends on it, directly or indirectly.
- **`mohu-dtype`** defines the `DType` enum (bool, signed/unsigned ints, floats,
complex) and the type-promotion rules used whenever two arrays of different
dtypes interact.
- **`mohu-buffer`** owns raw allocation, memory layout, and stride arithmetic —
this is where C-order vs. Fortran-order and DLPack interop live.
- **`mohu-array`** builds `NdArray<T>`, the core N-dimensional array type, on top
of `mohu-buffer` and `mohu-dtype`.
- **`mohu-core`** is a thin re-export facade over the four crates above, so
downstream code can depend on one crate instead of four.

## Dispatch & protocol layer

- **`mohu-simd`** provides AVX2/AVX-512/NEON kernel primitives that the compute
layer dispatches into based on CPU features detected at runtime.
- **`mohu-ufunc`** implements the universal-function protocol — broadcasting,
reduce, accumulate, and outer — that NumPy-style element-wise operations are
built on.
- **`mohu-index`** implements advanced indexing: fancy indexing, boolean masks,
and take/put.

## Compute layer

`mohu-ops`, `mohu-fft`, `mohu-random`, `mohu-special`, and `mohu-stats` each own
one area of numerical functionality (arithmetic/broadcasting, FFTs, PRNGs and
distributions, special functions, and descriptive statistics respectively). They
all sit on top of the dispatch layer rather than calling SIMD or ufunc logic
directly themselves, so a new compute crate only needs to implement its own
math, not its own broadcasting or SIMD dispatch.

## Data structure extensions

- **`mohu-sparse`** adds COO/CSR/CSC sparse matrix formats.
- **`mohu-masked`** adds masked arrays for null/invalid value propagation.

Both extend the dense `NdArray<T>` model rather than replacing it.

## I/O & tooling

- **`mohu-io`** handles `.npy`/`.npz`, CSV, and Arrow IPC, including
memory-mapped files.
- **`mohu-testing`** provides shared test fixtures, property-test strategies, and
array-comparison helpers (e.g. `assert_allclose`) used across the workspace's
test suites.

## Python bindings (`mohu-py`, planned)

Python interop is not yet implemented, but the plan (see `ROADMAP.md`) is a
tiered rollout of standard protocols on top of `mohu-buffer`, so that a `mohu`
array works with existing libraries without those libraries knowing mohu
exists:

1. **Buffer protocol / `__array__`** — zero-copy `np.asarray()`, pandas, PIL.
2. **DLPack** — zero-copy exchange with PyTorch, JAX, CuPy, TensorFlow.
3. **Array API standard / `__array_ufunc__` / `__array_function__`** — native
dispatch from scikit-learn, scipy, and NumPy's own function calls.

## Where to look next

- [`CRATE_MAP.md`](../../../CRATE_MAP.md) — full per-crate module and public API
breakdown; the best starting point if you're unsure which crate owns a change.
- [`docs/design/`](../../design/) — architecture decision records for
cross-cutting design questions (dtype system, memory layout).
- [`docs/rfcs/`](../../rfcs/) — larger proposals, such as the public Array API
surface.
Loading