testing: `benches/` directory exists but there is no documented baseline benchmark result, no CI performance regression gate, and no pinned Rust toolchain version for reproducible bench output

[divyanshim27]

Labels: testing, dx, priority: medium

Summary

The repository has a dedicated benches/ directory — a strong signal that performance benchmarking is intended and already partially set up (likely Criterion.rs given the Rust ecosystem standard). However, nothing in the README.md, CONTRIBUTING.md, or any doc file:

1. Shows a reference baseline benchmark result (e.g. "mohu array creation: 1.2µs, NumPy equivalent: 4.8µs") that communicates the performance story to potential contributors and users.
2. Documents how to run the benchmarks locally (cargo bench invocation, any required features flags).
3. Gates CI on a performance regression check — so a contributor could introduce a 10x regression in a core op and the PR would pass CI with no signal.
4. Enforces a reproducible Rust toolchain version for bench runs — rust-toolchain.toml exists per the file tree (good), but there's no documented pinning of LLVM optimization flags or target CPU baseline that guarantees cross-machine comparable numbers.

Why this matters

For a project whose entire raison d'être is "parallel by default, faster than NumPy" — and which explicitly compares itself to Polars' performance bet on Rust — having benchmarks in the repo with no baseline numbers and no regression gate is the single biggest missing credibility signal for potential contributors evaluating whether the project is actually fast yet, and for maintainers catching performance regressions in PRs.

Proposed solution

• Run cargo bench from the current benches/ and capture a baseline result table (Criterion generates HTML and JSON natively — commit the JSON baseline file under benches/baselines/).
• Add a README.md section "Performance Benchmarks" with the current baseline numbers and a cargo bench invocation example, explicitly noting the test hardware for context.
• Add a CI step using critcmp (Criterion's comparison tool) or cargo-bench-compare that compares PR benchmark output against the committed baseline and posts a comment on PRs showing the delta — flagging regressions above a configurable threshold (e.g. >5% slowdown on core ops) for human review.
• Document the intended comparison targets (NumPy operation equivalents by name) in benches/README.md so future contributors know what to benchmark against.

I'd like to be assigned this issue. The baseline capture + CI critcmp setup is additive CI infrastructure with no code changes — ideal as a first PR that has immediate high-visibility impact.

[Roopathanushree]

/assign gssoc

[github-actions]

Usage: /assign. Contributors can only assign themselves.

[Alishaa-987]

assign

[aaniya22]

/assign

[github-actions]

Assigned to @aaniya22.