perf(binary): inflate into uninitialized buffer, drop the zero-init memset

[jlucaso1]

What

Removes an avoidable memset from the history-sync decompression path. The binary decoder's zlib inflate spends ~6% of bench_process_history_sync (~2ms) zero-initializing the output buffer before every inflate call — bytes that inflate then overwrites immediately.

Root cause

flate2::Decompress::decompress_vec writes into the vector's spare capacity via the backend's decompress_uninit. flate2's zlib-rs backend doesn't override decompress_uninit, so it falls back to the trait default (initialize_buffer → write_bytes(0, len)), which memsets the entire spare region first. That zeroing is pure waste for inflate, which produces exactly the bytes it writes. The flate2 docs themselves call this out and point to the lower-level uninit API.

This showed up clearly in the deterministic Simulation flamegraph: __memset_avx2_unaligned_erms called from <flate2::mem::Decompress>::decompress_vec, 6% self time, dwarfed only by the inflate core itself.

How

Drive zlib_rs::Inflate directly in zlib_pool.rs and decompress into Vec::spare_capacity_mut() through decompress_uninit, then bump the length by the produced count (set_len). No zeroing. A small inflate_into_spare helper wraps the uninit + set_len so both call sites (the streaming InflateReader::pump and the one-shot decompress_zlib_pooled) share it.

zlib-rs is already in the dependency tree (transitively via flate2), is no_std and wasm/esp32-safe, and is pulled with the same std + rust-allocator features flate2 already selects for it — so no new external crate and no wasm regression. The format is unchanged (still zlib/deflate). flate2 moves to dev-dependencies since only the test/bench compression fixtures still use it (ZlibEncoder).

Why measure

The win is real and deterministic in the flamegraph, but it's ~6% of a path dominated by the inflate core (~25%) and buffer copies, so the end-to-end move on bench_process_history_sync will be modest. Opening non-draft so CodSpeed measures it and the AI reviewers run. libdeflate (a ~2x faster decoder) was considered and rejected: it's C (breaks the wasm32/esp32 builds this crate must support) and one-shot only (breaks the streaming reader).

Verification

• cargo test -p wacore-binary — 99 unit + roundtrip/proptest pass, incl. all 8 zlib pool tests (cross-chunk streaming, high-ratio expansion, reuse-after-error, max-size enforcement, oversized-buffer shrink)
• cargo clippy -p wacore-binary --all-targets -- -D warnings — clean
• cargo check --workspace — clean
• cargo shear — no new unused-dependency findings (flate2/zlib-rs both clean)

https://claude.ai/code/session_01NGEfhAP41Csiy7ptQWZrf3

[coderabbitai]

📝 Walkthrough

Summary by CodeRabbit

• Bug Fixes

  • Improved compressed-data handling by migrating the pooled zlib decompression backend, enhancing reliability while keeping streamed behavior and existing size limits.
  • Updated error mapping to treat invalid compressed input consistently during decompression.

• Chores

  • Refreshed and aligned workspace and binary component dependencies, replacing prior decompression-related dependencies with a unified zlib-based setup.

Walkthrough

This PR swaps the workspace and wacore/binary zlib backend from flate2 to zlib-rs. It updates dependency declarations and rewrites pooled, streaming, and one-shot decompression paths to use zlib_rs::Inflate.

Changes

zlib-rs decompression migration

Layer / File(s)	Summary
Dependency manifest updates Cargo.toml, wacore/binary/Cargo.toml	Adds zlib-rs (0.6.5, default-features off, std/rust-allocator features) to workspace and crate deps, removes flate2 from dev-dependencies, and reorders adjacent dependency entries.
Core Inflate types and inflate_into_spare helper wacore/binary/src/zlib_pool.rs	Replaces flate2 types with zlib_rs::Inflate, adds ZLIB_HEADER/WINDOW_BITS constants, changes InflateReader.decomp field type, adds inflate_into_spare using decompress_uninit and total_out deltas, and updates grow_by_observed_ratio to accept &Inflate.
InflateReader streaming path wacore/binary/src/zlib_pool.rs	Constructor now builds/resets Inflate via ZLIB_HEADER/WINDOW_BITS; pump switches from decompress_vec to inflate_into_spare with InflateFlush::NoFlush and remaps errors to InvalidData.
One-shot decompress_zlib_pooled path wacore/binary/src/zlib_pool.rs	Pooled decompressor reset via reset(ZLIB_HEADER), replaces decompress_vec(..., Finish) with inflate_into_spare(..., InflateFlush::Finish), updates error mapping, and adjusts a cap-enforcement comment.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

• oxidezap/whatsapp-rust#672: Both PRs modify wacore/binary/src/zlib_pool.rs around the pooled inflate path and InflateReader streaming behavior.
• oxidezap/whatsapp-rust#683: Both PRs change the same pooled decompression lifecycle in wacore/binary/src/zlib_pool.rs, including reset and pump handling.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly matches the main change: switching binary inflate to an uninitialized buffer and removing zero-init memset.
Description check	✅ Passed	The description is directly related and accurately explains the decompression optimization, motivation, implementation, and verification.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch perf/zlib-inflate-uninit

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@Cargo.toml`:
- Line 105: The zlib-rs dependency is pinned to an unpublished version, so Cargo
cannot resolve it. Update the dependency entry in Cargo.toml to a
crates.io-published zlib-rs release while keeping the existing feature flags
unchanged, and verify the manifest still points to the same dependency symbol
zlib-rs.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI (base), Organization UI (inherited)

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 16939dba-0659-4286-b214-32b0562bffc0

📥 Commits

Reviewing files that changed from the base of the PR and between 5b99262 and e4a8b18.

⛔ Files ignored due to path filters (1)

• Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (1)

• Cargo.toml

[codspeed]

Merging this PR will not alter performance

✅ 179 untouched benchmarks

---

_{Comparing perf/zlib-inflate-uninit (e4a8b18) with main (95bec41)}

[github-actions]

📦 Binary size report

Metric	main	PR	Δ
bin size (stripped)	10.11 MiB	10.11 MiB	-1.44 KiB (-0.01%) 🔽
bin .text	8.17 MiB	8.17 MiB	-832 B (-0.01%) 🔽
bin allocated (text+data+bss)	10.11 MiB	10.11 MiB	-4.21 KiB (-0.04%) 🔽
llvm-lines wacore	644,769	644,764	-5 (-0.00%) 🔽
llvm-lines wacore copies	17,882	17,880	-2 (-0.01%) 🔽
llvm-lines whatsapp-rust lib	658,443	658,443	0
llvm-lines whatsapp-rust lib copies	20,447	20,447	0
deps crates (Cargo.lock)	467	467	0

.text per crate

Crate	main	PR	Δ
.text whatsapp_rust	1.49 MiB	1.49 MiB	+292 B (+0.02%) 🔺
.text wacore	530.68 KiB	530.62 KiB	-64 B (-0.01%) 🔽
.text wacore_binary	157.54 KiB	157.84 KiB	+311 B (+0.19%) 🔺
.text wacore_libsignal	165.86 KiB	165.86 KiB	0
.text wacore_appstate	144.24 KiB	144.24 KiB	0
.text wacore_noise	27.71 KiB	27.71 KiB	0
.text waproto	871.99 KiB	871.99 KiB	0
.text whatsapp_rust_sqlite_storage	475.72 KiB	475.72 KiB	0
.text whatsapp_rust_tokio_transport	43.57 KiB	43.57 KiB	0
.text whatsapp_rust_ureq_http_client	8.81 KiB	8.81 KiB	0
.text std	998.84 KiB	998.17 KiB	-692 B (-0.07%) 🔽
.text other deps	3.28 MiB	3.28 MiB	-546 B (-0.02%) 🔽

Top movers (cargo-bloat attribution)

Crate	main	PR	Δ
flate2	14.85 KiB	(removed)	-14.85 KiB (-100.00%)
zlib_rs	15.89 KiB	30.63 KiB	+14.74 KiB (+92.79%)

Baseline: 95bec41ab (latest main run) · Head: 79d497529 · Graphs