Skip to content

perf(receipt): pre-size the offline-aggregation dedup map to avoid rehashing - #908

Closed
jlucaso1 wants to merge 1 commit into
mainfrom
claude/gallant-shannon-szot51
Closed

jlucaso1 wants to merge 1 commit into
mainfrom
claude/gallant-shannon-szot51

Conversation

@jlucaso1

@jlucaso1 jlucaso1 commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator

What

group_delivery_receipts aggregates buffered offline messages into one <receipt> per group (WA Web's sendAggregateOfflineReceipts). It walks the full receipt batch building a per-group dedup HashMap<Key, usize> keyed by the receipt-level attrs (to / participant / type / recipient).

The map was HashMap::new(), so on reconnect — when the batch can carry a large offline backlog — it rehashed repeatedly as it grew (each resize re-inserts every live entry). Pre-size it to the receipt count (infos.len()), an upper bound on the distinct keys, so it reaches its final capacity in one allocation.

The companion groups Vec is deliberately left growth-sized: it's the returned value and grouping can shrink it far below infos.len() (many messages collapse into a few chats), so pre-sizing it to the input count would leave an over-allocated Vec alive past the call. The map has no such issue — it's dropped when the function returns, so a heavy-grouping over-allocation is purely transient.

Why it's correct

Pure pre-sizing: HashMap::with_capacity(n) is behavior-identical to HashMap::new() — same entries, same lookup/insert semantics, only the initial capacity differs. No logic, ordering, or key handling changes, and infos.len() is always ≥ the number of distinct keys, so the hint never under-sizes.

Tests

No behavior to pin — covered by the existing receipt suite. cargo test -p whatsapp-rust --lib receipt (81) green; cargo fmt --all + cargo clippy clean.

Impact

Removes the incremental rehashing of the dedup map on the reconnect / offline-aggregation path; the win scales with backlog size and is nil for the common small-batch case (the map already fit). Modest and allocation-side — the rest of the path was already lean.

…hashing

group_delivery_receipts builds a per-group dedup HashMap by walking the full
receipt batch. On reconnect that batch can carry a large offline backlog, so the
map rehashed repeatedly as it grew. Pre-size it to the receipt count; it is
dropped at function end, so the transient over-allocation under heavy grouping
never outlives the call. `groups` stays growth-sized since it is returned and
grouping can shrink it sharply.
@coderabbitai

coderabbitai Bot commented Jun 19, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 11eb40ea-6291-4a47-b1b8-27b016210368

📥 Commits

Reviewing files that changed from the base of the PR and between 8d4d584 and 40c868a.

📒 Files selected for processing (1)
  • src/receipt.rs

📝 Walkthrough

Summary by CodeRabbit

  • Chores
    • Improved performance when handling large offline backlogs by optimizing internal data structure initialization.

Walkthrough

In group_delivery_receipts, the per-group deduplication HashMap initialization is changed from HashMap::new() to HashMap::with_capacity(infos.len()). Two comments are added explaining the transient over-allocation tradeoff and the goal of avoiding mid-grow rehashing during large offline receipt backlogs.

Changes

HashMap Pre-allocation in Receipt Grouping

Layer / File(s) Summary
HashMap capacity pre-allocation
src/receipt.rs
group_delivery_receipts initializes the dedup map with HashMap::with_capacity(infos.len()) instead of HashMap::new(), and adds inline comments about avoiding mid-grow rehashing for offline backlogs.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

Possibly related PRs

  • oxidezap/whatsapp-rust#820: Directly modifies the same group_delivery_receipts function in src/receipt.rs, substantially reworking the offline receipt aggregation and grouping logic that this PR further optimizes.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically describes the main change: pre-sizing a HashMap to avoid rehashing during offline message aggregation, which is exactly what the PR does.
Description check ✅ Passed The description is comprehensive and directly related to the changeset, explaining the optimization, its correctness, test coverage, and performance impact.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/gallant-shannon-szot51

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 40c868ab8e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/receipt.rs
// over-allocation when grouping is heavy never outlives the call. `groups` is
// returned and grouping can shrink it sharply, so it's left to grow.
let mut index: std::collections::HashMap<Key, usize> =
std::collections::HashMap::with_capacity(infos.len());

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid preallocating receipt groups by message count

In the offline flush path, flush_offline_receipts drains the entire replay buffer and then calls this grouping helper; when a reconnect replays many messages from the same chat/sender, groups.len() can be 1 while infos.len() is thousands. This line now forces the dedup map to allocate for every message before discovering they collapse into one group, whereas the previous lazy map stayed small in that common aggregate case, so a large catch-up can burn or OOM transient memory before any receipts are sent. Prefer leaving the map lazy or reserving only as groups are added.

Useful? React with 👍 / 👎.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

Re-trigger cubic

@codspeed-hq

codspeed-hq Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 179 untouched benchmarks


Comparing claude/gallant-shannon-szot51 (40c868a) with main (8d4d584)

Open in CodSpeed

@github-actions

Copy link
Copy Markdown

📦 Binary size report

Metric main PR Δ
bin size (stripped) 10.59 MiB 10.59 MiB -64 B (-0.00%) 🔽
bin .text 8.70 MiB 8.70 MiB -64 B (-0.00%) 🔽
bin allocated (text+data+bss) 10.59 MiB 10.59 MiB 0
llvm-lines wacore 640,205 640,205 0
llvm-lines wacore copies 17,673 17,673 0
llvm-lines whatsapp-rust lib 655,094 655,131 +37 (+0.01%) 🔺
llvm-lines whatsapp-rust lib copies 20,164 20,166 +2 (+0.01%) 🔺
deps crates (Cargo.lock) 347 347 0
.text per crate
Crate main PR Δ
.text whatsapp_rust 1.44 MiB 1.44 MiB -16 B (-0.00%) 🔽
.text wacore 520.75 KiB 520.75 KiB 0
.text wacore_binary 156.73 KiB 156.73 KiB 0
.text wacore_libsignal 166.63 KiB 166.63 KiB 0
.text wacore_appstate 36.98 KiB 36.98 KiB 0
.text wacore_noise 27.71 KiB 27.71 KiB 0
.text waproto 876.21 KiB 876.21 KiB 0
.text whatsapp_rust_sqlite_storage 207.48 KiB 207.48 KiB 0
.text whatsapp_rust_tokio_transport 32.49 KiB 32.49 KiB 0
.text whatsapp_rust_ureq_http_client 5.93 KiB 5.93 KiB 0
.text std 1.13 MiB 1.13 MiB 0
.text other deps 4.07 MiB 4.07 MiB 0
Top movers (cargo-bloat attribution)
Crate main PR Δ
prost 471.26 KiB 472.66 KiB +1.39 KiB (+0.30%)

Baseline: 8d4d58409 (latest main run) · Head: 68bbd983a · Graphs

@jlucaso1 jlucaso1 closed this Jun 19, 2026
@jlucaso1
jlucaso1 deleted the claude/gallant-shannon-szot51 branch July 3, 2026 02:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants