Skip to content

feat: WA Web phash parity — usync device_hash (#3), group-metadata phash (#7), bcl hash validation (#6) - #679

Merged
jlucaso1 merged 4 commits into
mainfrom
feat/wa-web-phash-parity
Jun 1, 2026
Merged

feat: WA Web phash parity — usync device_hash (#3), group-metadata phash (#7), bcl hash validation (#6)#679
jlucaso1 merged 4 commits into
mainfrom
feat/wa-web-phash-parity

Conversation

@jlucaso1

@jlucaso1 jlucaso1 commented Jun 1, 2026

Copy link
Copy Markdown
Collaborator

Follow-up to #678. The phash audit found three WA Web hash mechanisms we didn't fully replicate. This implements all three, faithful to WA Web where it's safe to be. None was a one-line "add a hash attr": our architecture (query-on-cache-miss + notification-driven updates) lacks the "re-query a known entity with its cached hash" pattern WA Web's hashes rely on, so each needed a small integration addition.

Shared building block: MessageUtils::participant_list_hash (phashV2, fixed to standard base64 in #678).

#3 — usync device_hash on own-device re-sync (feat(usync))

WA Web sends a per-user device_hash in the usync device query so the server returns only CHANGED lists. We never sent it (we query on cache-miss, or invalidate-then-full-query). Now sync_own_device_list sends the cached own device_hash (matching syncMyDeviceList), so an unchanged own device list is skipped server-side on every reconnect.

  • DeviceListSpec gains optional per-user (device_hash, ts) hints → <user><devices device_hash="2:.." ts="N"/></user>.
  • Response processing factored into process_device_list_response, shared with get_user_devices; it only touches users present in the response, so omitted (unchanged) users keep their cache.

whatsmeow omits this optimization entirely; it's WA-Web-specific.

#7 — group-metadata participant phash (feat(group))

WA Web's queryGroup carries a phashV2 of the cached participants so the server answers "not-modified" (absent <group>) for an unchanged group. We always re-queried in full.

  • GroupQueryIq gains an optional phash; Response becomes GroupInfoOutcome { Full(Box<..>), NotModified }.
  • query_info persists the resolved GroupInfo (new ProtocolStore group-metadata blob store: sqlite table + migration, in-memory map, no-op trait defaults for other backends) and on a later cache miss sends the participant phash; on NotModified it reuses the persisted metadata.

#6 — incoming bcl hash validation (feat(recv))

WA Web validates a self-synced broadcast/status: the sending device puts a phashV2 of the recipients in deviceSentMessage.phash; the receiver recomputes and drops on mismatch.

  • validate_bcl_hash + MessageInfo.bcl_participants (populated from <participants><to> for broadcasts only; a group's <participants> is the device fanout, not a bcl).
  • handle_decrypted_plaintext logs a warning on divergence but does not drop — the exact participant hash form is verified by code-reading only, so dropping could lose legitimate messages until validated against a real DSM. Can be flipped to a drop later.

Risk framing

Tests

cargo clippy --all-targets -- -D warnings clean; wacore + whatsapp-rust + sqlite-storage suites green.

jlucaso1 added 3 commits June 1, 2026 14:58
…iceList)

WA Web's usync device query carries a per-user device_hash so the server returns
only CHANGED device lists (omitting unchanged users). We never sent it: our query
paths either query on cache-miss (no hash) or invalidate-then-full-query.

- DeviceListSpec gains optional per-user (device_hash, ts) hints; build_iq emits
  <user jid="..."><devices device_hash="2:.." ts="N"/></user> when present, bare
  otherwise.
- sync_own_device_list stops invalidating; it reads the cached own PN/LID
  device_hash and sends it, so an unchanged own device list is skipped server-side
  on every reconnect (matches syncMyDeviceList). Changed lists are returned and
  applied; omitted (unchanged) users keep their cache.
- Extract the response-processing (LID mappings + record rebuild + batched write)
  into process_device_list_response, shared by get_user_devices and the own re-sync.
  It only touches users present in the response, so omitted users are preserved.

whatsmeow omits this optimization entirely; it is WA-Web-specific. The change is
fail-safe: a wrong/absent hash just yields a full list (today's behavior); the only
risk is the omitted-user case, covered by tests.

Tests: build_iq with/without device_hash; parse omits an unchanged user; and a
merge-safety test proving an omitted user's cached devices are preserved.
…ed (WA Web)

WA Web's queryGroup carries a phashV2 of the cached participant list so the
server answers "not-modified" (absent <group>) for an unchanged group. We always
re-queried in full.

- GroupQueryIq gains an optional phash; build_iq emits
  <query request="interactive" phash="2:.."/>. Its Response becomes
  GroupInfoOutcome { Full(Box<GroupInfoResponse>), NotModified } — NotModified
  when the server omits <group>.
- query_info persists the resolved GroupInfo (serde) via a new ProtocolStore
  group-metadata blob store, and on a later cache miss sends the participant
  phash; on NotModified it reuses the persisted metadata instead of re-parsing.
- New ProtocolStore::get_group_metadata/put_group_metadata with no-op defaults
  (non-breaking for other backends, incl. the test MockBackend); sqlite gets a
  group_metadata table + migration, in_memory a HashMap.

The participant-phash hash form is verified by code-reading only; the change is
fail-safe — a wrong/absent hash just yields a full group (today's behavior), so
the worst case is no skip, never a stale group.

Tests: build_iq with/without phash; parse Full vs NotModified; group_metadata
round-trip (sqlite + in_memory).
…clHash)

WA Web validates the broadcast-contact-list hash on a self-synced
broadcast/status: the sending device puts a phashV2 of the recipients in
deviceSentMessage.phash; the receiver recomputes over its <participants> view
and, on mismatch, drops the message.

We add the recompute + a warning on divergence, but do NOT drop (the exact
participant hash form is verified by code-reading only, so dropping could lose
legitimate messages until validated against a real DSM). This is fail-safe and
can be flipped to a drop later.

- MessageUtils::validate_bcl_hash(participants, expected) compares phashV2.
- parse_message_info collects <participants><to jid> into MessageInfo.bcl_participants
  for broadcast/status stanzas only (a group's <participants> is the device
  fanout, not a bcl, and is skipped).
- handle_decrypted_plaintext logs a warning when a device-sent broadcast's phash
  disagrees with our recomputed hash (before unwrapping the DSM).

Tests: validate_bcl_hash match/reject; broadcast populates bcl_participants; group
fanout participants do not.
@coderabbitai

coderabbitai Bot commented Jun 1, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: febc7453-edd8-4811-959f-1d0f751bda34

📥 Commits

Reviewing files that changed from the base of the PR and between 4b55352 and c08c381.

📒 Files selected for processing (1)
  • wacore/src/messages.rs

📝 Walkthrough

Summary by CodeRabbit

  • New Features

    • Group info caching to avoid re-downloading unchanged group data.
    • Device-list sync now supports per-user hints to skip unchanged users.
    • Broadcast/status message validation added to verify recipient lists.
  • Bug Fixes

    • More robust handling when group responses indicate “not modified”.
    • Device sync preserves omitted (unchanged) users; improved broadcast validation warnings.

Walkthrough

This PR adds persisted group metadata and phash-aware group queries, validates broadcast-contact-list hashes for broadcast/status messages, and optimizes device-list syncs with per-user hash hints and merge-safe response processing.

Changes

Group Metadata Caching & Broadcast BCL Validation

Layer / File(s) Summary
Message BCL Participant Structure & Hash Validation
wacore/src/types/message.rs, wacore/src/messages.rs
MessageInfo adds bcl_participants; MessageUtils::validate_bcl_hash recomputes and compares participant phash.
Broadcast Message BCL Hash Validation
src/message.rs, src/pdo.rs, wacore/src/messages.rs
Broadcast/status parsing populates bcl_participants; device-sent broadcast messages validate phash against recomputed hash and log warnings on mismatch; test helper and PDO path initialize the new field.
Group Query IQ with Optional Phash
wacore/src/iq/groups.rs
GroupQueryIq gains optional phash and returns GroupInfoOutcome to represent Full vs NotModified; phash is conditionally serialized and parsing distinguishes absence of <group>.
Group Metadata Storage Infrastructure
wacore/src/store/traits.rs, wacore/src/store/in_memory.rs, storages/sqlite-storage/src/schema.rs, storages/sqlite-storage/src/sqlite_store.rs, storages/sqlite-storage/migrations/2026-06-01-000000_add_group_metadata/*
Adds ProtocolStore methods get_group_metadata/put_group_metadata (defaults), in-memory map, SQLite migration and Diesel schema, SqliteStore upsert/read implementations, and round-trip tests.
Group Query with Phash & Persistence
src/features/groups.rs
Groups::query_info loads persisted metadata, computes phash, sends GroupQueryIq::with_phash, handles NotModified by returning cached metadata or error if absent, and persists Full responses; get_metadata requires Full responses.

Device List Sync Optimization

Layer / File(s) Summary
DeviceListSpec Hash Hints Contract
wacore/src/iq/usync.rs
DeviceListSpec adds hashes: HashMap<Jid, (String, i64)> and with_hashes constructor; build_iq conditionally emits per-user <devices device_hash ts/>.
Device List Response Merge-Safe Processing
src/usync.rs
Adds process_device_list_response to persist LID→phone mappings, preserve cached device key-index mappings, detect raw_id identity changes, batch-update the registry, and return resolved JIDs only for users present in the response.
Own Device List Re-sync with Hash Optimization
src/usync.rs
sync_own_device_list builds a hashes map from cached records, sends DeviceListSpec::with_hashes, and applies only returned updates via the new helper; includes a test validating omitted users are preserved.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the three main features being implemented: usync device_hash optimization, group-metadata phash support, and bcl hash validation—matching the core objectives.
Description check ✅ Passed The description directly addresses all three features with clear technical rationale, risk assessment, and testing coverage—comprehensively documenting the changeset.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/wa-web-phash-parity

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

github-actions Bot commented Jun 1, 2026

Copy link
Copy Markdown

Benchmark Results

1 regression(s) detected (>2% threshold):

Benchmark Current Baseline Change
libsignal_benchmark::group_messaging_group::bench_group_decrypt_message decrypt:setup_group_with_encrypted_message() 12,725,797 12,460,468 +2.1%
66 unchanged benchmark(s)
Benchmark Current Baseline Change
reporting_token_benchmark::content_extraction_group::bench_content_extraction simple:setup_simple_message() 3,933 3,933 +0.0%
reporting_token_benchmark::content_extraction_group::bench_content_extraction extended:setup_extended_message() 12,038 12,038 +0.0%
reporting_token_benchmark::key_derivation_group::bench_key_derivation 43,514 43,514 +0.0%
reporting_token_benchmark::token_calculation_group::bench_token_calculation 19,365 19,365 +0.0%
reporting_token_benchmark::full_generation_group::bench_full_token_generation simple:setup_full_gen_simple() 68,579 68,579 +0.0%
reporting_token_benchmark::full_generation_group::bench_full_token_generation extended:setup_full_gen_extended() 76,679 76,679 +0.0%
reporting_token_benchmark::message_encoding_group::bench_message_encoding simple:setup_simple_message() 2,230 2,230 +0.0%
reporting_token_benchmark::message_encoding_group::bench_message_encoding extended:setup_extended_message() 5,988 5,988 +0.0%
send_receive_benchmark::dm_send::bench_dm_send text:setup_dm_send() 182,945 182,958 -0.0%
send_receive_benchmark::dm_recv::bench_dm_recv text:setup_dm_recv() 2,204,889 2,204,889 +0.0%
send_receive_benchmark::group_send::bench_group_send group_10:setup_group_send_10() 933,052 932,926 +0.0%
send_receive_benchmark::group_send::bench_group_send group_50:setup_group_send_50() 1,212,508 1,211,442 +0.1%
send_receive_benchmark::group_send::bench_group_send group_256:setup_group_send_256() 2,732,394 2,732,941 -0.0%
send_receive_benchmark::group_send_skdm::bench_group_send_skdm skdm_10:setup_group_skdm_10() 1,143,628 1,143,021 +0.1%
send_receive_benchmark::group_send_skdm::bench_group_send_skdm skdm_50:setup_group_skdm_50() 2,191,169 2,181,367 +0.4%
send_receive_benchmark::group_send_skdm::bench_group_send_skdm skdm_256:setup_group_skdm_256() 7,639,052 7,615,499 +0.3%
send_receive_benchmark::group_recv::bench_group_recv text:setup_group_recv() 12,770,355 12,637,446 +1.1%
binary_benchmark::marshal_group::bench_marshal_allocating 71,296 71,296 +0.0%
binary_benchmark::marshal_group::bench_marshal_auto_allocating 71,349 71,349 +0.0%
binary_benchmark::marshal_group::bench_marshal_exact_allocating 98,416 98,416 +0.0%
binary_benchmark::marshal_group::bench_marshal_reusing_buffer 78,796 78,796 +0.0%
binary_benchmark::marshal_group::bench_marshal_reusing_buffer_vec_writer 71,396 71,396 +0.0%
binary_benchmark::marshal_group::bench_marshal_long_string 7,599 7,599 +0.0%
binary_benchmark::marshal_group::bench_marshal_auto_long_string 7,642 7,642 +0.0%
binary_benchmark::marshal_group::bench_marshal_exact_long_string 9,354 9,354 +0.0%
binary_benchmark::marshal_group::bench_marshal_huge_bytes_allocating 530,581 530,581 +0.0%
binary_benchmark::marshal_group::bench_marshal_auto_huge_bytes_allocating 530,149 530,149 +0.0%
binary_benchmark::marshal_group::bench_marshal_exact_huge_bytes_allocating 531,504 531,504 +0.0%
binary_benchmark::marshal_group::bench_marshal_many_children_allocating 8,506,104 8,506,104 +0.0%
binary_benchmark::marshal_group::bench_marshal_auto_many_children_allocating 8,450,452 8,450,452 +0.0%
binary_benchmark::marshal_group::bench_marshal_exact_many_children_allocating 19,677,941 19,677,941 +0.0%
binary_benchmark::unmarshal_group::bench_unmarshal small:setup_small_marshaled() 2,468 2,468 +0.0%
binary_benchmark::unmarshal_group::bench_unmarshal large:setup_large_marshaled() 33,558 33,558 +0.0%
binary_benchmark::unpack_group::bench_unpack_uncompressed 787 787 +0.0%
binary_benchmark::unpack_group::bench_unpack_compressed 526,830 526,830 +0.0%
binary_benchmark::attr_parser_group::bench_attr_parser attr_lookup:setup_attr_marshaled() 4,990 4,990 +0.0%
binary_benchmark::roundtrip_group::bench_roundtrip small:setup_small_marshaled() 5,315 5,315 +0.0%
binary_benchmark::roundtrip_group::bench_roundtrip large:setup_large_marshaled() 61,874 61,874 +0.0%
binary_benchmark::roundtrip_group::bench_roundtrip_auto small:setup_small_marshaled() 5,347 5,347 +0.0%
binary_benchmark::roundtrip_group::bench_roundtrip_auto large:setup_large_marshaled() 61,942 61,942 +0.0%
binary_benchmark::roundtrip_group::bench_roundtrip_exact small:setup_small_marshaled() 6,734 6,734 +0.0%
binary_benchmark::roundtrip_group::bench_roundtrip_exact large:setup_large_marshaled() 85,585 85,585 +0.0%
binary_benchmark::child_iteration_group::bench_get_children_by_tag 477,570 477,570 +0.0%
binary_benchmark::jid_optimization_group::bench_jid_to_owned_access jid_access:setup_jid_heavy_marshaled() 11,563 11,563 +0.0%
numeric_attr_benchmark::bench_group::bench_baseline_u32 396 396 +0.0%
numeric_attr_benchmark::bench_group::bench_proposed_u32 120 120 +0.0%
numeric_attr_benchmark::bench_group::bench_baseline_u64 439 439 +0.0%
numeric_attr_benchmark::bench_group::bench_proposed_u64 153 153 +0.0%
numeric_attr_benchmark::bench_group::bench_baseline_i64 499 499 +0.0%
numeric_attr_benchmark::bench_group::bench_proposed_i64 162 162 +0.0%
numeric_attr_benchmark::bench_group::bench_baseline_loop_100_u64 44,624 44,624 +0.0%
numeric_attr_benchmark::bench_group::bench_proposed_loop_100_u64 16,424 16,424 +0.0%
libsignal_benchmark::dm_group::bench_dm_session_establishment setup:setup_dm_users() 17,250,713 17,258,940 -0.0%
libsignal_benchmark::dm_group::bench_dm_encrypt_first_message first_msg:setup_dm_session() 157,179 157,179 +0.0%
libsignal_benchmark::dm_group::bench_dm_decrypt_first_message decrypt_prekey:setup_dm_with_first_message() 5,513,975 5,513,975 +0.0%
libsignal_benchmark::dm_group::bench_dm_encrypt_subsequent_message subsequent:setup_established_dm_session() 157,539 157,539 +0.0%
libsignal_benchmark::group_messaging_group::bench_group_create_distribution_message create:setup_group_sender() 296,767 296,767 +0.0%
libsignal_benchmark::group_messaging_group::bench_group_encrypt_message encrypt:setup_group_with_distribution() 706,282 706,282 +0.0%
libsignal_benchmark::conversation_group::bench_full_dm_conversation full:setup_conversation_data() 27,596,366 27,681,702 -0.3%
libsignal_benchmark::signature_group::bench_signature_creation sign:setup_keypair_with_message() 3,467,011 3,467,011 +0.0%
libsignal_benchmark::signature_group::bench_signature_verification verify:setup_keypair_with_message() 124,635,643 124,758,983 -0.1%
libsignal_benchmark::signature_group::bench_key_generation keygen 2,830,452 2,830,452 +0.0%
libsignal_benchmark::session_optimization_group::bench_decrypt_with_previous_session previous_session:setup_with_archived_sessions() 46,566 46,566 +0.0%
libsignal_benchmark::session_optimization_group::bench_out_of_order_decryption out_of_order:setup_out_of_order_messages() 5,197,012 5,197,012 +0.0%
libsignal_benchmark::session_optimization_group::bench_promote_matching_session promote:setup_promote_matching_session() 360,648 360,648 +0.0%
libsignal_benchmark::session_optimization_group::bench_message_key_eviction eviction:setup_message_key_eviction() 14,255,917 14,255,917 +0.0%

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@wacore/src/messages.rs`:
- Around line 57-64: Update the doc comment on validate_bcl_hash to remove the
misleading parenthetical about empty participants and clarify behavior: state
that validate_bcl_hash calls participant_list_hash and returns true only when
the computed hash equals expected (which includes the deterministic hash
computed for an empty participant list), or replace the parenthetical with the
suggested phrasing "returns true when the computed hash matches expected
(including for empty participant lists)". Ensure the comment refers to
validate_bcl_hash and participant_list_hash so readers can find the
implementation.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 706ec815-c684-4f36-b90b-4306ff2185f4

📥 Commits

Reviewing files that changed from the base of the PR and between 1613e56 and 4b55352.

📒 Files selected for processing (14)
  • src/features/groups.rs
  • src/message.rs
  • src/pdo.rs
  • src/usync.rs
  • storages/sqlite-storage/migrations/2026-06-01-000000_add_group_metadata/down.sql
  • storages/sqlite-storage/migrations/2026-06-01-000000_add_group_metadata/up.sql
  • storages/sqlite-storage/src/schema.rs
  • storages/sqlite-storage/src/sqlite_store.rs
  • wacore/src/iq/groups.rs
  • wacore/src/iq/usync.rs
  • wacore/src/messages.rs
  • wacore/src/store/in_memory.rs
  • wacore/src/store/traits.rs
  • wacore/src/types/message.rs

Comment thread wacore/src/messages.rs
The doc said empty participants pass "trivially", which is misleading:
validate_bcl_hash calls participant_list_hash, and an empty list hashes to its
own deterministic value — so it matches only when `expected` is that value, not
unconditionally. Reword and link participant_list_hash so the behavior and impl
are clear.
@jlucaso1
jlucaso1 merged commit 29d4689 into main Jun 1, 2026
11 checks passed
@jlucaso1
jlucaso1 deleted the feat/wa-web-phash-parity branch June 1, 2026 18:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant