From 8d7ae7856ce46640cf6c7a0bcf8282bce30e19f4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jo=C3=A3o=20Lucas?= <55464917+jlucaso1@users.noreply.github.com> Date: Wed, 5 Aug 2026 14:23:56 -0300 Subject: [PATCH 1/3] docs(signal-protocol): document persisted skipped-key seeds (#1210) SessionMessageKeyMaterial's Seed/Derived variants moved to fixed-width arrays and the seed now round-trips through export instead of only ever coming back as Derived. Update the legacy v1 interop section to reflect that only seedless (pre-#1210) records still hit ChainNotRepresentable::DerivedMessageKey. --- advanced/signal-protocol.mdx | 2103 +--------------------------------- 1 file changed, 1 insertion(+), 2102 deletions(-) diff --git a/advanced/signal-protocol.mdx b/advanced/signal-protocol.mdx index d3171100..2e21f642 100644 --- a/advanced/signal-protocol.mdx +++ b/advanced/signal-protocol.mdx @@ -1,2102 +1 @@ ---- -title: 'Signal Protocol Implementation' -description: 'Deep dive into end-to-end encryption, Double Ratchet algorithm, and Signal Protocol in whatsapp-rust' ---- - -## Overview - -whatsapp-rust implements the Signal Protocol for end-to-end encryption of both one-on-one and group messages. The implementation is based on Signal's libsignal library, adapted for WhatsApp's specific protocol requirements. - - - The Signal Protocol implementation handles cryptographic primitives. Any modifications to this code require expert-level understanding of cryptographic protocols to avoid security vulnerabilities. - - -## Architecture - -The Signal Protocol implementation is split across two main locations: - -- **`wacore/libsignal/`** - Platform-agnostic Signal Protocol core (Rust port of libsignal) -- **`src/store/signal*.rs`** - WhatsApp-specific storage integration with Diesel/SQLite - -### Key Components - -``` -wacore/libsignal/src/ -├── protocol/ -│ ├── session_cipher.rs # Encryption/decryption for 1:1 messages -│ ├── group_cipher.rs # Encryption/decryption for group messages -│ ├── ratchet.rs # Double Ratchet implementation -│ ├── sender_keys.rs # Sender Key protocol for groups -│ └── state/ # Session state management -└── crypto/ - ├── aes_cbc.rs # AES-256-CBC for message content - ├── aes_gcm.rs # AES-GCM for media encryption - └── hash.rs # HKDF and HMAC primitives -``` - -## Double ratchet protocol - -The Double Ratchet algorithm provides forward secrecy and post-compromise security for 1:1 messages. - -### Session Initialization - -Two participants initialize a session using Diffie-Hellman key exchange: - -```rust -// Alice initiates the session (sender) -pub fn initialize_alice_session( - parameters: &AliceSignalProtocolParameters, - csprng: &mut R, -) -> Result - -// Bob receives the session (recipient) -pub fn initialize_bob_session( - parameters: &BobSignalProtocolParameters -) -> Result -``` - -**Key Derivation:** - -1. Compute shared secrets from ephemeral key exchanges -2. Derive root key and chain key using HKDF-SHA256: - ``` - HKDF(discontinuity_bytes || DH1 || DH2 || DH3 [|| DH4]) - → (RootKey[32], ChainKey[32], PQRKey[32]) - ``` -3. Initialize sender and receiver chains - -Location: `wacore/libsignal/src/protocol/ratchet.rs:41-172` - -### Message Encryption - -Each message advances the sender chain and derives ephemeral message keys: - -```rust -// From wacore/libsignal/src/protocol/session_cipher.rs:65-183 -pub async fn message_encrypt( - ptext: &[u8], - remote_address: &ProtocolAddress, - session_store: &mut dyn SessionStore, - identity_store: &mut dyn IdentityKeyStore, -) -> Result -``` - -**Process:** - -1. Load current session state -2. Get sender chain key and derive message keys: - ```rust - let (message_keys_gen, next_chain_key) = chain_key.step_with_message_keys(); - let message_keys = message_keys_gen.generate_keys(); - // message_keys contains: cipher_key, mac_key, iv - ``` -3. Encrypt plaintext with AES-256-CBC: - ```rust - aes_256_cbc_encrypt_into(ptext, message_keys.cipher_key(), - message_keys.iv(), &mut buf) - ``` -4. Create SignalMessage with MAC for authentication -5. Advance chain key and save session state - -**Message Format:** - -- **SignalMessage**: Standard encrypted message -- **PreKeySignalMessage**: Includes prekey bundle for session establishment - - -**Plaintext padding.** Before encryption, the serialized `wa::Message` is padded with a uniform-random number of bytes in `1..=16` (the pad length is repeated as the byte value, matching WA Web's `rand % 16 + 1` and whatsmeow). v0.6 fixed a prior scheme that masked the length with `& 0x0F`, which skewed the distribution toward 15 and could never emit 16 — a subtle fingerprinting divergence from the official client. The receiver strips the padding by reading the final byte as the length. - - -### Message Decryption - -Decryption handles out-of-order delivery and tries multiple session states: - -```rust -// From wacore/libsignal/src/protocol/session_cipher.rs:292-363 -pub async fn message_decrypt_signal( - ciphertext: &SignalMessage, - remote_address: &ProtocolAddress, - session_store: &mut dyn SessionStore, - identity_store: &mut dyn IdentityKeyStore, - csprng: &mut R, -) -> Result> -``` - -**Process:** - -1. Try current session state first -2. If MAC verification fails, try previous (archived) sessions -3. Derive/retrieve message keys for the counter -4. Verify MAC: - ```rust - ciphertext.verify_mac(&their_identity_key, &local_identity_key, - message_keys.mac_key()) - ``` -5. Decrypt with AES-256-CBC -6. Promote successful session to current if needed - - - The implementation optimizes memory by using take/restore patterns to avoid cloning session states during decryption attempts (see `session_cipher.rs:495-619`). - - -### Chain key ratcheting - -Message keys are derived from chain keys, which advance with each message: - -```rust -pub struct ChainKey { - key: [u8; 32], - index: u32, -} - -impl ChainKey { - pub fn step_with_message_keys(self) -> Result<(MessageKeyGenerator, ChainKey)> { - let message_key_gen = MessageKeyGenerator::new(self.key, self.index); - let next_chain_key = self.next_chain_key()?; - Ok((message_key_gen, next_chain_key)) - } -} -``` - -Location: `wacore/libsignal/src/protocol/ratchet/keys.rs` - -### Chain key overflow protection - -The chain key index is a `u32` that increments with each message. Without overflow protection, the index could silently wrap past `u32::MAX` (4,294,967,295) back to 0, creating a counter reuse vulnerability that breaks cryptographic guarantees (nonce reuse in message key derivation). - -Both 1:1 and group chain keys use `checked_add()` to return a typed error instead of wrapping: - -```rust -// 1:1 chain keys (ratchet/keys.rs) -pub fn next_chain_key(&self) -> crate::protocol::Result { - Ok(Self { - key: self.calculate_base_material(Self::CHAIN_KEY_SEED), - index: self.index.checked_add(1).ok_or_else(|| { - SignalProtocolError::InvalidState( - "next_chain_key", - "chain key index overflow (u32::MAX)".to_string(), - ) - })?, - }) -} - -// Group sender chain keys (sender_keys.rs) -let new_iteration = self.iteration.checked_add(1).ok_or_else(|| { - SignalProtocolError::InvalidState( - "sender_chain_key_next", - "Sender chain is too long".into(), - ) -})?; -``` - - - A chain key reaching `u32::MAX` iterations indicates an abnormally long-lived session. In practice this should never occur — ratchet key rotations reset the chain counter with each new Diffie-Hellman exchange. - - -Location: `wacore/libsignal/src/protocol/ratchet/keys.rs`, `wacore/libsignal/src/protocol/sender_keys.rs` - -### Forward Jumps - -The protocol tolerates out-of-order messages up to a limit. Peer sessions and group sender-key chains match WhatsApp Web's `signalFutureMessagesMax`; a pairwise session with one of your **own** other devices gets a wider (but still bounded) ceiling, since multi-device app-sync legitimately jumps far ahead and the peer is trusted: - -```rust -// wacore/libsignal/src/protocol/consts.rs -pub const MAX_FORWARD_JUMPS: usize = 2_000; // peer sessions + group sender keys -pub const MAX_FORWARD_JUMPS_SELF: usize = 25_000; // sessions with your own other devices - -// wacore/libsignal/src/protocol/session_cipher.rs -const fn forward_jump_limit(is_self: bool) -> usize { - if is_self { MAX_FORWARD_JUMPS_SELF } else { MAX_FORWARD_JUMPS } -} - -if jump > forward_jump_limit(state.session_with_self()?) { - return Err(SignalProtocolError::InvalidMessage( - original_message_type, - "message from too far into the future", - )); -} -``` - - -Before this change, self-device sessions were exempt from the limit entirely (jumps beyond `MAX_FORWARD_JUMPS` were logged and allowed through). `MAX_FORWARD_JUMPS_SELF` (25,000) now bounds that path too — still wide enough for legitimate app-sync catch-up, but no longer unbounded. Peer sessions and group sender-key chains dropped from 25,000 to 2,000, matching WA Web's `signalFutureMessagesMax`; a message whose counter is more than 2,000 steps ahead is rejected (driving the retry-receipt path) instead of forcing thousands of KDF derivations per message. - - -Location: `wacore/libsignal/src/protocol/consts.rs`, `wacore/libsignal/src/protocol/session_cipher.rs` - -## DM device fanout - -When sending a direct message, the library resolves all known devices for both the recipient and your own account, then encrypts two different plaintexts for two categories of devices: - -- **Recipient devices** receive the actual message content -- **Own other devices** (your other linked devices) receive a `DeviceSentMessage` wrapper containing the message plus the destination JID, so your other devices can display the sent message in the correct chat - - -**Destination JID encoding via `DsmDestination` ([#1137](https://github.com/oxidezap/whatsapp-rust/pull/1137)).** The `DeviceSentMessage` wrapper writes its destination JID as a length-prefixed protobuf field, which needs the encoded length before the bytes. `wacore::messages::MessageUtils::encode_dm_plaintexts` and `dm_plaintexts_from_encoded` used to take `destination_jid: &str`, so the caller rendered the `Jid` into a `String` purely to measure and copy it. Both now take `impl DsmDestination` instead. `DsmDestination` is implemented directly on `Jid`, so it can measure and write its own wire form without an intermediate `String`. It's also implemented on `str` and the standard string wrappers (`String`, `Box`, `Rc`, `Arc`, `Cow`), carried through references of any depth via two blanket impls. The DM send path now passes `to_jid: &Jid` directly instead of `&to_jid.to_string()`. - - -### Device resolution - -The DM send path builds the full device list in a WA Web-compliant manner (matching `WAWebSendUserMsgJob` and `WAWebDBDeviceListFanout`): - -1. **Local registry first** — the client checks the local device registry via `get_devices_from_registry()` for both the recipient and own account. A network fetch (`get_user_devices`) is only triggered on a cache miss, avoiding unnecessary LID-migration side effects. -2. **Hosted device filtering** — devices flagged as hosted (via `is_hosted()`) are filtered out, matching WA Web's `DBDeviceListFanout` exclusion. -3. **Sender device exclusion** — the exact sender device is removed from the list so `ensure_e2e_sessions` never creates a self-session. This matches WA Web's `isMeDevice` check in `getFanOutList`. -4. **Self-DM deduplication** — when sending to your own account, the recipient and own device lists overlap. A `HashSet`-based dedup pass (matching WA Web's `Map` keyed by `toString`) removes duplicates. - -```rust -// Build device list — local registry first, network on miss -let mut recipient_cached = self.get_devices_from_registry(&recipient_bare).await; -if recipient_cached.is_none() { - let _ = self.get_user_devices(std::slice::from_ref(&to)).await; - recipient_cached = self.get_devices_from_registry(&recipient_bare).await; -} - -// Filter hosted devices, exclude sender, dedup for self-DMs -all_dm_jids.retain(|j| !j.is_hosted()); -all_dm_jids.retain(|j| !is_sender); -// HashSet dedup for self-DM overlap -``` - - -**Per-recipient memoization ([#1118](https://github.com/oxidezap/whatsapp-rust/pull/1118)).** The steps above — the registry lookups, the list rebuild, the partition, and the phash — are memoized per recipient in `dm_devices_memo`, keyed by the resolved wire JID and validated against the device-topology generation and the sending identity (own PN/LID). A warm repeat send to the same chat reuses the stored `ResolvedDmDevices` (an `Arc`, so a hit is a refcount bump — nothing is cloned) instead of redoing this resolution. Any device add/remove/replace, registry invalidation, PN↔LID mapping change, or re-pair invalidates the entry through the same topology tracker the existing `group_devices_memo` already uses; a resolution that had to fall back (e.g. a registry lookup miss) is never memoized. Explicitly requesting network-fresh data bypasses the memo entirely. See `dm_devices_memo` and `group_devices_memo` in [`memory_report()`](/api/client#memory_report). - - -### Device partitioning - -The `partition_dm_devices` function classifies all resolved devices into recipient and own groups, and excludes the exact sender device (the current device) entirely. It partitions `all_devices` in place — swapping recipient devices to the front of the passed-in `Vec` — instead of allocating two new device vectors per send: - -```rust -pub(crate) fn partition_dm_devices( - all_devices: Vec, - own_jid: &Jid, - own_lid: Option<&Jid>, -) -> PartitionedDmDevices - -pub(crate) struct PartitionedDmDevices { - devices: Vec, - recipient_count: usize, -} - -impl PartitionedDmDevices { - // recipient + own, sender excluded - pub(crate) fn valid_devices(&self) -> &[Jid] { - &self.devices - } - - // recipient devices only - pub(crate) fn recipient_devices(&self) -> &[Jid] { - &self.devices[..self.recipient_count] - } - - // own non-sender devices only - pub(crate) fn own_other_devices(&self) -> &[Jid] { - &self.devices[self.recipient_count..] - } -} -``` - -Since [#1118](https://github.com/oxidezap/whatsapp-rust/pull/1118), `partition_dm_devices` runs once per `dm_devices_memo` entry — wrapped by `ResolvedDmDevices::new(all_devices, own_jid, own_lid)` — rather than on every send; see [`DmStanzaRequest` and `ResolvedDmDevices`](#dmstanzarequest-and-resolveddmdevices) below. - -### Sender device exclusion - -The exact sender device is identified by matching both the user **and** device ID against your phone number JID (PN) or your Linked Identity JID (LID): - -```rust -fn is_exact_dm_sender_device(device_jid: &Jid, own_jid: &Jid, own_lid: Option<&Jid>) -> bool { - (device_jid.is_same_user_as(own_jid) && device_jid.device == own_jid.device) - || own_lid.is_some_and(|lid| - device_jid.is_same_user_as(lid) && device_jid.device == lid.device - ) -} -``` - -### Own device recognition - -After excluding the sender device, the remaining devices are classified using `matches_user_or_lid`, which checks if a device JID belongs to the same user as either your PN or LID: - -```rust -pub fn matches_user_or_lid(&self, user: &Jid, lid: Option<&Jid>) -> bool { - self.is_same_user_as(user) || lid.is_some_and(|l| self.is_same_user_as(l)) -} -``` - -This ensures that your own devices registered under your LID (common in multi-device setups) are correctly classified as "own" devices and receive the `DeviceSentMessage` plaintext — not the recipient plaintext. Without LID matching, your own LID-based devices would be misclassified as recipient devices, causing them to receive the wrong message format. - - - Both PN-based and LID-based devices must be checked because WhatsApp's multi-device architecture uses both addressing schemes. A user's devices may appear under either their phone number JID (`@s.whatsapp.net`) or their Linked Identity JID (`@lid`), depending on the device type and registration path. - - -### DmStanzaRequest and ResolvedDmDevices - -`prepare_dm_stanza` takes a `DmStanzaRequest` whose `devices` field is the already-resolved, already-partitioned fan-out for the recipient — a borrowed `&ResolvedDmDevices` — rather than a raw device list: - -```rust -pub struct DmStanzaRequest<'a> { - pub own_jid: &'a Jid, - pub account: Option<&'a wa::ADVSignedDeviceIdentity>, - pub to: &'a Jid, - pub message: &'a wa::Message, - pub message_id: &'a str, - pub edit: Option<&'a crate::types::message::EditAttribute>, - pub extra_nodes: &'a [Node], - /// The already-partitioned fan-out. Borrowed, not owned: the caller's - /// per-recipient memo hands out the same `Arc` on every repeat send, so - /// neither the device list nor its phash is rebuilt here. - pub devices: &'a ResolvedDmDevices, - pub pre_encoded: Option<&'a [u8]>, -} -``` - -`ResolvedDmDevices` (`wacore::send::ResolvedDmDevices`) wraps the partitioned device set from `partition_dm_devices` together with a lazily memoized phash — the type that `dm_devices_memo` caches per recipient: - -```rust -pub struct ResolvedDmDevices { /* partitioned devices + OnceLock phash */ } - -impl ResolvedDmDevices { - /// Partitions `all_devices` into recipient devices and own companions, - /// dropping the sending device itself. - pub fn new(all_devices: Vec, own_jid: &Jid, own_lid: Option<&Jid>) -> Self; - - /// Every device the stanza encrypts for, in partition order. - pub fn devices(&self) -> &[Jid]; - - /// The DM phash over the sent device set: a memo hit is an inline copy. - pub fn phash(&self) -> Option; -} -``` - - - **Breaking change ([#1118](https://github.com/oxidezap/whatsapp-rust/pull/1118)):** `DmStanzaRequest::own_lid` was removed — the sending identity's LID is now baked into the `ResolvedDmDevices` passed via `devices` at construction time (`ResolvedDmDevices::new`) — and `DmStanzaRequest::devices` changed from `Vec` to `&ResolvedDmDevices`. An out-of-tree caller constructing `DmStanzaRequest` directly needs to build a `ResolvedDmDevices` first (via `ResolvedDmDevices::new(all_devices, own_jid, own_lid)`) and pass it by reference. - - -### Session preflight and the 406 case - -Before `prepare_dm_stanza` builds the stanza, `ensure_e2e_sessions` runs as a preflight that fetches prekeys for any resolved device without an established Signal session — one IQ covering up to `SESSION_CHECK_BATCH_SIZE` (50) devices. The server can reject devices two ways: **by name**, returning a `` node whose bundle is replaced with an `` for that one device, or **batch-wide**, failing the IQ itself with a `406` that names nobody. - - -**A named `406` rejection now refreshes only that device's user, not the whole batch ([#1153](https://github.com/oxidezap/whatsapp-rust/pull/1153), closes [#1143](https://github.com/oxidezap/whatsapp-rust/issues/1143)).** `PreKeyUtils::parse_prekeys_response` returns a `PreKeyFetchOutcome { bundles, rejected }` instead of a bare bundle map. `rejected` carries the devices the server named directly in the response body, each with the `` code it was rejected with. When one of those codes is `406`, the preflight calls `invalidate_device_caches_for` with the named device JIDs. That call dedupes the JIDs to their distinct users and refreshes each user's whole device-list cache entry, since the cache is keyed per user rather than per individual device. The send proceeds afterward, and every device that did return a bundle is unaffected — so one rejected device no longer costs the rest of the batch. - -This corrects the premise of [#1139](https://github.com/oxidezap/whatsapp-rust/pull/1139) (closed [#1135](https://github.com/oxidezap/whatsapp-rust/issues/1135)). #1139 assumed the response "doesn't say which one" and so invalidated every distinct user in the batch on any `406`. WA Web's own parser (`FetchKeyBundlesUserError` in `WAWeb/Fetch/PrekeysJob`) already reads this per-device error. whatsapp-rust's parser fed the same node to the bundle parser instead: it failed on the missing fields, logged the failure as a malformed bundle, and discarded the code. The device was skipped, but its stale entry was never refreshed, so the next send resolved the same absent device again. #1139's whole-batch invalidation and propagated `?` error still apply to the one case a named rejection cannot cover: a `406` on the IQ itself. - -The asymmetry #1139 described between the DM and group paths is gone — DM handled at the preflight, group already covered by the per-device fan-out (`stale_device_users`). `SendContextResolver::fetch_prekeys_for_identity_check` now returns `wacore::prekeys::PreKeyFetchOutcome` instead of `HashMap`. If you implement a custom `SendContextResolver`, update this method's return type to match. The new type carries a named rejection into the group fan-out's `EncryptResult::rejected_devices`, so `stale_users_for` refreshes exactly those users instead of inferring staleness from whichever targets went unencrypted for unrelated reasons (missing bundle, malformed bundle, failed session setup). Both paths now act on the same named-device signal when the server provides it. - - -### PreparedDmStanza - -`prepare_dm_stanza` returns a `PreparedDmStanza` struct containing the stanza node and the locally computed phash for server ACK validation: - -```rust -pub struct PreparedDmStanza { - pub node: Node, - /// Locally computed phash from the sent device set. Not sent on the - /// wire (WA Web only sends phash for groups). Used by the caller to - /// compare against the server's ACK phash for device-list drift detection. - pub phash: Option, -} -``` - - - `phash` changed from `Option` to `Option` (`wacore_binary::CompactString`) in [#1118](https://github.com/oxidezap/whatsapp-rust/pull/1118), matching the type `ResolvedDmDevices::phash()` returns: a warm memo hit clones the cached `CompactString` inline instead of allocating a new `String`. - - -The phash is computed from the actual sent device set (after partitioning, with the sender excluded) using `MessageUtils::participant_list_hash()`. Unlike group messages, the DM phash is **not** sent on the wire — WA Web only includes `phash` in the `DeviceSentMessage` for groups. The DM phash is used purely for local validation against the server's ACK to detect device-list drift. - - - The `DeviceSentMessage.phash` field is set to `None` for DMs, matching WA Web's behavior where only group `DeviceSentMessage` wrappers include a phash. The DM phash is computed and tracked separately by the caller. - - -Location: `wacore/src/send.rs:675-820`, `src/send.rs` - -## PN→LID session migration - -WhatsApp's multi-device architecture uses two addressing schemes: phone number JIDs (PN, `@s.whatsapp.net`) and Linked Identity JIDs (LID, `@lid`). WhatsApp Web always resolves PN→LID before any session operation via `createSignalAddress()`. whatsapp-rust mirrors this behavior — when a LID mapping is discovered for a phone number, any Signal sessions stored under the PN address are automatically migrated to the corresponding LID address. - - - The automatic migration described below is also exposed for manual invocation: [`Signal::migrate_sessions(from, to)`](/api/signal#migrate_sessions) runs the same move for a caller-chosen JID pair, and [`Signal::session_info(jid)`](/api/signal#session_info) inspects a session (migrating a legacy PN-addressed one first if needed) without mutating it further. See the [Signal API reference](/api/signal) for both. - - -### Signal address resolution - -`Client::resolve_encryption_jid()` mirrors WA Web's `SignalAddress.toString()` (`WAWeb/Signal/Address.js`). It upgrades the JID's `server` to its LID counterpart when a mapping is known, and otherwise returns the input unchanged: - -| Input `Server` | Resolved `Server` (mapping known) | No mapping | -|----------------|------------------------------------|------------| -| `Pn` | `Lid` | `Pn` (preserved) | -| `Hosted` | `HostedLid` | `Hosted` (preserved) | -| Any other | unchanged | unchanged | - -The `device`, `agent`, and `integrator` fields always round-trip — only the `user` (replaced with the LID user) and `server` change. This keeps Cloud API / Meta Business hosted devices on a hosted-flavored LID address rather than collapsing them into the standard `@lid` server, matching WA Web's per-device session keying. - - - `resolve_encryption_jid()` upgrades PN → LID unconditionally whenever a mapping is known — it governs Signal **session** addressing only, matching WA Web's `SignalAddress.toString()`. The outbound DM **wire** namespace (the stanza `to`, ``, and `DeviceSentMessage` destination) is a separate, account-level decision — see [DM wire namespace vs. Signal session addressing](#dm-wire-namespace-vs-signal-session-addressing) below. - - -### DM wire namespace vs. Signal session addressing - -Since v0.6.x (fix for [#941](https://github.com/oxidezap/whatsapp-rust/issues/941)), a DM's outer `` / `` addressing is no longer derived directly from `resolve_encryption_jid()`. Some accounts are not yet **1:1-LID-migrated** on WhatsApp's servers, and those accounts get every LID-addressed DM rejected with `ack error="400"` even though the underlying Signal session is correctly LID-keyed. - -`Client::resolve_dm_wire_jid()` (`src/client/lid_pn.rs`) makes this account-level decision, mirroring WA Web's `Lid1X1MigrationUtils.isLidMigrated()` / `WAWebMessageDestinationChat`: - -```rust -pub(crate) async fn resolve_dm_wire_jid(&self, to: &Jid) -> Jid { - if self.is_lid_migrated().await { - return self.resolve_encryption_jid(to).await.into_non_ad(); - } - let bare = to.to_non_ad(); - if bare.is_lid() { - self.swap_pn_lid_namespace(&bare).await.unwrap_or(bare) - } else { - bare - } -} -``` - -- **Migrated account** (`Client::is_lid_migrated()` is `true`): behaves exactly like before — the wire namespace upgrades PN → LID whenever a mapping is known. -- **Unmigrated account**: DMs stay on PN even with a cached LID mapping. A caller-supplied LID with a known PN mapping is mapped back to the PN chat; a LID with no cached mapping is sent as-is (there is no reverse LID→PN network resolution, matching WA Web). - -`Client::is_lid_migrated()` is `true` when either is true: - -1. The persisted `Device.lid_migrated` flag (see [Storage — DeviceStore](/concepts/storage#devicestore)), set once from the primary's `pair-success` `` (`isChatDbLidMigrated`) or from a `lid_migration_mapping_sync_message` protocol message pushed to the primary's own companions (self-only — see [Authentication — one-to-one LID migration state](/concepts/authentication#one-to-one-lid-migration-state)). -2. The `lid_one_on_one_migration_enabled` ab prop, as a fallback for sessions paired before the flag existed. The first observation of this prop being on also latches the persisted flag, so the account doesn't flap back to PN addressing before the next props fetch. - -Once set, `lid_migrated` never reverts for the same account — only pairing a *different* account onto the same store resets it. Signal session addressing (`resolve_encryption_jid`) and inbound decrypt are unaffected by any of this; only the outbound DM wire namespace is gated. - - - This gate applies to 1:1 DMs only. Group sends, which already address everything by the group's own `AddressingMode`, are untouched. - - -### Why migration is needed - -After pairing, the primary phone may initially establish sessions under a PN address. Once the LID mapping becomes known (from usync, incoming messages, or device notifications), the phone begins sending from the LID address. Without migration, the client holds a session under the PN address but receives messages addressed to the LID — causing `SessionNotFound` decryption failures. - -### Proactive migration at LID discovery - -When a new LID-PN mapping is learned (via `add_lid_pn_mapping`), the client scans devices 0–99 for PN-keyed sessions and migrates them. All reads and writes go through the `SignalStoreCache` rather than the backend directly — this prevents reading stale data when the cache has unflushed mutations (e.g., after SKDM encryption ratcheted the session). The migrated state is flushed to the backend at the end so it survives restarts. - -```rust -// src/client/lid_pn.rs -pub(crate) async fn migrate_signal_sessions_on_lid_discovery(&self, pn: &str, lid: &str) { - for device_id in 0..=99u16 { - // Read from signal_cache (authoritative over backend) - // If PN session exists and no LID session → move session to LID via cache - // If both exist → delete the stale PN session from cache - // Identity keys are migrated independently of sessions - } - // Flush migrated state to backend so it survives restarts - self.signal_cache.flush(backend.as_ref()).await; -} -``` - -**Migration rules per device:** - -| PN session | LID session | Action | -|-----------|-------------|--------| -| Exists | Does not exist | Move session and identity from PN→LID address | -| Exists | Exists | Delete stale PN session (LID takes precedence) | -| Does not exist | Any | No action | - -Identity keys are migrated independently of sessions — they can outlive deleted sessions and survive session re-establishment. - - - The migration reads through the cache because the backend may contain stale session data when unflushed cache mutations exist. Reading directly from the backend could skip in-flight ratchet advances, causing the migrated session to decrypt with an outdated chain key. - - - - `add_lid_pn_mapping` also has a batch form, `Client::add_lid_pn_mappings(mappings, source)`, which durably records many LID↔PN pairs in one call and runs the same per-mapping migration as the single-entry path. It returns how many mappings were actually written, deduplicated against existing records. - - -### On-the-fly migration during decryption - -If a message arrives from a LID address and decryption fails with `SessionNotFound` or `InvalidPreKeyId`, the client attempts PN→LID migration as a fallback before requesting a retry: - -1. Look up the PN for the sender's LID -2. Attempt to migrate PN sessions to LID via the signal cache (same cache-first logic as proactive migration) -3. Retry decryption with the migrated session (already in the cache — no reload needed) -4. If `DuplicateMessage` occurs during post-migration retry, it is silently ignored -5. Fall back to retry receipt only if migration does not resolve the issue - -The `InvalidPreKeyId` case occurs when a `PreKeyMessage` references a consumed one-time prekey, but the session actually exists under a PN address (legacy migration). Migrating the session lets Signal use the existing ratchet state instead of looking up the consumed prekey. This migration is attempted in both the identity-change retry path and the initial decryption path. - -This ensures existing databases are fixed without requiring re-pairing. - -### Login-time session check - -At login, the client checks the session state of own device 0 (primary phone): - -- **LID session exists** — no action needed -- **PN session only** — logged; migration deferred to first message via on-the-fly path -- **No session** — will be established on first message exchange - -```rust -// src/client/sessions.rs -pub(crate) async fn establish_primary_phone_session_immediate(&self) -> Result<()> { - // Checks LID session → logs PN-only state → defers migration to message path -} -``` - - - Both migration paths route through the `SignalStoreCache`, ensuring they see the latest in-memory state. The proactive migration runs when a LID mapping is first discovered and flushes to the backend afterward. The on-the-fly migration handles the case where the database already contains stale PN sessions from before the mapping was known. - - -Location: `src/client/lid_pn.rs`, `src/client/sessions.rs`, `src/message.rs` - -## Sender keys (group encryption) - -Groups use the Sender Key protocol for efficient multi-recipient encryption. - -### Sender key address normalization - -Sender key records are keyed by a composite `SenderKeyName` containing the group JID and a sender protocol address string. WhatsApp delivers group stanzas with **inconsistent sender addressing** — the `pkmsg` (which carries the SKDM) arrives with a device-qualified participant JID (e.g., `100000000000001.1:75@lid`), while the `skmsg` (the actual encrypted group message) arrives with a bare participant JID (e.g., `100000000000001.1@lid`). - -Without normalization, the sender key would be stored under the device-qualified address during SKDM processing but looked up under the bare address during `skmsg` decryption, causing `NoSenderKeyState` failures. - -The client normalizes the sender JID to its bare form using `to_non_ad()` (which strips the device component, setting `device = 0, agent = 0`) at every point where a `SenderKeyName` is constructed. The `SenderKeyName::from_jid()` convenience method handles the `to_string()` conversion automatically: - -```rust -// Decryption path (src/message.rs) — normalize before group_decrypt -let sender_for_sk = info.source.sender.to_non_ad(); -let sender_address = sender_for_sk.to_protocol_address(); -let sender_key_name = SenderKeyName::from_jid(&info.source.chat, &sender_address); - -// SKDM storage path (src/message.rs) — normalize before process_sender_key_distribution_message -let sender_bare = sender_jid.to_non_ad(); -let sender_address = sender_bare.to_protocol_address(); -let sender_key_name = SenderKeyName::from_jid(&group_jid, &sender_address); -``` - -`SenderKeyName::from_jid()` is equivalent to `SenderKeyName::new(group_jid.to_string(), sender_address.to_string())` but avoids the manual `to_string()` calls and is the preferred constructor. - -This ensures the cache key is always in the form `"{group}:{bare_user}@{server}.0"`, regardless of whether the original stanza used a device-qualified or bare JID. - - - Custom implementations that construct `SenderKeyName` directly must also normalize the sender JID to its bare form. Failing to do so will cause sender key lookup mismatches and decryption failures for group messages. - - -Location: `src/message.rs`, `wacore/libsignal/src/store/sender_key_name.rs`, `wacore/binary/src/jid.rs` (`to_non_ad()`) - -### Sender key distribution - -Each participant generates and distributes a sender key: - -```rust -// From wacore/libsignal/src/protocol/group_cipher.rs:283-336 -pub async fn create_sender_key_distribution_message( - sender_key_name: &SenderKeyName, - sender_key_store: &mut dyn SenderKeyStore, - csprng: &mut R, -) -> Result -``` - -**Structure:** - -- **Chain ID**: Random 31-bit identifier for this sender key session -- **Iteration**: Message counter (starts at 0) -- **Chain Key**: 32-byte seed for deriving message keys -- **Signing Key**: Ed25519 public key for message authentication - -### Group Encryption - -Messages are encrypted with the sender's current chain key: - -```rust -// From wacore/libsignal/src/protocol/group_cipher.rs:53-116 -pub async fn group_encrypt( - sender_key_store: &mut dyn SenderKeyStore, - sender_key_name: &SenderKeyName, - plaintext: &[u8], - csprng: &mut R, -) -> Result -``` - -**Process:** - -1. Load sender key state for the group -2. Derive message keys from current chain key -3. Encrypt with AES-256-CBC -4. Sign message with Ed25519 private key -5. Advance chain key - -### Group Decryption - -Recipients decrypt using the sender's distributed key: - -```rust -// From wacore/libsignal/src/protocol/group_cipher.rs:162-250 -pub async fn group_decrypt( - skm_bytes: &[u8], - sender_key_store: &dyn SenderKeyStore, - sender_key_name: &SenderKeyName, -) -> Result> -``` - -**Process:** - -1. Parse SenderKeyMessage -2. Look up sender key state by chain ID -3. Verify Ed25519 signature -4. Derive message keys for iteration (handling out-of-order) -5. Decrypt with AES-256-CBC - - - Group decryption maintains up to MAX_FORWARD_JUMPS (2,000) cached message keys per sender. This prevents resource exhaustion attacks but limits tolerance for extreme out-of-order delivery. - - -### Unknown device detection - -During group message decryption, the client checks whether the sender's device is present in the local device registry via `is_from_known_device()`. This detection triggers in two places within the group message processing path: - -1. **After successful `skmsg` decrypt** — if the sender device is not in the registry, the decrypted message is still **processed and delivered normally**. Signal decryption success already proves the sender holds a valid session key, so discarding the message would only add latency via an unnecessary retry round-trip. A background device sync is triggered to update the local device registry. -2. **After a `NoSenderKeyState` error** — if the sender device is unknown, the retry reason is upgraded from `NoSession` to `UnknownCompanionNoPrekey` - -In both cases, the client queues a device list synchronization for the sender's user JID. The behavior depends on the connection state: - -- **Online**: the client immediately invalidates the cached device registry for the user and fires a background usync request to refresh the device list -- **Offline** (during offline sync): the unknown device's user JID is batched into a `PendingDeviceSync` set, which is flushed after offline sync completes (see [Deferred device sync](/concepts/architecture#deferred-device-sync)) - -Primary devices (device ID 0) are always treated as known — the check only applies to companion devices. - -This mechanism ensures that group messages from newly-paired companion devices are delivered immediately without waiting for a retry round-trip. The background device sync updates the local registry so future messages from the same device are recognized directly. - -```rust -// src/message.rs — simplified flow -async fn handle_unknown_device_sync(&self, info: &Arc) { - let user_jid = info.source.sender.to_non_ad(); - if !self.pending_device_sync.add(user_jid.clone()).await { - return; // already queued, dedup - } - if info.is_offline { - return; // batched for deferred flush - } - // Online: immediate sync - self.invalidate_device_cache(&user_jid.user).await; - self.get_user_devices(&[user_jid]).await.ok(); -} -``` - -Location: `src/message.rs`, `src/client/device_registry.rs`, `src/pending_device_sync.rs` - -### Retry receipt from unknown group device - -When the client receives a retry receipt, `handle_retry_receipt` checks whether the requesting device is present in the local device registry. Previously the handler dropped all retries from unregistered devices — this was safe for WA Web because WA Web keeps participant device lists fresh via a pre-send sync, so any legitimate requester is already known before the send. - -For a library client, a participant device can legitimately be absent from the local registry: if the device joined between the last device-list sync and the group send, it will have received the `skmsg` from the server but never obtained a sender key, causing it to retry indefinitely. The retry receipt may carry a `` bundle — the ADV-signed `device-identity`, the identity key, a one-time prekey, and the signed prekey — which is everything needed to establish a Signal session and resend. But a newly-linked device that has no bundle still retries forever if the client only drops it: the reconciliation that fires when a prekey fetch returns 406 never triggers for that device because it was never in the send set. - -Whenever a retry arrives from an unknown device, `handle_retry_receipt` now calls `schedule_unknown_device_sync` **before** consulting `should_drop_unknown_device_retry`. This treats the retry as a staleness signal: the requester's user JID is enqueued for a device-list resync (deduplicated via `PendingDeviceSync`, so a retry storm from a single device cannot fan out into a usync storm). Once the resync completes, the device appears in the registry and future sends include it in the sender-key distribution — the retries stop. This mirrors WA Web's `syncDeviceListJob` trigger on the retry path. - -The drop predicate still controls whether the *current* retry is recovered or dropped: - -```rust -// wacore/src/protocol/retry.rs -pub fn should_drop_unknown_device_retry(keys_present: bool, device_known: bool) -> bool { - !keys_present && !device_known -} -``` - -| `keys_present` | `device_known` | Result | -|---|---|---| -| `true` | `false` | **Recover** — build a session from the embedded bundle and resend; resync also triggered | -| `false` | `false` | **Drop** — no bundle to recover this message; device-list resync triggered so device is learned for the next send | -| any | `true` | **Resend** — device is in registry, proceed normally | - -When the bundle includes a ``, `process_retry_key_bundle` validates the ADV chain against the requester's account key (using the stored primary identity as a fallback when the server omits `account_signature_key`). A present-but-invalid ADV result is a hard error; the session is not built. If `` is absent from the bundle, or if no account key can be found, the check is skipped with a warning and the session is built anyway — matching the behaviour of the regular prekey-fetch path. The drop predicate only gates on syntactic `` presence, so the ADV guarantee is conditional on the bundle including a well-formed ``. This mirrors whatsmeow's approach of building the prekey session directly from the receipt bundle without a device-registry gate. - -Location: `src/retry.rs`, `wacore/src/protocol/retry.rs`, `src/pending_device_sync.rs` - -### Immutable sender key loading - -The `SenderKeyStore` trait's `load_sender_key` method takes `&self` (not `&mut self`), allowing sender key lookups to proceed under a **read lock**. This is safe because loading a sender key is a pure read operation — no state is mutated. The `store_sender_key` method still requires `&mut self` since it modifies state. - -This means concurrent group decryptions for different senders can load sender keys in parallel without contention, while writes (SKDM processing) still serialize correctly. - - -If you implement `SenderKeyStore` for a custom backend, `load_sender_key` must use `&self` (immutable reference). Implementations that previously required `&mut self` for internal caching should use interior mutability (e.g., `Mutex` or `RwLock`) instead. - - -### Sender key existence check - -Before distributing sender keys, the group message path checks whether the local sender key already exists. This check uses the `SignalStoreCache` with a **read lock** (`get_sender_key()`), matching the status broadcast path. This avoids acquiring a write lock and prevents unnecessary SKDM re-distribution on every group send. - -### Per-device sender key tracking - -To avoid resending Sender Key Distribution Messages on every group message, the client tracks sender key distribution status **per device** for each group. This uses a unified `sender_key_devices` table (see [Storage - ProtocolStore](/concepts/storage#protocolstore)) that matches WhatsApp Web's `participant.senderKey Map` model — a single boolean per device per group indicating whether that device has a valid sender key (`true`) or needs fresh SKDM distribution (`false`). - -The tracking update is **deferred until after the server acknowledges** the message stanza. This matches WhatsApp Web's behavior where `markHasSenderKey()` is only called after the server confirms receipt. - -**Why deferred?** If the tracking were updated immediately after building the stanza (but before sending), a network failure between stanza build and send would leave stale entries — devices would be marked as having the sender key when they never actually received it. Subsequent messages would skip SKDM for those devices, causing decryption failures. - -**`PreparedGroupStanza` return value:** - -`prepare_group_stanza` returns a `PreparedGroupStanza` struct containing the stanza `node` and a `skdm_devices: Vec` field listing exactly which devices received SKDM in this stanza. This eliminates the need for callers to re-resolve devices after sending, closing a race window where the device list could change between stanza preparation and post-ACK tracking update. - -```rust -pub struct PreparedGroupStanza { - pub node: Node, - /// Devices that received SKDM in this stanza. Empty when no SKDM was distributed. - pub skdm_devices: Vec, -} -``` - -**Implementation:** - -- **Group path:** After `send_node()` succeeds, the caller uses the `skdm_devices` list from `PreparedGroupStanza` to call `set_sender_key_status(group, devices, true)`. No re-resolution needed. -- **Status path:** A late-init boolean tracks whether full distribution occurred. The sender key tracking is only updated after the status stanza is successfully sent. -- **Error recovery:** If `prepare_group_stanza` fails with `NoSenderKeyState`, all sender key device tracking for that group is cleared and the send is retried with full distribution. -- **Sender key rotation:** On `rotateKey`, the Signal sender key is also deleted for forward secrecy (matching WhatsApp Web's `deleteGroupSenderKeyInfo`), and all device tracking is cleared via `reset_sender_key_device_tracking` — a DB-first clear with a cold-mark fallback (see below). -- **Group `` notification (number/LID migration):** A `w:gp2` `` notification (a participant's number or LID changed) unconditionally force-rotates the own group sender key and invalidates both the persisted and in-memory group metadata cache, matching WhatsApp Web's `modifyParticipantInfo` (`rotateKey: true`). The next send regenerates and redistributes a fresh sender key against the current participant list instead of risking a stale entry for the migrated device. See `Client::force_rotate_own_sender_key`, `src/handlers/notification/groups.rs`. - -**Incremental targeting:** - -Rather than distributing the sender key to all group devices on every message, the client: -1. Loads the per-device sender key map — first checking the in-memory cache, falling back to the database via `get_sender_key_devices` -2. Resolves all current group participant devices -3. Computes the diff — only devices with `has_key=false` or not yet tracked receive the SKDM -4. Passes the targeted device list to `prepare_group_stanza` via the `skdm_target_devices` parameter - - -On the **first** group send (or any send where the cached map is empty), the filter still runs unconditionally — every resolved participant device is treated as `has_key=false` and receives the SKDM. This matches WhatsApp Web, which iterates an empty `senderKey` Map as `false` per participant. There is no early-exit for an empty cache; otherwise the very first message after a fresh start would skip distribution entirely. - - - -**Own devices are never marked `has_key=true` ([#999](https://github.com/oxidezap/whatsapp-rust/pull/999)).** The post-ACK warm mark excludes the account's own companion devices, matching WhatsApp Web's `!isMeDevice` guard on `markHasSenderKey`. They therefore never leave the "not yet tracked" bucket above and are re-included as SKDM targets on every send — see the follow-up note under ["Parallelized group encrypt fan-out"](#parallelized-group-encrypt-fan-out) for why. External group members are unaffected: a successful distribution still marks them warm. - - -Location: `src/send.rs`, `src/client/sender_keys.rs`, `wacore/src/send.rs` - -### Parallelized group encrypt fan-out - -The group send path no longer serializes encryption behind a client-level lock. `prepare_group_stanza` and `encrypt_for_devices` now take an explicit `&runtime` handle (`&*self.runtime`) so per-device encryption can run on `runtime::blocking()` tasks concurrently. Combined with the move to `update_device_lists` (batched device-registry writes) and a no-lock `IdentityAdapter::is_trusted_identity` stub, group fan-out scales with the runtime's worker count rather than with a single critical section. - -This is an internal performance change — no public method on `Client::send_message` was renamed, and the order of `` children in the resulting stanza is unchanged. If you implemented a custom `SignalStore`, note that `update_device_lists(records: Vec)` is now part of the trait so the fan-out can batch its writes. - - -While *per-device* encryption runs concurrently, the sender-key chain is protected by **two separate locks per `(group, sender)` pair**: - -1. **Session-setup lock** (`SenderKeyStore::session_setup_lock`) — held only across `ensure_sessions_for_devices` (prekey fetch + X3DH). May span network I/O. Warm sends (no SKDM needed) never take it, so they are never blocked by a cold send's network round-trip. - -2. **Chain lock** (`SenderKeyStore::sender_key_lock`) — held across SKDM creation + pairwise encrypt fan-out + `skmsg` encrypt. Pure CPU; never spans network I/O. This is the invariant that prevents two concurrent sends from splitting the key between the SKDM and the `skmsg`. - -Prior to [#807](https://github.com/oxidezap/whatsapp-rust/pull/807), a single chain lock covered both phases, causing concurrent group sends to serialize behind a server round-trip whenever a new session needed to be established. Now only the CPU phase is in the critical section. Different groups (or different senders) encrypt fully in parallel, unchanged. - -`encrypt_for_devices` is composed of two public halves: `ensure_sessions_for_devices` (network, returns `SessionPlan`) and `encrypt_for_devices_with_sessions` (CPU, consumes `SessionPlan`). The DM path calls `encrypt_for_devices` unchanged; the group path calls them separately with the chain lock taken only around the second. - - - -**Per-device session lock around the SKDM fan-out (v0.6).** The chain lock above only serializes the sender-key chain — it does not cover the *pairwise* Signal sessions that `encrypt_for_devices_with_sessions` mutates for each SKDM target device. Those are the same pairwise sessions the DM path locks (see "DM per-device locking" under [Single-allocation session lock keys](#single-allocation-session-lock-keys) below) via `session_lock_for()` / `session_guards_for()`. Before [#990](https://github.com/oxidezap/whatsapp-rust/pull/990), the group fan-out held only the chain lock, a disjoint key, so a concurrent DM (or another group send) sharing a device could race that device's pairwise ratchet — both sides load chain index *N* and both store *N+1*, silently dropping one advance. When the lost advance carried the SKDM, that member never received the sender key and every subsequent `skmsg` stayed undecryptable for it until a retry re-distributed. - -`prepare_group_stanza` now acquires the SKDM targets' per-device session locks through `SendContextResolver::lock_device_sessions()` before taking the chain lock, and releases them right after the fan-out — the `skmsg` chain encrypt that follows only touches the sender-key chain, never a pairwise session. The `Client` implementation of this hook reuses `build_session_lock_keys()` + `session_guards_for()`, so the group and DM paths serialize on the exact same mutexes, in the same sorted order, and always acquire session locks before the chain lock — no path takes the reverse order, so this cannot deadlock. The hook defaults to a no-op, so a custom `SendContextResolver` (as used in tests and benches) is unaffected unless it opts in. - - - -**Session-setup failures are isolated per device (v0.6).** `ensure_sessions_for_devices` used to abort with `Err` the moment `process_prekey_bundle` failed for *any* one target device. Since `prepare_group_stanza` gates the entire SKDM fan-out on `session_plan.is_some()`, one device's X3DH failure nulled the plan and **every** device in the cohort — not just the failing one — got no SKDM, even though the `skmsg` still shipped and the phash covered the full set. An external member recovers via a retry receipt, but an own companion's retry hits `mark_forget_sender_key` with `exclude_own_devices=true`, which filters own-user JIDs and returns early — so that companion stayed `has_key=true` forever and couldn't decrypt the group from that device until an unrelated full rotation (participant removal or PN↔LID migration). - -As of [#996](https://github.com/oxidezap/whatsapp-rust/pull/996), a device whose session setup fails is logged and skipped rather than aborting the plan — matching WhatsApp Web's `GroupKeyDistributionMsg`, which wraps each device's `ensureE2ESessions` in its own try/catch and drops only the failing one. The sessionless device is then naturally excluded by the encrypt fan-out (which already skips devices without a session), so every other device still receives its pairwise SKDM. - -[#996](https://github.com/oxidezap/whatsapp-rust/pull/996) closed the primary harm — an *unrelated* device's setup failure no longer suppresses the whole cohort's SKDM. A narrower window remained: the **warm mark** (`update_sender_key_devices`, called after the server ACK) recorded the *full* distribution target as `has_key=true`, including our own companion devices, regardless of whether each one's pairwise SKDM encryption actually succeeded. Since the forget path (`mark_forget_sender_key`) excludes own devices for the reason above, an own companion whose one SKDM encryption failed — or that was warm-marked without ever receiving a node — was marked warm and could **never** be un-marked: a permanent orphan until an unrelated full rotation. External devices didn't have this problem; they recover through the retry-receipt forget path. - -[#999](https://github.com/oxidezap/whatsapp-rust/pull/999) closes this residual by excluding own devices from the warm mark too (`exclude_own_devices=true`), mirroring WhatsApp Web's `ParticipantStore` helper, which guards *both* `markHasSenderKey` and `markForgetSenderKey` with the same `!isMeDevice` check. Own companions are therefore never memoized as `has_key=true` — `filter_skdm_targets` (["Per-device sender key tracking → Incremental targeting"](#per-device-sender-key-tracking) above) always re-includes them, so they get a fresh SKDM on every group send. This is a deliberate trade-off (a few extra pairwise SKDM nodes per send when the account has companions) in exchange for making the orphan impossible. External devices are unaffected: a successful distribution still marks them warm, and the retry-receipt path still repairs any that go stale. - - - -**The group distribution lane now guards the full audit-reset-redistribute sequence, not just the SKDM fan-out ([#1043](https://github.com/oxidezap/whatsapp-rust/pull/1043)).** Previously `Client::group_distribution_lock()` (see ["Parallelized group encrypt fan-out"](#parallelized-group-encrypt-fan-out) above) was taken only around the cold SKDM send itself. Sender-key deletion (participant-removal rotation, forced own-key rotation), per-device tracker resets, and the status-broadcast distribution path could run concurrently with that lock held elsewhere, letting an encrypt racing a rotation restore a retired key after deletion, or a tracker reset race stale delivery marks back onto a new chain. - -`rotate_sender_key_on_participant_remove`, `force_rotate_own_sender_key` (now taking `&Jid` instead of a pre-stringified group ID), warm group sends, status sends, phash-mismatch recovery, and periodic sender-key rotation all now hold the same per-group lane across their own-key delete/reset and the following redistribution. A rotation that arrives while a send is mid-fan-out waits for the lane instead of deleting the chain state out from under it; a send that arrives mid-rotation waits for the rotation to finish before re-auditing device state. Held lanes are never capacity-evicted, so a live rotation or fan-out cannot be silently dropped from the map mid-operation (see `group_distribution_locks_capacity` in the [Cache Configuration reference](/api/bot#cache-configuration-reference)). - - - -**Sender-key tracker resets are DB-first ([#1043](https://github.com/oxidezap/whatsapp-rust/pull/1043)).** `reset_sender_key_device_tracking` replaces the old direct `clear_sender_key_devices` + cache-invalidate call at every rotation and redistribution site. It clears the per-device tracking row-by-row in the database first, and only invalidates the in-memory `SenderKeyDeviceCache` after that durable clear succeeds. If the DB clear fails, every existing tracked row is instead marked cold (`has_key=false`) as a fallback so the next send still re-distributes; if that fallback write also fails, the operation returns an error and the send stays fail-closed rather than risking a stale `has_key=true` row surviving onto a freshly rotated chain. - -The unknown-participant rotation in [retry receipt handling](/advanced/retry-admission) is a special case: `handle_retry_receipt` deletes the own sender key and resets tracking for a `` from an unrecognized group participant, then must still fall through to the per-chat resend rate limiter and other throttles further down the same function. The signal cache is now explicitly flushed right after the rotation — before any later throttle can return early — so a rotation is never left un-persisted by an unrelated early exit later in the same call. - - - -**Observability: distribution-lane pressure is exposed on `memory_report()` ([#1043](https://github.com/oxidezap/whatsapp-rust/pull/1043)).** `Client::memory_report()` now reports `group_distribution_locks` (live lane count), `group_distribution_lock_evictions` (cumulative cold evictions), and `group_distribution_lock_eviction_blocks` (cumulative evictions skipped because the lane was live) — see [`memory_report()`](/api/client#memory_report). These update only under capacity pressure and add no allocation or per-message cost below the soft cap. - - -### In-memory sender key device cache - -The `SenderKeyDeviceCache` provides an in-memory caching layer over the per-device sender key tracking data stored in the database. Without this cache, every group send would require a database round-trip to load the sender key device map — the cache eliminates that overhead after the first load for each group. - -```rust -pub(crate) struct SenderKeyDeviceCache { - inner: Cache>, -} -``` - -**Key design decisions:** - -- **Time-to-idle eviction:** The cache uses TTI semantics (default: 1 hour, 500 entries), so entries for inactive groups are automatically evicted while frequently-used groups stay cached -- **Pre-parsed, pre-indexed maps:** Database rows are parsed into a `SenderKeyDeviceMap` struct that provides O(1) lookups by user and device ID, avoiding per-query string parsing -- **Single-flight initialization:** The `get_or_init` method uses `PortableCache`'s single-flight `get_with` — if multiple concurrent group sends for the same group trigger a cache miss simultaneously, only one database read executes and all callers share the result -- **Explicit invalidation:** The cache is invalidated when sender key state changes (rotation, error recovery, retry failures) so stale data is never served - -```rust -// Atomic get-or-init: concurrent callers for the same group -// share the single database read result -let cached_map = self - .sender_key_device_cache - .get_or_init(group_jid, async { - let db_rows = pm.get_sender_key_devices(group_jid).await.unwrap_or_default(); - Arc::new(SenderKeyDeviceMap::from_db_rows(&db_rows)) - }) - .await; -``` - -**`SenderKeyDeviceMap` structure:** - -The `SenderKeyDeviceMap` pre-parses JID strings from the database into a user-to-devices HashMap for efficient lookup: - -```rust -pub(crate) struct SenderKeyDeviceMap { - /// user → (device_id → has_key) - devices: HashMap, HashMap>, - /// Users with at least one has_key=false device - forgotten_users: HashSet>, -} -``` - -**Cache invalidation points:** - -| Event | Action | -|-------|--------| -| Sender key rotation (`rotateKey`) | Invalidate group entry | -| `NoSenderKeyState` error during send | Invalidate group entry | -| Retry failure for a group message | Invalidate group entry | -| Server rejects group stanza | Invalidate group entry | -| New device added (`patch_device_add`) | Invalidate all entries | -| Device removed (`patch_device_remove`) | Invalidate all entries | -| Identity change (`clear_device_record`) | No global tracker wipe — per-device SKDM redistribution is driven by retry receipts (`markForgetSenderKey`), matching WhatsApp Web's `WAWebUpdateLocalSignalSession`. The `status@broadcast` sender key is still deleted for forward secrecy. | - -You can tune the cache capacity and TTI via the `sender_key_devices_cache` field in [`CacheConfig`](/api/bot#cache-configuration-reference). - -Location: `src/sender_key_device_cache.rs`, `src/send.rs` - -### Phash validation for stale device list detection - -When sending group, status, or DM messages, the library validates the participant hash (`phash`) returned in the server's acknowledgment against the locally computed `phash`. A mismatch indicates that the server's view of participant devices differs from the client's — meaning the local device list is stale. - -**How it works:** - -1. Before sending, the client obtains the locally computed `phash` — from the stanza `phash` attribute for group/status messages, or from `PreparedDmStanza.phash` for DMs -2. A `PhashWaiter` (expected hash, target JID, whether to also invalidate the group cache) is registered for the message ID via `register_phash_waiter` — a map entry, not a channel or a task -3. The message stanza is sent to the server -4. When the server's ack for that message ID arrives, the read loop compares its `phash` attribute against the expected value inline, with no task involved -5. On a match the entry is just dropped; on a mismatch the client spawns a task to invalidate caches — so a send only pays for a task in the uncommon case, not on every send ([#1116](https://github.com/oxidezap/whatsapp-rust/pull/1116)) - -**On mismatch, the following caches are invalidated:** - -| Send path | Sender key device cache | Group info cache | Device registry | -|-----------|------------------------|-----------------|----------------| -| Group messages | Invalidated | Invalidated | — | -| Status messages | Invalidated | Not invalidated | — | -| DM messages | — | — | Recipient + own PN devices invalidated | - -For DM messages, the phash covers both recipient and own devices (matching WA Web's `syncDeviceListJob([recipient, me])`). On mismatch, the client invalidates the device registry cache for both the recipient's user JID and your own phone number (PN) JID, ensuring the next send re-fetches the current device list for both parties. - -```rust -// src/send.rs — simplified phash validation flow - -// Group/status path: phash from stanza attribute -let our_phash = stanza.attrs().optional_string("phash").map(|s| s.into_owned()); - -// DM path: phash from PreparedDmStanza (not on the wire) -let dm_phash = prepared.phash; - -// On DM phash mismatch: -if !jid.is_group() && !jid.is_status_broadcast() { - client.invalidate_device_cache(&jid.user).await; - if let Some(own_pn) = &client.persistence_manager.get_device_snapshot().pn { - client.invalidate_device_cache(&own_pn.user).await; - } -} -``` - - -The phash check never blocks the send path. If the server's ack never arrives, nothing polls the waiter directly — it is swept out on the keepalive tick, and is guaranteed to survive the sweep immediately following its registration (so a waiter is never dropped mid-flight) but is removed on the sweep after that. Since each keepalive tick lands 15–30 seconds after the last, the actual time-to-live is roughly one to two tick intervals — about 15 to 60 seconds, depending on where registration falls relative to the sweep cycle — rather than the old fixed 10-second timeout. The sweep runs before keepalive's own idle early-return, so a connection with steady inbound traffic — which skips sending pings — still gets its stale waiters cleared; a stranded waiter would otherwise read as an outstanding IQ and suppress pings for the life of the connection. This matches WhatsApp Web's approach of using phash as a best-effort staleness detector rather than a hard requirement. - - -#### WA Web phash parity (v0.6) - -Two corrections aligned the group phash with WA Web's `phashV2`: - -- **Full device set, every send.** The group phash is now computed over the *complete* resolved participant device set plus the sending device on every send — not just the devices that received an SKDM in that stanza. Warm sends (which distribute no new SKDM) now pass the full resolved set via the `all_devices_for_phash` parameter to `prepare_group_stanza`, so the phash matches the server's view even when the SKDM target set is empty. Status broadcasts keep their prior phash behavior. -- **Standard base64 alphabet.** The phash now encodes with the standard base64 alphabet (`+` / `/`) instead of URL-safe (`-` / `_`), matching WA Web and whatsmeow. - -The client also now **persists group metadata** locally after a query and sends the stored participant phash on the next group query, letting the server answer `not-modified` (304) when membership is unchanged — saving a full metadata round-trip. - -Location: `src/send.rs`, `src/client.rs` - -## Cryptographic Primitives - -### AES-256-CBC (message content) - -Used for encrypting message bodies in both 1:1 and group messages: - -```rust -pub fn aes_256_cbc_encrypt_into( - plaintext: &[u8], - key: &[u8], // 32 bytes - iv: &[u8], // 16 bytes - output: &mut Vec, -) -> Result<()> -``` - -Location: `wacore/libsignal/src/crypto/aes_cbc.rs` - -### Thread-Local Buffers - -The implementation uses thread-local buffers to reduce allocations: - -```rust -thread_local! { - static ENCRYPTION_BUFFER: RefCell = ...; - static DECRYPTION_BUFFER: RefCell = ...; -} - -// Usage in session_cipher.rs:99-111 -let ctext = ENCRYPTION_BUFFER.with(|buffer| { - let mut buf_wrapper = buffer.borrow_mut(); - let buf = buf_wrapper.get_buffer(); - aes_256_cbc_encrypt_into(ptext, message_keys.cipher_key(), - message_keys.iv(), buf)?; - let result = std::mem::take(buf); - buf.reserve(EncryptionBuffer::INITIAL_CAPACITY); - Ok::, SignalProtocolError>(result) -})?; -``` - -Location: `wacore/libsignal/src/protocol/session_cipher.rs:14-54` - -### HKDF-SHA256 - -Used for key derivation in session initialization: - -```rust -pub fn derive_keys(secret_input: &[u8]) -> (RootKey, ChainKey, InitialPQRKey) { - let mut secrets = [0; 96]; - hkdf::Hkdf::::new(None, secret_input) - .expand(b"WhisperText", &mut secrets) - .expect("valid length"); - // Split into RootKey[32], ChainKey[32], PQRKey[32] -} -``` - -Location: `wacore/libsignal/src/protocol/ratchet.rs:18-39` - -## PreKey Management - -Pre-keys enable asynchronous session establishment in the Signal Protocol. whatsapp-rust manages pre-key generation and upload to match WhatsApp Web's behavior. - -### Configuration - -The per-batch upload count is configurable through the builder/factory API (default `812`, matching WhatsApp Web's `UPLOAD_KEYS_COUNT`). The upload-trigger threshold is a private constant. - -| Setting | Default | Description | -|---------|---------|-------------| -| [`BotBuilder::with_wanted_pre_key_count`](/api/bot#with-wanted-pre-key-count) / [`Client::set_wanted_pre_key_count`](/api/client#set-wanted-pre-key-count) | 812 | Number of pre-keys generated and uploaded per batch. Clamped to `5..=65_535` at upload time. | -| `MIN_PRE_KEY_COUNT` (private const) | 5 | Minimum server-side pre-key count before triggering an upload. | - -```rust -// Via the Bot builder -Bot::builder() - .with_wanted_pre_key_count(256) // smaller batches for embedded consumers - // ... - -// Or directly on a Client constructed by hand (before connect) -client.set_wanted_pre_key_count(256); -``` - -Values outside `[5, 65_535]` are clamped at upload time (an out-of-range value logs a `warn!`). The floor avoids an empty-but-flagged pool or a re-upload loop. The ceiling is the wire-format limit: the upload IQ encodes the pre-key list length as a `u16`, so a larger batch would generate keys locally and then fail to encode. - -Per-key X25519 generation and protobuf encoding for the batch are offloaded via `wacore::runtime::blocking` (runtime-agnostic; runs inline on wasm) since the caller-controlled batch size can be large. - -### Pre-key ID counter and wrap-around - -Pre-key IDs use a persistent monotonic counter (`Device::next_pre_key_id`) that only increases, matching WhatsApp Web's `NEXT_PK_ID` pattern: - -```rust -// Determine starting ID using both the persistent counter AND the store max -let max_id = backend.get_max_prekey_id().await?; -let start_id = if device_snapshot.next_pre_key_id > 0 { - std::cmp::max(device_snapshot.next_pre_key_id, max_id + 1) -} else { - // Migration: start from MAX(key_id) + 1 - max_id + 1 -}; -``` - -This approach prevents ID collisions when pre-keys are consumed non-sequentially from the store. - -**24-bit wrap-around:** - -WhatsApp Web uses 24-bit pre-key IDs on the wire (3-byte big-endian), so valid IDs range from 1 to 16,777,215 (2^24 − 1). When the persistent counter grows past this boundary, modular arithmetic wraps IDs back into the valid range: - -```rust -const MAX_PREKEY_ID: u32 = 16777215; // 2^24 - 1 - -// Wrap start ID into valid [1, MAX_PREKEY_ID] range -let start_id = ((raw_start as u64 - 1) % MAX_PREKEY_ID as u64) as u32 + 1; - -// Each key ID in the batch is also wrapped -let pre_key_id = (((start_id as u64 - 1) + i as u64) % (MAX_PREKEY_ID as u64)) as u32 + 1; - -// After upload, the persisted next_pre_key_id wraps too -let next_id = (((start_id as u64 - 1) + key_pairs_to_upload.len() as u64) - % (MAX_PREKEY_ID as u64)) as u32 + 1; -``` - - - If the counter wraps while unconsumed high-ID pre-keys still exist in the store, the database upsert (`ON CONFLICT DO UPDATE`) silently overwrites them. This is an accepted trade-off because the server consumes keys well before a full 16M cycle completes. - - -Location: `src/prekeys.rs` - -### Retry-receipt prekey marking - -When building a retry receipt that includes keys (`should_include_keys`), the one-time prekey handed directly to the peer is now also marked uploaded via `mark_single_prekey_uploaded`, matching WhatsApp Web's `markKeyAsUploaded`. Without this, the same prekey ID could be re-offered to the server pool in the next batch upload — a third party fetching the bundle could then consume the identical one-time ID and fail to decrypt. - -`mark_single_prekey_uploaded` requires a held `prekey_upload_lock` guard (a compile-time proof, not just a runtime convention) so the get-or-gen and the watermark write are atomic against the batch upload path. It only advances `first_unupload_pre_key_id` when the id being marked is still the current window head (idempotent no-op otherwise), and collapses `next_pre_key_id` onto the wrapped low watermark only when marking the terminal id at the 24-bit edge — a non-terminal high-end head keeps its surviving window key. - -The device account is validated **before** the prekey is reserved/marked, so a missing account fails the retry-receipt build without silently abandoning a one-time prekey from the upload window. - -Location: `src/prekeys.rs`, `src/retry.rs` - -### Force-refreshing pre-keys for device migration - -When migrating a device from an external source (e.g., a Baileys session into an `InMemoryBackend`), the server may still hold pre-key IDs whose private key material you cannot reconstruct. Any `pkmsg` referencing those IDs will fail permanently with `InvalidPreKeyId`. - -The public `refresh_pre_keys()` method force-uploads a fresh batch of `Client::wanted_pre_key_count()` pre-keys (default 812; tunable via [`with_wanted_pre_key_count`](/api/bot#with-wanted-pre-key-count) / [`set_wanted_pre_key_count`](/api/client#set-wanted-pre-key-count)), giving the server new IDs the caller has locally. Old unmatched IDs drain naturally as peers consume them. - -```rust -// After restoring a session from another library -client.refresh_pre_keys().await?; -``` - -Internally, this acquires `prekey_upload_lock` to prevent races with the count-based and digest-repair upload paths, then calls `upload_pre_keys_with_retry(force: true)` which uses Fibonacci backoff (1s, 2s, 3s, 5s, 8s, ... capped at 610s). - -Two related public methods build on the same `prekey_upload_lock`-guarded path: - -- `Client::refresh_pre_keys_with_count(count)` — same force-upload as `refresh_pre_keys()`, but with a caller-chosen batch size instead of the configured [`wanted_pre_key_count`](#configuration). -- `Client::ensure_pre_keys()` — a non-forced check-and-top-up: uploads only if the server-side pool is below the low-water mark, rather than unconditionally replacing it. - -Location: `src/prekeys.rs:263-266` - -### Digest key validation - -After connection, the client validates that the server's copy of the key bundle matches local keys. This matches WhatsApp Web's `WAWebDigestKeyJob.digestKey()` flow. - -**Wire format:** - -```xml - - - - - - - - - [4-byte BE registration ID] - [1-byte: 5] - [32-byte identity public key] - - [3-byte BE signed pre-key ID] - [32-byte signed pre-key public] - [64-byte signature] - - - [3-byte BE prekey ID] - ... - - [20-byte SHA-1 hash] - - -``` - -**Validation process:** - -1. Query the server for the key bundle digest via `DigestKeyBundleSpec` -2. If the server returns **404** (no record), trigger a full pre-key re-upload -3. If the server returns **406/503** or other errors, log and skip -4. On success, compare registration IDs -5. Load each pre-key referenced by the server and extract its public key -6. Compute a local SHA-1 digest over: identity public key + signed pre-key public + signed pre-key signature + all pre-key public keys -7. Compare the local hash against the server-provided hash - - - The `` node contains `` children (not `` children). The parser iterates all children of `` without tag filtering, matching WhatsApp Web's `mapChildren` behavior which does not filter by tag name. - - - - Hash mismatches or missing local pre-keys are logged but do **not** trigger a re-upload. Only a 404 response (server has no record) triggers re-upload. This matches WhatsApp Web's behavior where `validateLocalKeyBundle` exceptions are caught without re-uploading — the normal [`RotateKeyJob`](#signed-pre-key-rotation-rotatekeyjob) eventually refreshes the signed pre-key. - - - - `Client::validate_digest_key()` is a public method — callers can trigger this validation pass on demand instead of only relying on the automatic post-connection check. - - -Location: `src/prekeys.rs:218-344`, `wacore/src/iq/prekeys.rs:170-302` - -### Signed pre-key rotation (RotateKeyJob) - -The signed pre-key minted at pairing was otherwise **permanent** — a forward-secrecy gap. whatsapp-rust mirrors WhatsApp Web's `RotateKeyJob`: on a cadence, generate a fresh signed pre-key, upload it, and retain the previous keys so prekey messages already in flight against a rotated-out key still decrypt. - -**Cadence:** - -- Checked once per connection. Spawned right after the startup pre-key upload during post-login init, so a slow or failing rotation IQ never delays the rest of login. -- Rotates once `now - last_signed_pre_key_rotation_ms >= SIGNED_PRE_KEY_ROTATION_INTERVAL_MS` (7 days / weekly). This interval is the one value **not** grounded in the captured WA Web bundle — there `RotateKeyJob` is a persisted, server-tuned background job — so it's a tunable policy default. -- A device upgraded in with the field at `0` gets a one-time baseline stamp (`DeviceCommand::SetSignedPreKeyRotationBaseline`) instead of rotating immediately, so its first rotation lands a full interval out. -- Single-flighted via `Client::signed_pre_key_rotation_lock` so overlapping post-login tasks (from reconnect churn) can't run the rotate/upload/prune sequence concurrently; a losing task just no-ops for that check. - -**Wire format** (`RotateSignedPreKeySpec`, reuses the upload path's `` encoder so the two can never drift): - -```xml - - - - [3-byte BE signed pre-key ID] - [32-byte signed pre-key public] - [64-byte signature] - - - -``` - -**Rotation sequence** (`Client::rotate_signed_pre_key`): - -1. Compute `new_id = current_id + 1`, wrapping to `1` at the 24-bit border (same scheme as one-time pre-key IDs). -2. Stage the new key pair in the `signed_prekeys` backend table **before** upload — an already-staged candidate for that id is reused verbatim, so a retry after an ambiguous failure re-uploads the exact key the server may have already accepted instead of minting a different one under the same id. -3. Retain the outgoing (current) key in the backend table **before** upload, so once the server accepts the new key the old id's decrypt window is already durable — no post-acceptance write can strand it. -4. Upload via the `` IQ above. -5. On success, `DeviceCommand::SetSignedPreKey` atomically installs the new key pair, id, signature, and rotation timestamp. The now-redundant staged copy is dropped, and retained signed pre-keys are pruned (newest-id-first) to `SIGNED_PRE_KEY_RETENTION` (3 total: the current key + the 2 most recent rotated-out keys). - -**Error handling** (mirrors WA Web's `RotateKeyJob` ladder; a rotation failure never fails login): - -| Response | Action | -|----------|--------| -| `406` / `409` (deterministic rejection of this key) | Drop the staged candidate and remint a fresh one on a later connect — reusing it would wedge rotation forever. | -| Other server error (rate limits, `>=500`, ...) | Keep the staged candidate and retry it as-is on a later connect. | -| Transport failure (ambiguous — the server may have accepted it) | Keep the staged candidate and retry it as-is on a later connect. | - -**Backend fallback for rotated-out ids:** - -Before this feature, `Device::load_signed_prekey` (`src/store/signal.rs`) returned a record only when the requested id matched the *current* `signed_pre_key_id` field — the `signed_prekeys` backend table (which already existed, with full CRUD) was never consulted for other ids. Rotating the key in place would therefore make any in-flight prekey message naming the old id fail with `InvalidSignedPreKeyId`. `load_signed_prekey` and `contains_signed_prekey` now fall back to the backend table for non-current ids, which is what makes rotation safe to ship. - -**Retry, not NACK, once the id ages past retention:** A sender's `PreKeySignalMessage` can still name a signed pre-key id that has since aged past `SIGNED_PRE_KEY_RETENTION` (3 total: current + 2 rotated-out) — the backend fallback above has nothing left to return, and `InvalidSignedPreKeyId` is the correct, permanent answer. On the 1:1 decrypt path (`src/message/receive.rs`), this now routes to a retry receipt (`RetryReason::InvalidKeyId`) carrying the current bundle, mirroring the sibling `InvalidPreKeyId` arm — instead of falling through to the catch-all `UnhandledError` nack, which would drop the stanza from the offline queue and lose the 1:1 message permanently and silently. - - - `Client::rotate_signed_pre_key()` is a public method — callers can force an out-of-cadence rotation directly instead of waiting for the weekly check. It shares `signed_pre_key_rotation_lock` with the automatic path (so a manual call can't race a background rotation) and propagates upload failures to the caller rather than swallowing them. - - -Location: `src/features/rotate_key.rs`, `src/store/signal.rs`, `src/message/receive.rs`, `wacore/src/iq/prekeys.rs`, `wacore/src/store/commands.rs` - -### Re-pair pre-key healing (v0.6) - -If the user re-pairs the device (for example by re-scanning the QR code), the server discards its copy of our pre-key bundle even though `Device::server_has_prekeys` may still read `true` from the previous pairing. v0.6 resets `server_has_prekeys = false` immediately after a successful re-pair so the next connect uploads a fresh batch instead of trusting the stale flag. - -The lock-acquisition for the digest-key validator also moved into `validate_digest_key` itself. Previously the caller held `prekey_upload_lock` before calling the validator, which would deadlock when validation hit a 404 and tried to acquire the same lock to perform the re-upload. The lock now wraps only the re-upload path, so the 404→re-upload transition completes without contention. - -Location: `src/handlers/notification.rs`, `src/pair.rs`, `src/prekeys.rs` - -### ADV companion identity validation - -When fetching a pre-key bundle for a contact's companion device (WhatsApp Web / Desktop), the bundle's `` element is validated to confirm that the fetched identity key is cryptographically bound to the account. This guards against a relay substituting a forged identity key, matching WA Web's `SessionApi.createSignalSession`. - -**Account key resolution** mirrors WA Web's `validateADVwithIdentityKey` (`e.accountSignatureKey || t`): - -1. **In-blob key**: If `ADVSignedDeviceIdentity.account_signature_key` is present and non-empty, it is used directly. -2. **Stored identity fallback**: The server legitimately omits this field for a contact's companion because the client already holds the contact's primary (device 0) identity in the Signal identity store. When the field is absent, `Client::load_account_identity` loads it — reading through the `SignalStoreCache` so any unflushed mutations from the current session are visible. `PreKeyFetchSpec::with_account_identities` threads the pre-loaded map into `wacore`'s stateless prekey parser, keeping store access in the `whatsapp-rust` crate. - -**Validation results** (`wacore::adv::AdvValidation`): - -| Variant | Condition | Action | -|---------|-----------|--------| -| `Valid` | Both account and device signatures verified | Session is established normally | -| `Invalid` | Blob is malformed, or signatures fail against the available key | Bundle is rejected — a relay swapping in a forged identity lands here | -| `NoAccountKey` | Neither the blob nor the store has the key | Bundle is kept, ADV check skipped (logged as `warn!`) | - -`NoAccountKey` does not weaken security beyond the pre-existing "device-identity absent" path: a relay could already strip the entire `` element to bypass the check. It exists so brand-new contacts whose primary identity has never been seen are not silently dropped. - -The same three-state validation applies in the retry-receipt handler (`src/retry.rs`) when a companion device requests a re-send. - -Location: `wacore/src/adv.rs`, `wacore/src/iq/prekeys.rs`, `src/prekeys.rs` - -## Storage Integration - -whatsapp-rust integrates Signal Protocol storage through a layered architecture: - -``` -src/store/ -├── signal.rs # SignalStore trait impl for Device (identity, session, prekey, sender key) -├── signal_adapter.rs # SignalProtocolStoreAdapter — cache-backed adapter bridging wacore traits to libsignal traits -└── signal_cache.rs # Re-export of wacore::store::signal_cache::SignalStoreCache -``` - -The `Device` struct implements the libsignal `SessionStore`, `IdentityKeyStore`, and other traits. These are wrapped by `SignalProtocolStoreAdapter`, which adds the `SignalStoreCache` layer — sessions are cached as `SessionRecord` objects (not bytes), with serialization deferred to `flush()`. - -Each store (sessions, identities, sender keys) is flushed **independently** under its own lock. Only one store is locked during its I/O — the other two remain free for concurrent encrypt/decrypt operations. The lock is held from snapshot through write through clear, so mutations to the same store are blocked until flush completes, preventing dirty-set races: - -```rust -// SignalProtocolStoreAdapter reads/writes through the cache -#[async_trait] -impl SessionStore for SessionAdapter { - async fn load_session( - &self, - address: &ProtocolAddress, - ) -> Result, SignalProtocolError> { - // Returns cached SessionRecord object directly (no deserialization) - // Cold load deserializes from backend bytes once and caches the object - self.cache.get_session(&addr_str, &*device.backend).await - } - - async fn store_session( - &mut self, - address: &ProtocolAddress, - record: SessionRecord, // Takes ownership — zero-cost move - ) -> Result<(), SignalProtocolError> { - // Stores the SessionRecord object in cache, marks dirty - // Serialization happens only during flush() - self.cache.put_session(&addr_str, record).await; - Ok(()) - } -} -``` - -### Flush scheduling: send vs. receive - -*When* the dirty Signal cache reaches the backend differs by direction, because the two directions have different recovery properties: - -- **Send (DM/1:1 sessions)** persists through a batched **counter lease**. `SessionRecord` reserves its outbound sender-chain counter `SENDER_CHAIN_RESERVATION_BATCH` (64) values at a time, via `SessionRecord::reserve_sender_chain_counters`. A send covered by an unexhausted lease is already durable — it only schedules the same coalesced write-behind as the receive path below. The send that exhausts the lease, roughly 1 in 64, raises the ceiling and flushes **synchronously, before the stanza reaches the wire**. If that flush fails, the send aborts instead of transmitting an advance it couldn't save. Reusing an outbound counter reuses its message key and IV, so no counter can ever be used before its lease is durable. The lease field is local-only: it's field 100 in the encoded `SessionRecord`, outside the vendored `whatsapp.proto`. By default (`SessionRecord::deserialize`), every load fast-forwards the sender chain to the lease ceiling, so a crash mid-lease can never re-derive a possibly-spent counter — the store-backed load path relaxes this only for a *trusted* reload, via the incarnation marker described below. -- **Send (group and status sends)** follows the same lease pattern as DMs. `SenderKeyRecord` reserves its outbound chain iteration `SENDER_CHAIN_RESERVATION_BATCH` (64) at a time via `SenderKeyRecord::reserve_iterations`, using the same field-100 local-only encoding and the same fast-forward-on-load recovery as `SessionRecord`. A send within an unexhausted lease rides the coalesced write-behind; only the send that raises the ceiling, roughly 1 in 64, flushes synchronously before the stanza reaches the wire. Status posts go through the same group-encrypt path and inherit this behavior; status *reactions* are a DM-branch send and always followed the DM lease instead. Production encryption (`wacore::send::encrypt_group_message`) delegates to the same `group_encrypt` primitive these guarantees live in, so there is exactly one sender-key encrypt/advance/store implementation — earlier, a second unguarded copy on the production path meant most warm group sends skipped the pre-wire flush entirely. -- **Receive** (live traffic, outside the offline-drain batcher) routes through a single-flight coalescing scheduler (`src/signal_flush.rs`) instead of flushing per stanza: a burst of receives folds into one flush per ~25ms window, retried with exponential backoff (up to a 5s cap) on backend failure. This is safe because a lost receive-side advance simply re-derives forward on the next message (the receiving chain derives `CK_n → CK_n+1`), and a consumed one-time prekey stays buffered until its session is durable — a crash inside the window is recoverable. - -**Every direct Signal-mutating call site now shares the same gate ([#1048](https://github.com/oxidezap/whatsapp-rust/pull/1048)).** The public [`Signal::encrypt_group_message` and `Signal::create_participant_nodes`](/api/signal) accessors, plus VoIP's outbound call-key fanout (`place_call` in `src/voip/facade.rs`), used to call `Client::flush_signal_cache_batch_safe()` — an unconditional flush — after releasing their session or sender-key chain locks, regardless of whether the mutation actually raised a lease's durable ceiling. They now call the same `Client::persist_signal_state_pre_wire()` gate the primary `wacore::send` path uses: a warm call already covered by an existing lease rides the coalesced write-behind and returns immediately, while only the roughly-1-in-64 call that raises the ceiling still flushes synchronously before its ciphertext is used or sent. Measured against the same real-SQLite harness: warm group encryption dropped from 84.69 ms to 23.98 ms per 1,024-call sample (-71.7%), and warm 4-recipient participant fanout dropped from 49.16 ms to 8.69 ms per 256-call sample (-82.3%). Safety is unchanged — the per-entry durability check that guards what gets written is identical in both cases; the difference is that `flush_signal_cache_batch_safe()` always entered the flush path even on a warm lease, while `persist_signal_state_pre_wire()` skips it entirely once the raised lease is already durable. This removes redundant synchronous I/O, not any safety guarantee. - - -Downgrading to a version that predates the counter lease after running a leased version: the older version ignores the lease field(s) and could reuse counters/iterations that were only reserved (not yet actually sent) by the lease. This applies to `SenderKeyRecord` (group/status sends) as well as `SessionRecord` (DM sends). Avoid downgrading a device's local state across this boundary. - - - -The pre-wire gate is a point-in-time check, not a lock held across the flush, and this specifically affects **DM sessions**: `needs_pre_wire_flush()` inspects pending reservations once, and `flush()` skips any session entry that is currently checked out by a concurrent operation (`SessionEntry::CheckedOut`), still returning `Ok` for the entries it did persist. If another task checks out the same session between this send's lock release and its flush, that session's reservation can remain pending even though the flush "succeeded" — the caller proceeds to write its stanza regardless. This is a property of `SignalStoreCache::flush` itself, not specific to retries; it affects any pre-wire-gated DM send that races a concurrent, *still in-progress* operation on the same session. Sender-key entries have no analogous checked-out state — `get_sender_key` clones an `Arc` without removing the cached record, so every dirty sender-key entry is included in a flush's batch — so group/status sends are not exposed to this race. This is unrelated to what happens if that concurrent operation is then *cancelled*: see [Cancellation-safe session checkouts](#cancellation-safe-session-checkouts) — a dropped checkout restores its entry synchronously rather than leaving it stranded, but the deferred entry still isn't included in a flush that already ran before the restore. - - -Deleting a session or sender-key record — an identity change, a session reset, a rotated sender key — creates a tombstone rather than clearing the record's pending gate immediately. If a durability gate was open on the record at delete time, `SignalStoreCache` keeps it open until the backend's `delete_session`/`delete_sender_key` call actually succeeds; a failed delete leaves `needs_pre_wire_flush()` returning `true` and is retried on the next flush. Earlier, the gate was released as soon as the tombstone was applied to the in-memory cache, so a failed delete or a crash in that window could let ciphertext reach the wire while the pre-delete chain state was still loadable from the backend, re-deriving already-used key material on reload. A lossy `clear()` still drops a pending tombstone gate — the tombstone is discarded from the in-memory cache without issuing the backend delete, so the old chain state may remain in the backend. - -### Clean reload vs. crash recovery - -Fast-forwarding past a lease's reserved ceiling on every reload is the safe default, but it's also overly conservative for the common case: a clean reconnect or a same-process store re-creation never actually risked losing an in-flight send, yet unconditionally fast-forwarding still burned a full unused batch every time. 32 clean reconnects could push a sender-key chain 2,048 iterations ahead and get rejected once a peer who missed the intervening messages hit `MAX_FORWARD_JUMPS` (2,000). - -`SignalStoreCache`, and the direct, non-cached `Device` store, now tag a durably-reserved record with a random 128-bit **store incarnation**, carried in a second local-only field (101, alongside the lease's field 100) on both `SessionRecord` and `SenderKeyRecord`: - -- A live `SignalStoreCache` generates one incarnation marker when it's constructed. A reload that observes a matching marker proves the record came from *this same live cache* and was never lost to a crash, so it's exact: the chain resumes at the next iteration instead of fast-forwarding. -- A reload with no marker, a different marker, or a malformed/duplicated marker is untrusted and keeps the conservative fast-forward — this covers process restarts, a freshly constructed cache, and any genuinely lossy discard. -- [`SignalStoreCache::clear_after_flush()`](/concepts/architecture#disconnect-cleanup), used by connection teardown, only evicts a store once it's fully settled — no dirty, deleted, checked-out, or pending-wire-gate entries. Anything a concurrent write installs after the flush stays resident (and the incarnation stays put) until a later flush settles it; only an actual lossy discard rotates the marker. -- Direct `Device` stores hold one process-level incarnation in an `OnceLock` instead of a per-cache one. Since these stores synchronously await the backend write before returning ciphertext, a new `Device` wrapping the same backend in the same process is a clean, trusted reload; a process restart gets a fresh marker and stays conservative. - -This adds no field to the public `Device` struct and no synchronous I/O — a marker is generated only when a cache or process starts, or when a lossy boundary invalidates trust. - -The scheduler is generation-scoped (embeds the connection generation in its atomic state), so a reconnect during an in-flight flush needs no explicit reset: a stale worker from the previous connection cannot mutate the new generation's state, and stands down when it observes a foreign generation. - -The offline drain, identity-change recovery, and teardown all keep their own **synchronous** flushes — they gate acks, receipts, or follow-up reads on durability and are not routed through the receive coalescer. (Teardown's flush is itself conditional: `teardown_inbound_commits_bounded` only flushes on a durable drain with no outstanding batch entries; on a timeout or non-durable/pending entries, it clears the cache without flushing instead, relying on server redelivery rather than risking a persisted-but-incomplete advance.) See [Inbound Durability Hook](/advanced/inbound-durability) for the drain-batch commit ordering, which this coalescing does not change. - -Retry-receipt recovery (`handle_retry_receipt` resending to a DM or group requester, `src/retry.rs`) instead shares the DM/group send durability rule above through one helper, `send_retry_stanza`: the session lock is released first, then `persist_signal_state_pre_wire()` runs — flushing synchronously if the retry's Signal advance crossed an unpersisted lease boundary — before the stanza is written to the wire. An `Err` from that flush aborts the retry instead of transmitting an advance it couldn't save; for a DM retry specifically, see the point-in-time caveat above for the narrower case where the flush reports success without having actually persisted this retry's own checked-out session (group retries are not exposed to that race — see the same caveat). This replaced an earlier unconditional full flush that ran only *after* the retry stanza had already reached the wire — a crash or persistence failure in that window could reload the old chain state after the ciphertext was already sent. - -Call [`Client::flush_pending_signal_state()`](/api/client#flush_pending_signal_state) to force a deterministic settle — e.g. before reading persisted Signal state directly, or ahead of a non-graceful shutdown. Never call it from inside an `InboundDurabilityHook` or a synchronous, inline `EventHandler::handle_event` implementation, since settling re-enters the processing permit those run under and would deadlock during an offline-sync drain. Ordinary `Bot` closure handlers are unaffected — both default delivery modes run the callback in a detached task off the permit. - -### DH ratchet resets rebase the lease - -A DH ratchet doesn't extend the current sender chain — it replaces it in place. The ratchet installs fresh key material from a new random ephemeral at counter zero and drops the retired chain instead of archiving it. The counter lease described above is a **record-level** ceiling, but the chain it bounds is **per-ratchet-epoch**. Without a matching lease rebase, the ceiling keeps describing a chain that no longer exists. - -For ping-pong traffic, that gap is one batch and you'd never notice it. It's different for a peer you only ever monologue at — say, your own other device, which gets a copy of every message you send but rarely replies. The chain climbs past `MAX_RESERVATION_FAST_FORWARD` before one reply triggers the ratchet, and the ceiling ends up stranded thousands of counters above a chain that just restarted at zero. - -A live reload never surfaces this, since a trusted-incarnation reload (above) skips the fast-forward entirely. The gap only shows up on **recovery** — a restart, or any lossy cache reset. There, the reload has to fast-forward across a span no send ever created. It refuses past `MAX_RESERVATION_FAST_FORWARD` and fails the whole record load. From that point the address is stranded: every path that could repair the session — inbound decrypt, the group-send fan-out, the retry-receipt handler — has to load the unloadable record first. - -`SessionRecord::rebase_lease_after_sender_chain_reset()` closes this gap. As part of the same mutation that swaps in the fresh chain, it lowers the ceiling to at most one `SENDER_CHAIN_RESERVATION_BATCH`. It only ever lowers, never raises, so you can never publish a counter under a ceiling that isn't yet durable. The lowering happens atomically with the chain swap, so no snapshot can pair the retired chain with a ceiling rebased for it, or vice versa. Rebasing to one batch instead of zero keeps the fresh chain's first counters lease-covered, so steady-state ping-pong keeps its write-behind send path instead of paying a synchronous flush on the very next send. A chain that's *archived* rather than discarded keeps its claim on the lease instead: `promote_fresh_state` burns the outgoing state to the ceiling before resetting it. - -If a record already has a stranded ceiling — written by a build that predates this fix — it recovers on its own the next time you use that address; you don't need to delete the row by hand. See [undecodable session rows](/concepts/storage#signalstorecache). - -## Record components - -`wacore-libsignal` exposes owned, validated projections of `SessionRecord` and `SenderKeyRecord` called **components**. Use them when you need to interchange or inspect session and sender-key record state without depending on the generated protobuf schema directly — for example in custom store implementations, migration tooling, or offline debugging. This API is purely additive: the protobuf-backed `serialize()`/`deserialize()` path is unchanged. A record does not round-trip through `into_components()` → `from_components()` → `serialize()` byte-for-byte — the conversion applies the validated, normalized export rules described below (counter-lease advancement, stale-chain removal, and bounded truncation), so treat it as a safe normalized re-encoding rather than a lossless copy. - -```rust -// wacore/libsignal/src/protocol/record_components.rs, re-exported from -// wacore/libsignal/src/protocol/mod.rs -pub use record_components::{ - PendingKeyExchangeComponents, PendingPreKeyComponents, SenderChainKeyComponents, - SenderKeyRecordComponents, SenderKeyStateComponents, SenderMessageKeyComponents, - SenderSigningKeyComponents, SessionChainComponents, SessionChainKeyComponents, - SessionComponents, SessionMessageKeyComponents, SessionMessageKeyMaterial, - SessionRecordComponents, -}; -``` - -### Session and sender-key shapes - -`SessionRecordComponents` mirrors the `current_session` / archived `previous_sessions` split already described in [Arc previous sessions](#arc-previous-sessions); `SenderKeyRecordComponents` mirrors a `SenderKeyRecord`'s state list: - -```rust -pub struct SessionRecordComponents { - pub current_session: Option, - pub previous_sessions: Vec, -} - -pub struct SessionComponents { - pub session_version: Option, - pub local_identity_public: Option>, - pub remote_identity_public: Option>, - pub root_key: Option>, - pub previous_counter: Option, - pub sender_chain: Option, - pub receiver_chains: Vec, - pub pending_key_exchange: Option, - pub pending_pre_key: Option, - pub remote_registration_id: Option, - pub local_registration_id: Option, - pub needs_refresh: Option, - pub alice_base_key: Option>, -} - -pub struct SessionChainComponents { - pub sender_ratchet_key: Option>, - pub sender_ratchet_key_private: Option>, - pub chain_key: Option, - pub message_keys: Vec, -} - -pub struct SenderKeyRecordComponents { - pub states: Vec, -} - -pub struct SenderKeyStateComponents { - pub key_id: u32, - pub chain_key: SenderChainKeyComponents, // { iteration: u32, seed: Vec } - pub signing_key: SenderSigningKeyComponents, // { public: Vec, private: Option> } - pub message_keys: Vec, // { iteration: u32, seed: Vec } -} -``` - -`SessionMessageKeyComponents` and `SenderMessageKeyComponents` hold skipped out-of-order message keys, keyed by chain index/iteration. A session message key's secret material is `SessionMessageKeyMaterial`, either the compact `Seed(Vec)` import form (expanded through the same canonical derivation used elsewhere — see [`MessageKeyGenerator`](#chain-key-ratcheting)) or the `Derived { cipher_key, mac_key, iv }` form that `into_components()` always produces on export. - -Conversions: - -```rust -impl SessionRecord { - pub fn from_components(value: SessionRecordComponents) -> Result; - pub fn into_components(mut self) -> Result; -} -impl SenderKeyRecord { - pub fn from_components(value: SenderKeyRecordComponents) -> Result; - pub fn into_components(mut self) -> Result; -} -``` - -### Import validation - -`from_components` enforces the same structural invariants the canonical protobuf reader relies on elsewhere in this codebase, rather than accepting whatever shape the caller hands it: - -- A **sender chain** (the local sending ratchet chain within a pairwise session — not to be confused with a group sender-key chain) must be structurally complete: ratchet public key present, a 32-byte ratchet private key, and a chain key with both its index and 32-byte secret set. An incomplete sender chain fails with `SignalProtocolError::InvalidArgument`; a session with *no* sender chain at all (e.g. one just received and not yet replied to) is fine and imports as `sender_chain: None`. -- A **receiver chain** must never carry `sender_ratchet_key_private` — receiver chains never own the remote party's private key, so import fails if one is set. Symmetrically, projecting a persisted record into components always reports a receiver chain's `sender_ratchet_key_private` as `None`, silently dropping any non-canonical private material a legacy record might contain, matching how the canonical reader already treats that field. -- Raw 32-byte public keys and canonically-serialized public keys (type byte + 32 bytes) are both accepted on import; whichever form was imported, `into_components()` always exports canonical serialization. - -### Export normalization - -`into_components()` never hands back a session or sender-key chain whose counter could be replayed on re-import: - -- Any durably reserved sender-chain counter range — see the counter-lease mechanics in [Flush scheduling: send vs. receive](#flush-scheduling-send-vs-receive) — is advanced to its exclusive ceiling before export. Re-importing the exported components can't reuse a counter value that was only reserved, not yet actually sent. -- A sender chain too stale to fast-forward past its reservation is dropped from the exported chain rather than aborting the whole export — the rest of the record (receiver chains, other archived sessions) still exports normally. -- `SessionRecordComponents.previous_sessions` is truncated to `ARCHIVED_STATES_MAX_LENGTH` (40) and `SenderKeyRecordComponents.states` is truncated to `MAX_SENDER_KEY_STATES` (5) — the same bounds the records themselves already enforce. See [Protocol safety limits](#protocol-safety-limits). - -### `has_usable_sender_chain` - -```rust -impl SessionState { - pub fn has_usable_sender_chain(&self) -> Result; -} -impl SessionRecord { - pub fn has_usable_sender_chain(&self) -> Result; -} -``` - -Checks whether a session has a sender chain it could actually encrypt with, rather than assuming one exists. Previously this was effectively best-effort/always-true; the current implementation returns `Ok(false)` (not an error) when no sender chain is set at all, and otherwise structurally validates the ratchet public key, the ratchet private key, and the chain key are all present before returning `Ok(true)` — the same completeness check `from_components` applies on import. `SessionRecord::has_usable_sender_chain` delegates to the current session's check, returning `Ok(false)` when there is no current session. - -Location: `wacore/libsignal/src/protocol/state/session.rs` - -### Debug output redacts secrets - -`Debug` on every `*Components` type — `SessionComponents`, `SessionChainComponents`, `SessionChainKeyComponents`, `PendingKeyExchangeComponents`, `PendingPreKeyComponents`, `SenderChainKeyComponents`, `SenderSigningKeyComponents`, `SenderMessageKeyComponents`, and `SessionMessageKeyMaterial` — prints private keys, chain/root keys, seeds, and cipher/mac/IV material as ``, while structural fields (indices, counters, iteration numbers, key presence) print plainly: - -```rust -println!("{chain:?}"); -// SessionChainKeyComponents { index: Some(7), key: } -``` - -This makes it safe to log or assert against a `*Components` value in application code without writing a custom `Debug` impl to avoid leaking key material. - -**Example — inspecting whether a session can currently send, without touching protobuf types:** - -```rust -// `record: SessionRecord` loaded from your store -if record.has_usable_sender_chain()? { - let components = record.into_components()?; - println!("{:?}", components.current_session); // secrets redacted -} -``` - -Location: `wacore/libsignal/src/protocol/record_components.rs` - -### Legacy session v1 interop - -Behind the opt-in `legacy-session-interop` Cargo feature (default off, forwarded through `wacore` and the root `whatsapp-rust` crate), `wacore-libsignal` exposes a typed, transport-agnostic model of the **decoded** legacy libsignal `SessionRecord` v1 layout — the format this project's stores used before the canonical shapes above. It exists for migration tooling importing an externally produced v1 store into the canonical `SessionRecord`, or projecting a canonical record back into v1 terms; ordinary clients never enable it, so the model compiles out of native builds entirely. - -Container decoding — turning a legacy store's transport bytes into these typed fields — is explicitly out of scope; callers own that step and hand this module owned values (`Bytes`, integers, enums). The module owns everything downstream: chain-role selection, counter translation, lifecycle ordering, pruning, ratchet reconstruction, and skipped-key derivation. - -```rust -// wacore/libsignal/src/protocol/legacy_session.rs, re-exported from -// wacore/libsignal/src/protocol/mod.rs behind `legacy-session-interop` -pub use legacy_session::{ - LegacyIndexedSessionV1, LegacySessionBaseKeyRoleV1, LegacySessionChainCounterV1, - LegacySessionChainKeyV1, LegacySessionChainRoleV1, LegacySessionChainV1, - LegacySessionDispositionV1, LegacySessionFieldV1, LegacySessionIndexV1, - LegacySessionInteropError, LegacySessionKeyPairV1, LegacySessionLocalContext, - LegacySessionMessageKeyV1, LegacySessionPendingPreKeyV1, LegacySessionRatchetV1, - LegacySessionRecordV1, LegacySessionUnrepresentableFieldV1, LegacySessionV1, -}; -``` - -**Import — v1 into canonical:** - -```rust -impl LegacySessionRecordV1 { - pub fn from_indexed_sessions( - sessions: Vec, - ) -> Result; - - pub fn into_session_record( - self, - context: LegacySessionLocalContext, - ) -> Result; -} -``` - -`from_indexed_sessions` validates that each entry's outer map key matches its own session's base key and rejects duplicate base keys or more than one `Current` session. `into_session_record` then validates every session — chain roles, key lengths, counters, skipped-key indexes, pending pre-keys, and the same canonical limits enforced in [Import validation](#import-validation) — retains archived sessions by close time, reorders them by last use to match the v1 decrypt search, truncates to `ARCHIVED_STATES_MAX_LENGTH`, and reuses `SessionRecord::from_components` to build the canonical record. `LegacySessionLocalContext` supplies the local identity key and registration ID, since v1 sessions never persisted them. - -A v1 sending chain seeds `previous_counter` at `-1` for a ratchet step over a chain that has never sent; `LegacySessionChainCounterV1` floors that at zero on import instead of treating it as an error. Every other out-of-range counter fails typed as `InvalidChainCounter`. - -**Export — canonical into v1 (operational, not byte-exact):** - -```rust -impl SessionRecord { - pub fn into_legacy_session_v1_operational( - self, - ) -> Result; -} -``` - -This is a deterministic operational projection, not a round trip: v1 lifecycle timestamps and base-key lookup roles are reconstructed from canonical search/eviction order rather than recovered verbatim, since the canonical record never persisted them in the first place. State the v1 format genuinely cannot represent is rejected with a typed error instead of being silently dropped or inferred — a session with no sender chain, a non-current `session_version`, a pending key exchange, or a `needs_refresh` flag all fail as `NotRepresentable`; a receiver chain holding a derived (non-seed) skipped-message key with no inverse to a v1 seed fails as `ChainNotRepresentable::DerivedMessageKey`; and a pending pre-key whose base key doesn't match the session's own base key fails as `PendingPreKeyBaseMismatch` rather than producing v1 state the importer would reject on the way back in. - -Every type in the module redacts key material from `Debug` — root keys, chain keys, ratchet key pairs, skipped-message seeds, and identity keys all print as ``; only structural fields (roles, counters, indexes, session/chain counts) print plainly, the same convention as the canonical [`*Components` types](#debug-output-redacts-secrets). - -Location: `wacore/libsignal/src/protocol/legacy_session.rs` - -## Security Considerations - -### Identity key trust - -The implementation verifies identity keys before encryption/decryption: - -```rust -if !crate::protocol::storage::is_trusted_identity( - identity_store, - remote_address, - &their_identity_key, - Direction::Sending, -) -.await? -{ - return Err(SignalProtocolError::UntrustedIdentity( - remote_address.clone(), - )); -} -``` - -As of [#1124](https://github.com/oxidezap/whatsapp-rust/pull/1124), this calls the free `is_trusted_identity` resolver (see [Unboxed identity and session hooks](#unboxed-identity-and-session-hooks)) rather than the trait method directly, so a store's `try_is_trusted_identity` hook gets a chance to answer first. - -Location: `wacore/libsignal/src/protocol/session_cipher.rs:309-325` - -### Self-only protocol message gating - -`app_state_sync_key_share`, `app_state_sync_key_request`, and `history_sync_notification` are protocol messages WhatsApp Web treats as "self-only": they only carry meaning when delivered from your own account to another of your linked devices. v0.6 hardens `handle_decrypted_plaintext` so that incoming copies of these messages are dropped unless `MessageInfo.source.is_from_me` is `true`, matching WA Web's `WAWebKeyManagementHandleKeyShareApi` and whatsmeow's gating. - -The consequences if the gate is missing: - -- A spoofed `app_state_sync_key_share` from a peer would let an attacker inject an app-state encryption key, leading to attacker-controlled mutations of your contacts, blocklist, archive state, etc. -- A spoofed `app_state_sync_key_request` from a peer would make the client share its app-state encryption keys with an attacker instead of only with the account's own companion devices. -- A spoofed `history_sync_notification` would point the client at attacker-supplied media for ingestion as your own history. - -If you implement a custom message dispatcher, replicate this `is_from_me` check before honoring any of these protocol messages. Other protocol-message types (`REVOKE`, `EPHEMERAL_SETTING`, `MESSAGE_EDIT`, …) keep their existing semantics. - -Location: `src/message.rs` (`handle_decrypted_plaintext`) - -### Duplicate message detection - -The protocol detects and rejects duplicate messages: - -```rust -if chain_index > counter { - return match state.get_message_keys(their_ephemeral, counter)? { - Some(keys) => Ok(keys), - None => Err(SignalProtocolError::DuplicateMessage(chain_index, counter)), - }; -} -``` - -Location: `wacore/libsignal/src/protocol/session_cipher.rs:822-827` - -As of [#1072](https://github.com/oxidezap/whatsapp-rust/pull/1072), a `DuplicatedMessage` from one candidate session is not terminal — the search keeps trying the remaining current and archived sessions, including a **closed** receiver chain (one with no chain-key seed left, only leftover skipped keys). This covers re-initiations, which reuse the peer's signed pre-key as a ratchet key: a delayed message whose skipped key survives only in an archived session still decrypts, and a closed chain still recognizes a replay of an already-consumed counter instead of falling through to a generic failure. - -Once every session has been tried, a recognized `DuplicatedMessage` outranks `BadMac` in the final classification: a sibling session that happens to share the same ratchet key derives different message keys for the same counter and fails its MAC as expected noise, but the decrypting client already knows this counter was consumed elsewhere. Classifying that as `BadMac` would trigger a retry receipt for a message the peer already delivered; the duplicate verdict wins instead, so the replay is acknowledged and silently dropped. - -### Log level discipline - -The protocol layer follows strict rules about what cryptographic material appears in logs and at which level: - -- **No private keys or secrets are ever logged** — `ChainKey`, `MessageKeys`, and `RootKey` types do not expose their key bytes through logging -- **Public keys appear only at `warn`/`error` levels** — and only when something has gone wrong (untrusted identity, MAC failure) -- **MAC key fingerprints are truncated** — only the first 4 bytes (8 hex chars) are logged during MAC verification failures, not the full key: - ```rust - let mac_key_fingerprint: String = hex::encode(mac_key_bytes).chars().take(8).collect(); - ``` -- **Ratchet keys in debug logs** — successful decryptions log the sender ratchet public key (never private) at `debug` level for diagnostics -- **Pre-key operations** use `debug` for routine operations and `warn`/`info` for exceptional conditions - - - The Signal protocol layer (`wacore/libsignal/src/protocol/`) uses no `trace!`-level logging. Sensitive operations stay at `debug` or above to avoid leaking material in verbose log configurations. - - -### Session state corruption - -Detailed logging helps diagnose crypto failures: - -```rust -fn create_decryption_failure_log( - remote_address: &ProtocolAddress, - errs: &[SignalProtocolError], - record: &SessionRecord, - ciphertext: &SignalMessage, -) -> Result -``` - -This generates comprehensive error logs showing: -- All attempted session states -- Receiver chain information -- Message metadata (sender ratchet key, counter) - -Location: `wacore/libsignal/src/protocol/session_cipher.rs:365-454` - -### Protocol safety limits - -The implementation enforces several hard limits to prevent resource exhaustion and cryptographic failures: - -| Constant | Value | Purpose | -|---|---|---| -| `MAX_PREKEY_ID` | 16,777,215 (2^24 − 1) | Maximum valid pre-key ID (24-bit wire format) | -| `MAX_FORWARD_JUMPS` | 2,000 | Maximum message skip in a ratchet chain — peer sessions and group sender keys | -| `MAX_FORWARD_JUMPS_SELF` | 25,000 | Maximum message skip for a session with your own other devices (wider, still bounded) | -| `MAX_MESSAGE_KEYS` | 2,000 | Maximum cached out-of-order message keys per chain | -| `MAX_RECEIVER_CHAINS` | 5 | Maximum receiver chains per session | -| `ARCHIVED_STATES_MAX_LENGTH` | 40 | Maximum archived session states | -| `MAX_SENDER_KEY_STATES` | 5 | Maximum sender key states per group | -| `MESSAGE_KEY_PRUNE_THRESHOLD` | 50 | Amortized eviction trigger for old message keys | -| Chain key index | u32::MAX | Overflow returns `InvalidState` error (not silent wrap) | - -Location: `wacore/libsignal/src/protocol/consts.rs` - -### Self-DM / sibling decryption recovery - -When a message from your own primary phone or another linked companion fails to decrypt, the v0.6 client distinguishes the failure mode and applies the matching recovery strategy: - -| Internal `RetryReason` | Trigger | Recovery | -|---|---|---| -| `NoSession` | `SessionNotFound` (no Signal session yet for the device) | Request a fresh prekey bundle via retry receipt; install the new session before retrying decryption. | -| `BadMac` | Ratchet desync (`InvalidMessage` / mac failure) on an existing session | Mark the session for re-creation, throttled per peer via the `session_recreate_history` cache so repeated BadMacs don't loop, and re-send via a peer-addressed `pkmsg` carrying our identity. | - -The throttle is a per-peer cooldown (1-hour TTL after the last recreate). In v0.6 the implementation moved from a `Mutex>` to a bounded TTL cache (`PortableCache`, ~256 entries): the per-peer check-and-stamp is now atomic (serialized by the existing per-peer session lock) and lock-free at the map level, so concurrent retry-receipt spawns from the same peer can't trigger duplicate recreates. The behavior is unchanged — if a peer is already in cooldown, the client skips re-creation and falls back to a normal retry receipt rather than thrashing the session. Under more than ~256 distinct peers retrying within the window, the cache may evict a recent entry, costing at most one extra recreate (bounded and self-healing). Peer-addressed `pkmsg` carries the protocol identity so the receiver can verify ownership before installing the new session, blocking spoofed sibling recoveries. - -This closed a deadlock where self-DM fan-out to a sibling device produced repeated BadMac decrypt failures: the recipient would request a retry, the sender would re-encrypt against the same broken session, and the cycle would continue until the user manually relogged. With the throttled re-creation plus identity-validated pkmsg, the second receipt installs a fresh session and decryption resumes. - -Self-DM fan-out also gained WA Web parity for the BadMac case: when our own primary phone reports BadMac, the client now treats it as a session-level recovery rather than dropping the message, matching `WAWebDecryptOrThrow`'s branch on session divergence. - -Location: `src/client.rs`, `src/retry.rs`, `wacore/libsignal/src/protocol/session_cipher.rs`, `wacore/src/send.rs` - -## Performance optimizations - -### Session object cache - -The `SignalStoreCache` stores sessions and sender keys as deserialized objects (`SessionRecord` and `SenderKeyRecord`) rather than serialized bytes, matching WhatsApp Web's architecture where the JS object IS the cache. Serialization only happens during `flush()` to the database — not on every `store_session` or `put_sender_key` call. - -```rust -// wacore/src/store/signal_cache.rs -enum SessionEntry { - /// Arc so peek_session (retry / LID-migration checks) bumps a refcount - /// instead of deep-cloning the record (KBs with archived states). - Present(Arc), - Absent, - /// Taken by a destructive-update load; has_session treats as present. - /// Carries enough identity to reject a stale owner — see - /// "Cancellation-safe session checkouts" below. - CheckedOut { - had_session: bool, - token: NonZeroU64, - }, -} - -struct SessionStoreState { - cache: HashMap, SessionEntry>, // Objects, not bytes - dirty: HashSet>, - deleted: HashSet>, - incarnation: StoreIncarnation, - checkout_generation: u64, - next_checkout_token: u64, -} - -// Sender keys use the same object-caching pattern -struct SenderKeyStoreState { - cache: HashMap, Option>, // Objects, not bytes - dirty: HashSet>, -} -``` - -This eliminates protobuf encode (on store) and decode (on load) from the per-message hot path for both 1:1 and group messages. The `store_session` method takes `SessionRecord` by value, enabling zero-cost moves from the protocol layer: - -```rust -// wacore/libsignal/src/protocol/storage/traits.rs -pub trait SessionStore { - async fn store_session( - &mut self, - address: &ProtocolAddress, - record: SessionRecord, // Owned — zero-cost move, no clone - ) -> Result<()>; -} -``` - -All four protocol-layer call sites (`message_encrypt`, `message_decrypt_signal`, `message_decrypt_prekey`, `process_prekey_bundle`) — plus PreKey setup and group preflight mutations — take ownership of the record and drop it immediately after storing. Taking ownership eliminates the `.clone()` in the adapter and the compiler enforces no use-after-store. As of [#1044](https://github.com/oxidezap/whatsapp-rust/pull/1044), these call sites obtain the record through the `SessionCheckout` guard (below) rather than a bare `load_session`/`store_session` pair, so the zero-cost move still applies but the checkout is now cancellation-safe. - -**Per-message hot path impact:** - -| Operation | Before | After | -|-----------|--------|-------| -| `store_session` | clone all fields + protobuf encode | move (zero-cost) | -| `load_session` | protobuf decode + construct | clone current session only (`previous_sessions` O(1) via Arc) | -| `peek_session` | deep-clone record (1–2 KB) | `Arc` refcount bump; returns `Option>` | -| `store_sender_key` | serialize to bytes + store bytes | store `SenderKeyRecord` object directly | -| `load_sender_key` (`&self`) | load bytes + deserialize | return cached `SenderKeyRecord` object (read lock only) | -| `flush` (batched) | write bytes to DB | serialize sessions + sender keys + write bytes to DB | - -### Cancellation-safe session checkouts - -As of [#1044](https://github.com/oxidezap/whatsapp-rust/pull/1044), the record backing a `CheckedOut` entry is owned by a `SessionCheckout<'a>` guard (`wacore/libsignal/src/protocol/storage/traits.rs`) instead of being a bare marker. This closes a gap where cancelling the future mid-mutation — a Tokio task abort, a `select!` losing a race, a timeout — could leave only the marker behind: the ratchet advance the future was computing was lost, and a decrypt's ratchet advance could persist without its matching identity write, or vice versa. - -`SessionCheckout` is obtained via `SessionCheckout::load` (existing session) or `load_or_create` (synthesizes `SessionRecord::new_fresh()` if absent, tracked via `had_session`), exposes `record()`/`record_mut()` for the protocol layer to mutate in place, and is consumed by either: -- `commit(self)` — stores the mutated record back through `try_store_session_from_checkout`, or -- `discard(self)` — releases a deliberately-rejected *fresh* (never-had-a-session) checkout without storing. - -If neither runs because the guard is dropped first (the cancellation case), `Drop` synchronously puts the record back itself — no caller code has to remember to handle cancellation. If the sessions mutex is uncontended, restoration is immediate; if a concurrent `flush()` holds the lock, the restore is queued as a `PendingSessionRestore` and replayed the next time any operation acquires the sessions lock, so an aborted task's state is never dropped even though it can't wait for the lock itself. The queue itself (`SyncMutex>`) is unbounded — every lock acquisition drains it first, so it only grows if cancellations under lock contention arrive faster than anything else touches the session store, which does not happen under normal load. - -Every checkout carries a `SessionCheckoutKey { generation, token }`: `checkout_generation` increments on every lossy `clear()`, and `token` is assigned monotonically per checkout of a given address. A restore is rejected (rather than silently applied) unless both match the cache's current state — this stops a checkout issued before a session reset from resurrecting stale state, and stops a stale owner from clobbering a newer checkout of the same address. - -`flush()` and cache eviction (`clear_after_flush`) now preserve live `CheckedOut` entries — and any PreKey deletion buffered against that address — instead of dropping them; the deferred prekey delete and the session's own persistence both wait for a later flush once the checkout completes, rather than being lost. - -`SessionStore` gained five `#[doc(hidden)]` methods with pass-through default bodies (`load_session_for_update`, `try_load_session_for_update`, `try_store_session_from_checkout`, `cancel_session_checkout`, `complete_session_checkout`), so existing custom `SessionStore` implementations keep compiling unchanged — only a backend that wants the destructive-update fast path needs to implement them. - -### Unboxed identity and session hooks - -As of [#1124](https://github.com/oxidezap/whatsapp-rust/pull/1124), the same "sync hook first, boxed async fallback second" pattern above extends to three more per-message checks: `IdentityKeyStore::try_is_trusted_identity`, `IdentityKeyStore::try_save_identity`, and `SessionStore::try_has_session`. `#[async_trait]` boxes every store method into a `Pin>`, regardless of whether the answer is already known. The bundled `IdentityAdapter`'s async `is_trusted_identity` body is an unconditional `Ok(true)` — WA Web `isTrustedIdentity` parity, since identity *changes* surface through `save_identity` rather than through trust checks. That box was therefore pure overhead, paid once per encrypt and once per decrypt. - -Each hook defaults to `None`, and `None` means "I cannot answer synchronously" — never "the answer is the default": -- `try_save_identity` declines when nothing is cached for the address, so the caller reads the backend. Answering from an empty cache would report every identity as new. -- `try_has_session` declines when the cache cannot answer, rather than reporting "no session" and forcing a needless session rebuild. -- `try_is_trusted_identity` can always answer **for the bundled adapter specifically**, because its async `is_trusted_identity` is already the unconditional `Ok(true)` above. This is not a general license to short-circuit trust checks: a custom `IdentityKeyStore` whose async `is_trusted_identity` enforces a real policy must not copy this hook verbatim — returning `Some(Ok(true))` unconditionally would let it bypass that policy. Such a store should return `None` unless it can decide synchronously without skipping any of its own trust logic. - -Three free functions in `wacore::libsignal::protocol` — `is_trusted_identity`, `save_identity`, `has_session` — express "ask the hook, then await the fallback" once instead of at each of the eight call sites that previously called the trait methods directly: - -```rust -// wacore/libsignal/src/protocol/storage/traits.rs -pub async fn is_trusted_identity( - store: &S, - address: &ProtocolAddress, - identity: &IdentityKey, - direction: Direction, -) -> Result { - match store.try_is_trusted_identity(address, identity, direction) { - Some(answer) => answer, - None => store.is_trusted_identity(address, identity, direction).await, - } -} -``` - -`message_encrypt`, `message_decrypt_signal`, and `ensure_sessions_for_devices` now call through these resolvers instead of the trait methods directly. The hooks are purely additive: a custom `IdentityKeyStore`/`SessionStore` that implements only the async methods keeps compiling and behaving unchanged, since declining both hooks falls through to exactly the async path it always ran. - -### Arc previous sessions - -`SessionRecord.previous_sessions` is wrapped in `Arc>`, making clone O(1) for the ~40 archived previous sessions that previously accounted for ~40% of the serialize cost: - -```rust -// wacore/libsignal/src/protocol/state/session.rs -pub struct SessionRecord { - current_session: Option, - /// Wrapped in Arc so cloning is O(1). Only mutated on rare paths via Arc::make_mut. - previous_sessions: Arc>, -} -``` - -Only rare operations (archive current session, promote previous session, take/restore during session setup) trigger `Arc::make_mut` and a deep copy. - -### Chain key buffer reuse - -As of [#1137](https://github.com/oxidezap/whatsapp-rust/pull/1137), advancing a chain key no longer allocates a fresh buffer for its persisted 32-byte key material on every step. `SessionState` stores each chain key's bytes as `Option` on the underlying protobuf `ChainKey` field. `Bytes` is immutable, so writing the ratcheted key used to be an unconditional `Bytes::copy_from_slice(..)` on every send or receive that advances a session's chain key. In the harness benchmark that motivated this change — a single-device 1:1 pingpong session — a full message round trip advances chain keys three times (twice sending, once receiving); a real send can touch more sessions than that, since [DM device fanout](#dm-device-fanout) encrypts separately for every resolved recipient and own-device session. - -`write_chain_key` (`wacore/libsignal/src/protocol/state/session.rs`) instead reuses the existing buffer in place when it safely can: - -```rust -fn write_chain_key(field: &mut Option, key: &[u8]) { - if let Some(existing) = field.take() - && existing.len() == key.len() - && let Ok(mut owned) = existing.try_into_mut() - { - owned.copy_from_slice(key); - *field = Some(owned.freeze()); - return; - } - *field = Some(bytes::Bytes::copy_from_slice(key)); -} -``` - -Reuse only happens when both guards pass: `try_into_mut()` succeeds solely when the `Bytes` is uniquely owned (no other clone observing the old key), and the length check keeps a differently-sized buffer (e.g. from a legacy record) from reaching `copy_from_slice` at all — like the slice method it resolves to via `DerefMut`, a length mismatch there panics rather than writing anything. Either guard failing falls back to the original allocating behavior. In steady state a checked-out session record is uniquely owned — the cache takes it out of its `Arc` via `try_unwrap` (see [Session object cache](#session-object-cache) above) — so the fallback is rare. - -### Redundant signal store write elimination - -The `SignalStoreCache` uses targeted deduplication strategies per store type. For identities (which rarely change), `put_dedup()` compares incoming bytes against the cached value and skips if identical: - -```rust -// wacore/src/store/signal_cache.rs — ByteStoreState -fn put_dedup(&mut self, address: &str, data: &[u8]) { - if let Some(Some(existing)) = self.cache.get(address) - && existing.as_ref() == data - { - return; // Skip — data unchanged, no dirty mark - } - self.put(address, data); -} -``` - -Sessions and sender keys use unconditional `put()` since they change with every message — dedup would always fail and waste CPU cycles. This split avoids unnecessary database writes during `flush()` while not adding overhead where it provides no benefit. - -### Key reuse in cache - -The `key_for()` method on `SessionStoreState`, `SenderKeyStoreState`, and `ByteStoreState` reuses existing `Arc` keys from the HashMap via `get_key_value()`, avoiding a heap allocation on every cache operation: - -```rust -fn key_for(&self, address: &str) -> Arc { - match self.cache.get_key_value(address) { - Some((existing, _)) => existing.clone(), // O(1) refcount bump - None => Arc::from(address), // Only on first insert - } -} -``` - -On the hot path (put/delete for addresses already in the cache), this is always a refcount bump instead of a heap allocation. - -### Single-allocation session lock keys - -Session lock keys use the full Signal protocol address string (e.g., `5511999887766@c.us.0`). The `JidExt` trait provides methods for generating these strings, defined in `wacore/src/types/jid.rs`: - -```rust -pub trait JidExt { - /// Construct a fresh ProtocolAddress for this JID. - fn to_protocol_address(&self) -> ProtocolAddress; - - /// Signal address string: `{user}[:device]@{server}` - /// Device part only included when device != 0. - fn to_signal_address_string(&self) -> String; - - /// Full protocol address string: `{signal_address_string}.0` - /// Equivalent to `to_protocol_address().to_string()` but avoids the - /// intermediate ProtocolAddress allocation — one String instead of two. - fn to_protocol_address_string(&self) -> String; - - /// Rewrite a reusable ProtocolAddress in place for this JID. - /// See [Reusable hot-loop address construction](#reusable-hot-loop-address-construction). - fn reset_protocol_address(&self, addr: &mut ProtocolAddress); -} -``` - -`to_protocol_address_string()` is used on hot paths (message encryption and decryption) as the key for `session_locks`. It pre-sizes the output buffer and builds the `String` in a single allocation. Constructing a `ProtocolAddress` itself no longer allocates for addresses that fit inline (see [Single-buffer ProtocolAddress](#single-buffer-protocoladdress) below), but `.to_string()` on top of it still does, so `to_protocol_address_string()` remains the cheaper path when only the string is needed. - -The `write_protocol_address_to()` free function provides the same formatting but writes into a caller-supplied `&mut String` buffer, enabling buffer reuse across multiple JIDs. - -**Format examples:** - -| JID | Signal address | Protocol address string | -|-----|---------------|------------------------| -| `5511999887766@s.whatsapp.net` | `5511999887766@c.us` | `5511999887766@c.us.0` | -| `5511999887766:33@s.whatsapp.net` | `5511999887766:33@c.us` | `5511999887766:33@c.us.0` | -| `123456789@lid` | `123456789@lid` | `123456789@lid.0` | -| `123456789:33@lid` | `123456789:33@lid` | `123456789:33@lid.0` | - - - The server `s.whatsapp.net` is mapped to `c.us` in address strings, matching WhatsApp Web's internal format. The trailing `.0` is the Signal device_id (always 0 in WhatsApp's usage). - - -**Usage in message processing:** - -```rust -// In message decryption (src/message.rs) — single lock per sender device -let signal_addr_str = sender_encryption_jid.to_protocol_address_string(); -let session_mutex = self.session_locks - .get_with(signal_addr_str.clone(), async { - Arc::new(async_lock::Mutex::new(())) - }).await; -let _session_guard = session_mutex.lock().await; - -// In peer message encryption (src/send.rs) — single lock -let signal_addr_str = encryption_jid.to_protocol_address_string(); - -// In DM message encryption (src/send.rs) — per-device locks for all devices -let lock_jids = self.build_session_lock_keys(&all_dm_devices).await; -let session_guards = self.session_guards_for(&lock_jids).await; -// Each lock taken as its mutex is resolved, in sorted order, to prevent deadlocks -``` - -**DM multi-device fanout:** - -The DM send path resolves all known recipient devices and own companion devices, encrypting per-device for each. This matches WA Web's `WAWebSendUserMsgJob` behavior where the local device table is read on the send path, and `WAWebDBDeviceListFanout` filters out hosted devices. The client checks the local device registry first (via `get_devices_from_registry()`); a network fetch is only triggered on a cache miss to avoid unnecessary LID-migration side effects from `get_user_devices`. The sender device is excluded (matching WA Web's `isMeDevice` in `getFanOutList`), and for self-DMs, overlapping device lists are deduplicated using a `HashSet` (matching WA Web's `Map` keyed by `toString`). - -**Own-device namespace alignment (v0.6):** When the recipient is addressed in the LID namespace (`@lid`), the client converts its own companion devices from the PN namespace to LID before fanning out. Without this alignment, a `` mix of `@lid` and `@s.whatsapp.net` participants caused the server to reject the stanza for LID-addressed DMs. Outgoing messages to PN-addressed recipients are unaffected. - - - The recipient's namespace here is decided by [`resolve_dm_wire_jid()`](#dm-wire-namespace-vs-signal-session-addressing), not a raw mapping lookup — on an account that isn't 1:1-LID-migrated, the recipient (and therefore the whole fanout) stays PN even when a LID mapping is cached. - - -Own companion devices (your other linked devices) receive per-device encryption for multi-device self-sync via `DeviceSentMessage`. - - -WA Web has a bare-`` fast path for single primary device (`WAWebSendMsgCreateFanoutStanza`). This is not implemented in whatsapp-rust because `encrypt_for_devices` always wraps in `` nodes. The `` form is accepted by the server regardless. - - - -**Fail-fast on total encrypt failure (v0.6).** If per-device encryption fails for *every* recipient device, the DM send now returns an error instead of emitting a stanza with an empty participant list (which the server would silently swallow, making the message look sent when it wasn't). A partial failure — some devices encrypt, some don't — still sends to the devices that succeeded. - - -**DM per-device locking:** - -To prevent ratchet desync when concurrent sends and receives operate on the same Signal session, the DM path acquires session locks for all devices involved — the bare recipient plus own companion devices. The `build_session_lock_keys()` helper resolves encryption JIDs and sorts them for deadlock-free lock acquisition: - -1. Resolves the recipient to its bare encryption JID via `resolve_encryption_jid().to_non_ad()` (stripping device component) -2. Resolves own companion device JIDs -3. Sorts by `(server, user, device)` using `cmp_for_lock_order()` and deduplicates -4. Returns sorted `Vec` — no intermediate `String` allocations needed for sorting - -The `session_guards_for()` helper (PR [#1131](https://github.com/oxidezap/whatsapp-rust/pull/1131)) then takes each device's lock as its mutex is resolved from the sorted JIDs, reusing a single `ProtocolAddress` via `reset_protocol_address()` to name each lock key without a formatting buffer: - -```rust -// In DM message encryption (src/send.rs) -let lock_jids = self.build_session_lock_keys(&all_dm_devices).await; -let session_guards = self.session_guards_for(&lock_jids).await; -// Each lock taken as its mutex is resolved, in sorted order, to prevent deadlocks -``` - - - Resolving every mutex first and then locking each in a second pass (the previous implementation) is still available as `session_mutexes_for()`, but only under `#[cfg(test)]` — production code no longer builds the intermediate `Vec` of mutex handles. - - -The recipient lock key is always the bare form (e.g., `100000012345678@lid.0`), matching the decrypt path's lock format. This ensures send and receive paths serialize on the exact same lock key. - -Location: `wacore/src/types/jid.rs:4-51`, `src/send.rs:1481-1507` - -### Single-buffer ProtocolAddress - -The `ProtocolAddress` struct stores the full address string `"{name}.{device_id}"` in a single `AddressBuf` buffer, with a `name_len` marker to split name from suffix. Since [#1131](https://github.com/oxidezap/whatsapp-rust/pull/1131), `AddressBuf` is inline-first: addresses up to 47 bytes (`INLINE_CAPACITY`) live inside the value itself with no heap allocation at all — every real WhatsApp address fits, e.g. `"5511987650001:5@c.us.0"` is 22 bytes. A longer address spills to a `String`. - -Which arm holds the bytes is not part of the value: `Eq`, `Ord`, and `Hash` all read the rendered string via `as_str()`, so an inline-built key finds a heap-spilled entry (and vice versa) in the session cache. That equivalence is what makes the optimization safe rather than a silent cache-miss generator. - -```rust -// wacore/libsignal/src/core/address.rs -const INLINE_CAPACITY: usize = 47; - -pub struct ProtocolAddress { - buf: AddressBuf, // inline up to 47 bytes, spills to a `String` beyond that - name_len: usize, // marks where the name ends - device_id: DeviceId, -} - -impl ProtocolAddress { - /// One-shot construction: writes `name` into the buffer and appends the suffix. - pub fn new(name: &str, device_id: DeviceId) -> Self; - - /// An address with no name yet, ready for `reset_with()`. No capacity argument — - /// the buffer starts inline and only allocates if an address ever exceeds it. - pub fn empty(device_id: DeviceId) -> Self; - - /// Rewrite the address in place via closure. Single write pass — no intermediate copy. - pub fn reset_with(&mut self, write_name: impl FnOnce(&mut AddressBuf)); - - /// Zero-cost slice of the name portion. - pub fn name(&self) -> &str; - - /// Zero-cost slice of the full buffer ("{name}.{device_id}"). - pub fn as_str(&self) -> &str; -} -``` - -Both `name()` and `as_str()` are zero-cost slices into the same buffer — no allocations on access, whether inline or spilled. - - -**API change (PR [#1131](https://github.com/oxidezap/whatsapp-rust/pull/1131)):** `ProtocolAddress::new` now takes `name: &str` instead of an owned `String`. `with_capacity(capacity, device_id)` was removed in favor of `empty(device_id)` — the buffer no longer needs a capacity hint, since it starts inline and only allocates on overflow. `reset_with()`'s closure now receives `&mut AddressBuf` instead of `&mut String`; `AddressBuf` implements `push_str`, `push`, and `std::fmt::Write`, which covers existing call sites. - - - -`Debug` on `AddressBuf` (and `ProtocolAddress`) must format `as_str()`, never the backing byte array: clearing an inline buffer only rewinds its length, so the unused tail still holds whichever address occupied it before. A derived `Debug` would print the whole array and could leak an unrelated peer's JID into a log line or error that formats a reused address. - - -### Reusable hot-loop address construction - -When iterating over many devices (e.g., during group stanza preparation or session resolution), allocating a fresh `ProtocolAddress` per device is wasteful. The `JidExt` trait provides `reset_protocol_address()` to rewrite a pre-allocated address in place, and `make_reusable_protocol_address()` creates the initial buffer: - -```rust -// wacore/src/types/jid.rs -pub fn make_reusable_protocol_address() -> ProtocolAddress { - ProtocolAddress::empty(SIGNAL_DEVICE_ID) -} - -pub trait JidExt { - /// Rewrite a reusable ProtocolAddress in place for this JID. - fn reset_protocol_address(&self, addr: &mut ProtocolAddress); -} -``` - -**Usage in group stanza preparation:** - -```rust -// wacore/src/send.rs — session resolution loop -let mut reusable_addr = make_reusable_protocol_address(); - -for device_jid in devices { - // Rewrite the same buffer — no new allocation per device - device_jid.reset_protocol_address(&mut reusable_addr); - - if stores.session_store.load_session(&reusable_addr).await?.is_some() { - // Session exists — use it - } -} -``` - -Since [#1131](https://github.com/oxidezap/whatsapp-rust/pull/1131), a fresh `ProtocolAddress` for a typical WhatsApp address (up to 47 bytes) is already allocation-free — inline storage covers it. Reuse still matters for addresses that spill to the heap: the first spill on a reused buffer still allocates its backing `String`, but every reset after that keeps the existing allocation instead of dropping and reallocating one per device — for a group with 100 participant devices past the inline limit sharing a reused buffer, that saves up to 99 heap allocations on the send path. - - - Use `to_protocol_address()` for one-shot address construction (e.g., cache keys, single lookups). Use `make_reusable_protocol_address()` + `reset_protocol_address()` when iterating over multiple JIDs in a tight loop. - - -Location: `wacore/libsignal/src/core/address.rs`, `wacore/src/types/jid.rs`, `wacore/src/send.rs` - -### Zero-Allocation JID Deduplication - -Group stanza preparation needs to deduplicate participant JIDs at two stages: before device resolution (by user identity) and after LID conversion (by device identity). Two utility functions in `wacore/src/types/jid.rs` handle this with in-place sorted dedup instead of `HashSet` allocations: - -```rust -/// Sort and deduplicate by user identity (user + server). -pub fn sort_dedup_by_user(jids: &mut Vec); - -/// Sort and deduplicate by device identity (user + server + device + integrator + identity_agent). -pub fn sort_dedup_by_device(jids: &mut Vec); -``` - -`sort_dedup_by_device` keys on `Jid::identity_agent()` rather than the raw `agent` field, so its notion of "same device" matches exactly what `Jid`'s `PartialEq`/`Hash` already treat as equal (see [Binary Protocol](/advanced/binary-protocol#jid-encoding) for why the two fields differ). That has to hold in both directions: keying on the raw `agent` would let two JIDs that are actually one device — an inert agent byte on `Pn`/`Lid`/`Hosted`/`HostedLid`, same AD-JID, same Signal address — both survive the dedup and pick up two concurrent encryption jobs against one session; dropping `agent` from the key entirely would go too far the other way and silently collapse two genuinely distinct `@bot`/`@interop` devices, which *do* render it, losing a fan-out destination. - -Both use `sort_unstable_by` followed by `dedup_by`, comparing JID fields directly without allocating intermediate strings or hash sets. This is more efficient than the `HashSet<(String, String)>` approach because: - -- No per-JID `String::clone()` for hash keys -- No `HashSet` allocation or hashing overhead -- Stable dedup order (sorted) instead of hash-dependent iteration - -**Usage in group sends (`wacore/src/send.rs`):** - -```rust -// Before device resolution — dedup participants by user identity -sort_dedup_by_user(&mut jids_to_resolve); - -// After LID conversion — dedup devices by full device identity -// Catches duplicates where both phone and LID queries resolve -// to the same device (e.g., 559980000003:33 and 100000037037034:33@lid) -sort_dedup_by_device(&mut resolved_list); -``` - -Location: `wacore/src/types/jid.rs:33-51` - -### Take/Restore Pattern - -Avoids cloning session states during decryption attempts: - -```rust -// Take ownership instead of cloning -if let Some(mut current_state) = record.take_session_state() { - let result = decrypt_message_with_state(&mut current_state, ...); - match result { - Ok(ptext) => { - record.set_session_state(current_state); - return Ok(ptext); - } - Err(e) => { - record.set_session_state(current_state); // Restore - } - } -} -``` - -Location: `wacore/libsignal/src/protocol/session_cipher.rs:495-564` - -### Buffer Reuse - -Thread-local buffers eliminate per-message allocations: - -```rust -struct EncryptionBuffer { - buffer: Vec, - usage_count: usize, -} - -const INITIAL_CAPACITY: usize = 1024; -const MAX_CAPACITY: usize = 16 * 1024; -const SHRINK_THRESHOLD: usize = 100; - -fn get_buffer(&mut self) -> &mut Vec { - self.usage_count += 1; - if self.usage_count.is_multiple_of(SHRINK_THRESHOLD) { - if self.buffer.capacity() > MAX_CAPACITY { - self.buffer = Vec::with_capacity(INITIAL_CAPACITY); - } - } - &mut self.buffer -} -``` - -Location: `wacore/libsignal/src/protocol/session_cipher.rs:20-54` - -## Public API - -The `client.signal()` accessor exposes low-level Signal protocol operations for direct use. This includes 1:1 and group encryption/decryption, session validation, session deletion, participant node creation, and device resolution. - -See [Signal API reference](/api/signal) for full method documentation and examples. - -## Related Components - -- [Binary Protocol](/advanced/binary-protocol) - How encrypted messages are serialized -- [State Management](/advanced/state-management) - How session state is persisted -- [WebSocket Handling](/advanced/websocket-handling) - Transport layer for encrypted messages -- [Signal API](/api/signal) - Public API for Signal protocol operations - -## References - -- [Signal Protocol Specification](https://signal.org/docs/) -- [libsignal Repository](https://github.com/signalapp/libsignal) -- Source: `wacore/libsignal/src/protocol/` -- Storage: `src/store/signal.rs`, `src/store/signal_adapter.rs`, `wacore/src/store/signal_cache.rs` \ No newline at end of file +FILE_CONTENT_PLACEHOLDER \ No newline at end of file From 6d55e8b1635439708cf9b42fd02e682d49680ab5 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 5 Aug 2026 17:25:15 +0000 Subject: [PATCH 2/3] docs(signal-protocol): document persisted skipped-key seeds (#1210) SessionMessageKeyMaterial's Seed/Derived variants moved to fixed-width arrays and the seed now round-trips through export instead of only ever coming back as Derived. Update the legacy v1 interop section to reflect that only seedless (pre-#1210) records still hit ChainNotRepresentable::DerivedMessageKey. Co-Authored-By: Claude Sonnet 5 --- advanced/signal-protocol.mdx | 2127 +++++++++++++++++++++++++++++++++- 1 file changed, 2126 insertions(+), 1 deletion(-) diff --git a/advanced/signal-protocol.mdx b/advanced/signal-protocol.mdx index 2e21f642..2dac8bbc 100644 --- a/advanced/signal-protocol.mdx +++ b/advanced/signal-protocol.mdx @@ -1 +1,2126 @@ -FILE_CONTENT_PLACEHOLDER \ No newline at end of file +--- +title: 'Signal Protocol Implementation' +description: 'Deep dive into end-to-end encryption, Double Ratchet algorithm, and Signal Protocol in whatsapp-rust' +--- + +## Overview + +whatsapp-rust implements the Signal Protocol for end-to-end encryption of both one-on-one and group messages. The implementation is based on Signal's libsignal library, adapted for WhatsApp's specific protocol requirements. + + + The Signal Protocol implementation handles cryptographic primitives. Any modifications to this code require expert-level understanding of cryptographic protocols to avoid security vulnerabilities. + + +## Architecture + +The Signal Protocol implementation is split across two main locations: + +- **`wacore/libsignal/`** - Platform-agnostic Signal Protocol core (Rust port of libsignal) +- **`src/store/signal*.rs`** - WhatsApp-specific storage integration with Diesel/SQLite + +### Key Components + +``` +wacore/libsignal/src/ +├── protocol/ +│ ├── session_cipher.rs # Encryption/decryption for 1:1 messages +│ ├── group_cipher.rs # Encryption/decryption for group messages +│ ├── ratchet.rs # Double Ratchet implementation +│ ├── sender_keys.rs # Sender Key protocol for groups +│ └── state/ # Session state management +└── crypto/ + ├── aes_cbc.rs # AES-256-CBC for message content + ├── aes_gcm.rs # AES-GCM for media encryption + └── hash.rs # HKDF and HMAC primitives +``` + +## Double ratchet protocol + +The Double Ratchet algorithm provides forward secrecy and post-compromise security for 1:1 messages. + +### Session Initialization + +Two participants initialize a session using Diffie-Hellman key exchange: + +```rust +// Alice initiates the session (sender) +pub fn initialize_alice_session( + parameters: &AliceSignalProtocolParameters, + csprng: &mut R, +) -> Result + +// Bob receives the session (recipient) +pub fn initialize_bob_session( + parameters: &BobSignalProtocolParameters +) -> Result +``` + +**Key Derivation:** + +1. Compute shared secrets from ephemeral key exchanges +2. Derive root key and chain key using HKDF-SHA256: + ``` + HKDF(discontinuity_bytes || DH1 || DH2 || DH3 [|| DH4]) + → (RootKey[32], ChainKey[32], PQRKey[32]) + ``` +3. Initialize sender and receiver chains + +Location: `wacore/libsignal/src/protocol/ratchet.rs:41-172` + +### Message Encryption + +Each message advances the sender chain and derives ephemeral message keys: + +```rust +// From wacore/libsignal/src/protocol/session_cipher.rs:65-183 +pub async fn message_encrypt( + ptext: &[u8], + remote_address: &ProtocolAddress, + session_store: &mut dyn SessionStore, + identity_store: &mut dyn IdentityKeyStore, +) -> Result +``` + +**Process:** + +1. Load current session state +2. Get sender chain key and derive message keys: + ```rust + let (message_keys_gen, next_chain_key) = chain_key.step_with_message_keys(); + let message_keys = message_keys_gen.generate_keys(); + // message_keys contains: cipher_key, mac_key, iv + ``` +3. Encrypt plaintext with AES-256-CBC: + ```rust + aes_256_cbc_encrypt_into(ptext, message_keys.cipher_key(), + message_keys.iv(), &mut buf) + ``` +4. Create SignalMessage with MAC for authentication +5. Advance chain key and save session state + +**Message Format:** + +- **SignalMessage**: Standard encrypted message +- **PreKeySignalMessage**: Includes prekey bundle for session establishment + + +**Plaintext padding.** Before encryption, the serialized `wa::Message` is padded with a uniform-random number of bytes in `1..=16` (the pad length is repeated as the byte value, matching WA Web's `rand % 16 + 1` and whatsmeow). v0.6 fixed a prior scheme that masked the length with `& 0x0F`, which skewed the distribution toward 15 and could never emit 16 — a subtle fingerprinting divergence from the official client. The receiver strips the padding by reading the final byte as the length. + + +### Message Decryption + +Decryption handles out-of-order delivery and tries multiple session states: + +```rust +// From wacore/libsignal/src/protocol/session_cipher.rs:292-363 +pub async fn message_decrypt_signal( + ciphertext: &SignalMessage, + remote_address: &ProtocolAddress, + session_store: &mut dyn SessionStore, + identity_store: &mut dyn IdentityKeyStore, + csprng: &mut R, +) -> Result> +``` + +**Process:** + +1. Try current session state first +2. If MAC verification fails, try previous (archived) sessions +3. Derive/retrieve message keys for the counter +4. Verify MAC: + ```rust + ciphertext.verify_mac(&their_identity_key, &local_identity_key, + message_keys.mac_key()) + ``` +5. Decrypt with AES-256-CBC +6. Promote successful session to current if needed + + + The implementation optimizes memory by using take/restore patterns to avoid cloning session states during decryption attempts (see `session_cipher.rs:495-619`). + + +### Chain key ratcheting + +Message keys are derived from chain keys, which advance with each message: + +```rust +pub struct ChainKey { + key: [u8; 32], + index: u32, +} + +impl ChainKey { + pub fn step_with_message_keys(self) -> Result<(MessageKeyGenerator, ChainKey)> { + let message_key_gen = MessageKeyGenerator::new(self.key, self.index); + let next_chain_key = self.next_chain_key()?; + Ok((message_key_gen, next_chain_key)) + } +} +``` + +Location: `wacore/libsignal/src/protocol/ratchet/keys.rs` + +### Chain key overflow protection + +The chain key index is a `u32` that increments with each message. Without overflow protection, the index could silently wrap past `u32::MAX` (4,294,967,295) back to 0, creating a counter reuse vulnerability that breaks cryptographic guarantees (nonce reuse in message key derivation). + +Both 1:1 and group chain keys use `checked_add()` to return a typed error instead of wrapping: + +```rust +// 1:1 chain keys (ratchet/keys.rs) +pub fn next_chain_key(&self) -> crate::protocol::Result { + Ok(Self { + key: self.calculate_base_material(Self::CHAIN_KEY_SEED), + index: self.index.checked_add(1).ok_or_else(|| { + SignalProtocolError::InvalidState( + "next_chain_key", + "chain key index overflow (u32::MAX)".to_string(), + ) + })?, + }) +} + +// Group sender chain keys (sender_keys.rs) +let new_iteration = self.iteration.checked_add(1).ok_or_else(|| { + SignalProtocolError::InvalidState( + "sender_chain_key_next", + "Sender chain is too long".into(), + ) +})?; +``` + + + A chain key reaching `u32::MAX` iterations indicates an abnormally long-lived session. In practice this should never occur — ratchet key rotations reset the chain counter with each new Diffie-Hellman exchange. + + +Location: `wacore/libsignal/src/protocol/ratchet/keys.rs`, `wacore/libsignal/src/protocol/sender_keys.rs` + +### Forward Jumps + +The protocol tolerates out-of-order messages up to a limit. Peer sessions and group sender-key chains match WhatsApp Web's `signalFutureMessagesMax`; a pairwise session with one of your **own** other devices gets a wider (but still bounded) ceiling, since multi-device app-sync legitimately jumps far ahead and the peer is trusted: + +```rust +// wacore/libsignal/src/protocol/consts.rs +pub const MAX_FORWARD_JUMPS: usize = 2_000; // peer sessions + group sender keys +pub const MAX_FORWARD_JUMPS_SELF: usize = 25_000; // sessions with your own other devices + +// wacore/libsignal/src/protocol/session_cipher.rs +const fn forward_jump_limit(is_self: bool) -> usize { + if is_self { MAX_FORWARD_JUMPS_SELF } else { MAX_FORWARD_JUMPS } +} + +if jump > forward_jump_limit(state.session_with_self()?) { + return Err(SignalProtocolError::InvalidMessage( + original_message_type, + "message from too far into the future", + )); +} +``` + + +Before this change, self-device sessions were exempt from the limit entirely (jumps beyond `MAX_FORWARD_JUMPS` were logged and allowed through). `MAX_FORWARD_JUMPS_SELF` (25,000) now bounds that path too — still wide enough for legitimate app-sync catch-up, but no longer unbounded. Peer sessions and group sender-key chains dropped from 25,000 to 2,000, matching WA Web's `signalFutureMessagesMax`; a message whose counter is more than 2,000 steps ahead is rejected (driving the retry-receipt path) instead of forcing thousands of KDF derivations per message. + + +Location: `wacore/libsignal/src/protocol/consts.rs`, `wacore/libsignal/src/protocol/session_cipher.rs` + +## DM device fanout + +When sending a direct message, the library resolves all known devices for both the recipient and your own account, then encrypts two different plaintexts for two categories of devices: + +- **Recipient devices** receive the actual message content +- **Own other devices** (your other linked devices) receive a `DeviceSentMessage` wrapper containing the message plus the destination JID, so your other devices can display the sent message in the correct chat + + +**Destination JID encoding via `DsmDestination` ([#1137](https://github.com/oxidezap/whatsapp-rust/pull/1137)).** The `DeviceSentMessage` wrapper writes its destination JID as a length-prefixed protobuf field, which needs the encoded length before the bytes. `wacore::messages::MessageUtils::encode_dm_plaintexts` and `dm_plaintexts_from_encoded` used to take `destination_jid: &str`, so the caller rendered the `Jid` into a `String` purely to measure and copy it. Both now take `impl DsmDestination` instead. `DsmDestination` is implemented directly on `Jid`, so it can measure and write its own wire form without an intermediate `String`. It's also implemented on `str` and the standard string wrappers (`String`, `Box`, `Rc`, `Arc`, `Cow`), carried through references of any depth via two blanket impls. The DM send path now passes `to_jid: &Jid` directly instead of `&to_jid.to_string()`. + + +### Device resolution + +The DM send path builds the full device list in a WA Web-compliant manner (matching `WAWebSendUserMsgJob` and `WAWebDBDeviceListFanout`): + +1. **Local registry first** — the client checks the local device registry via `get_devices_from_registry()` for both the recipient and own account. A network fetch (`get_user_devices`) is only triggered on a cache miss, avoiding unnecessary LID-migration side effects. +2. **Hosted device filtering** — devices flagged as hosted (via `is_hosted()`) are filtered out, matching WA Web's `DBDeviceListFanout` exclusion. +3. **Sender device exclusion** — the exact sender device is removed from the list so `ensure_e2e_sessions` never creates a self-session. This matches WA Web's `isMeDevice` check in `getFanOutList`. +4. **Self-DM deduplication** — when sending to your own account, the recipient and own device lists overlap. A `HashSet`-based dedup pass (matching WA Web's `Map` keyed by `toString`) removes duplicates. + +```rust +// Build device list — local registry first, network on miss +let mut recipient_cached = self.get_devices_from_registry(&recipient_bare).await; +if recipient_cached.is_none() { + let _ = self.get_user_devices(std::slice::from_ref(&to)).await; + recipient_cached = self.get_devices_from_registry(&recipient_bare).await; +} + +// Filter hosted devices, exclude sender, dedup for self-DMs +all_dm_jids.retain(|j| !j.is_hosted()); +all_dm_jids.retain(|j| !is_sender); +// HashSet dedup for self-DM overlap +``` + + +**Per-recipient memoization ([#1118](https://github.com/oxidezap/whatsapp-rust/pull/1118)).** The steps above — the registry lookups, the list rebuild, the partition, and the phash — are memoized per recipient in `dm_devices_memo`, keyed by the resolved wire JID and validated against the device-topology generation and the sending identity (own PN/LID). A warm repeat send to the same chat reuses the stored `ResolvedDmDevices` (an `Arc`, so a hit is a refcount bump — nothing is cloned) instead of redoing this resolution. Any device add/remove/replace, registry invalidation, PN↔LID mapping change, or re-pair invalidates the entry through the same topology tracker the existing `group_devices_memo` already uses; a resolution that had to fall back (e.g. a registry lookup miss) is never memoized. Explicitly requesting network-fresh data bypasses the memo entirely. See `dm_devices_memo` and `group_devices_memo` in [`memory_report()`](/api/client#memory_report). + + +### Device partitioning + +The `partition_dm_devices` function classifies all resolved devices into recipient and own groups, and excludes the exact sender device (the current device) entirely. It partitions `all_devices` in place — swapping recipient devices to the front of the passed-in `Vec` — instead of allocating two new device vectors per send: + +```rust +pub(crate) fn partition_dm_devices( + all_devices: Vec, + own_jid: &Jid, + own_lid: Option<&Jid>, +) -> PartitionedDmDevices + +pub(crate) struct PartitionedDmDevices { + devices: Vec, + recipient_count: usize, +} + +impl PartitionedDmDevices { + // recipient + own, sender excluded + pub(crate) fn valid_devices(&self) -> &[Jid] { + &self.devices + } + + // recipient devices only + pub(crate) fn recipient_devices(&self) -> &[Jid] { + &self.devices[..self.recipient_count] + } + + // own non-sender devices only + pub(crate) fn own_other_devices(&self) -> &[Jid] { + &self.devices[self.recipient_count..] + } +} +``` + +Since [#1118](https://github.com/oxidezap/whatsapp-rust/pull/1118), `partition_dm_devices` runs once per `dm_devices_memo` entry — wrapped by `ResolvedDmDevices::new(all_devices, own_jid, own_lid)` — rather than on every send; see [`DmStanzaRequest` and `ResolvedDmDevices`](#dmstanzarequest-and-resolveddmdevices) below. + +### Sender device exclusion + +The exact sender device is identified by matching both the user **and** device ID against your phone number JID (PN) or your Linked Identity JID (LID): + +```rust +fn is_exact_dm_sender_device(device_jid: &Jid, own_jid: &Jid, own_lid: Option<&Jid>) -> bool { + (device_jid.is_same_user_as(own_jid) && device_jid.device == own_jid.device) + || own_lid.is_some_and(|lid| + device_jid.is_same_user_as(lid) && device_jid.device == lid.device + ) +} +``` + +### Own device recognition + +After excluding the sender device, the remaining devices are classified using `matches_user_or_lid`, which checks if a device JID belongs to the same user as either your PN or LID: + +```rust +pub fn matches_user_or_lid(&self, user: &Jid, lid: Option<&Jid>) -> bool { + self.is_same_user_as(user) || lid.is_some_and(|l| self.is_same_user_as(l)) +} +``` + +This ensures that your own devices registered under your LID (common in multi-device setups) are correctly classified as "own" devices and receive the `DeviceSentMessage` plaintext — not the recipient plaintext. Without LID matching, your own LID-based devices would be misclassified as recipient devices, causing them to receive the wrong message format. + + + Both PN-based and LID-based devices must be checked because WhatsApp's multi-device architecture uses both addressing schemes. A user's devices may appear under either their phone number JID (`@s.whatsapp.net`) or their Linked Identity JID (`@lid`), depending on the device type and registration path. + + +### DmStanzaRequest and ResolvedDmDevices + +`prepare_dm_stanza` takes a `DmStanzaRequest` whose `devices` field is the already-resolved, already-partitioned fan-out for the recipient — a borrowed `&ResolvedDmDevices` — rather than a raw device list: + +```rust +pub struct DmStanzaRequest<'a> { + pub own_jid: &'a Jid, + pub account: Option<&'a wa::ADVSignedDeviceIdentity>, + pub to: &'a Jid, + pub message: &'a wa::Message, + pub message_id: &'a str, + pub edit: Option<&'a crate::types::message::EditAttribute>, + pub extra_nodes: &'a [Node], + /// The already-partitioned fan-out. Borrowed, not owned: the caller's + /// per-recipient memo hands out the same `Arc` on every repeat send, so + /// neither the device list nor its phash is rebuilt here. + pub devices: &'a ResolvedDmDevices, + pub pre_encoded: Option<&'a [u8]>, +} +``` + +`ResolvedDmDevices` (`wacore::send::ResolvedDmDevices`) wraps the partitioned device set from `partition_dm_devices` together with a lazily memoized phash — the type that `dm_devices_memo` caches per recipient: + +```rust +pub struct ResolvedDmDevices { /* partitioned devices + OnceLock phash */ } + +impl ResolvedDmDevices { + /// Partitions `all_devices` into recipient devices and own companions, + /// dropping the sending device itself. + pub fn new(all_devices: Vec, own_jid: &Jid, own_lid: Option<&Jid>) -> Self; + + /// Every device the stanza encrypts for, in partition order. + pub fn devices(&self) -> &[Jid]; + + /// The DM phash over the sent device set: a memo hit is an inline copy. + pub fn phash(&self) -> Option; +} +``` + + + **Breaking change ([#1118](https://github.com/oxidezap/whatsapp-rust/pull/1118)):** `DmStanzaRequest::own_lid` was removed — the sending identity's LID is now baked into the `ResolvedDmDevices` passed via `devices` at construction time (`ResolvedDmDevices::new`) — and `DmStanzaRequest::devices` changed from `Vec` to `&ResolvedDmDevices`. An out-of-tree caller constructing `DmStanzaRequest` directly needs to build a `ResolvedDmDevices` first (via `ResolvedDmDevices::new(all_devices, own_jid, own_lid)`) and pass it by reference. + + +### Session preflight and the 406 case + +Before `prepare_dm_stanza` builds the stanza, `ensure_e2e_sessions` runs as a preflight that fetches prekeys for any resolved device without an established Signal session — one IQ covering up to `SESSION_CHECK_BATCH_SIZE` (50) devices. The server can reject devices two ways: **by name**, returning a `` node whose bundle is replaced with an `` for that one device, or **batch-wide**, failing the IQ itself with a `406` that names nobody. + + +**A named `406` rejection now refreshes only that device's user, not the whole batch ([#1153](https://github.com/oxidezap/whatsapp-rust/pull/1153), closes [#1143](https://github.com/oxidezap/whatsapp-rust/issues/1143)).** `PreKeyUtils::parse_prekeys_response` returns a `PreKeyFetchOutcome { bundles, rejected }` instead of a bare bundle map. `rejected` carries the devices the server named directly in the response body, each with the `` code it was rejected with. When one of those codes is `406`, the preflight calls `invalidate_device_caches_for` with the named device JIDs. That call dedupes the JIDs to their distinct users and refreshes each user's whole device-list cache entry, since the cache is keyed per user rather than per individual device. The send proceeds afterward, and every device that did return a bundle is unaffected — so one rejected device no longer costs the rest of the batch. + +This corrects the premise of [#1139](https://github.com/oxidezap/whatsapp-rust/pull/1139) (closed [#1135](https://github.com/oxidezap/whatsapp-rust/issues/1135)). #1139 assumed the response "doesn't say which one" and so invalidated every distinct user in the batch on any `406`. WA Web's own parser (`FetchKeyBundlesUserError` in `WAWeb/Fetch/PrekeysJob`) already reads this per-device error. whatsapp-rust's parser fed the same node to the bundle parser instead: it failed on the missing fields, logged the failure as a malformed bundle, and discarded the code. The device was skipped, but its stale entry was never refreshed, so the next send resolved the same absent device again. #1139's whole-batch invalidation and propagated `?` error still apply to the one case a named rejection cannot cover: a `406` on the IQ itself. + +The asymmetry #1139 described between the DM and group paths is gone — DM handled at the preflight, group already covered by the per-device fan-out (`stale_device_users`). `SendContextResolver::fetch_prekeys_for_identity_check` now returns `wacore::prekeys::PreKeyFetchOutcome` instead of `HashMap`. If you implement a custom `SendContextResolver`, update this method's return type to match. The new type carries a named rejection into the group fan-out's `EncryptResult::rejected_devices`, so `stale_users_for` refreshes exactly those users instead of inferring staleness from whichever targets went unencrypted for unrelated reasons (missing bundle, malformed bundle, failed session setup). Both paths now act on the same named-device signal when the server provides it. + + +### PreparedDmStanza + +`prepare_dm_stanza` returns a `PreparedDmStanza` struct containing the stanza node and the locally computed phash for server ACK validation: + +```rust +pub struct PreparedDmStanza { + pub node: Node, + /// Locally computed phash from the sent device set. Not sent on the + /// wire (WA Web only sends phash for groups). Used by the caller to + /// compare against the server's ACK phash for device-list drift detection. + pub phash: Option, +} +``` + + + `phash` changed from `Option` to `Option` (`wacore_binary::CompactString`) in [#1118](https://github.com/oxidezap/whatsapp-rust/pull/1118), matching the type `ResolvedDmDevices::phash()` returns: a warm memo hit clones the cached `CompactString` inline instead of allocating a new `String`. + + +The phash is computed from the actual sent device set (after partitioning, with the sender excluded) using `MessageUtils::participant_list_hash()`. Unlike group messages, the DM phash is **not** sent on the wire — WA Web only includes `phash` in the `DeviceSentMessage` for groups. The DM phash is used purely for local validation against the server's ACK to detect device-list drift. + + + The `DeviceSentMessage.phash` field is set to `None` for DMs, matching WA Web's behavior where only group `DeviceSentMessage` wrappers include a phash. The DM phash is computed and tracked separately by the caller. + + +Location: `wacore/src/send.rs:675-820`, `src/send.rs` + +## PN→LID session migration + +WhatsApp's multi-device architecture uses two addressing schemes: phone number JIDs (PN, `@s.whatsapp.net`) and Linked Identity JIDs (LID, `@lid`). WhatsApp Web always resolves PN→LID before any session operation via `createSignalAddress()`. whatsapp-rust mirrors this behavior — when a LID mapping is discovered for a phone number, any Signal sessions stored under the PN address are automatically migrated to the corresponding LID address. + + + The automatic migration described below is also exposed for manual invocation: [`Signal::migrate_sessions(from, to)`](/api/signal#migrate_sessions) runs the same move for a caller-chosen JID pair, and [`Signal::session_info(jid)`](/api/signal#session_info) inspects a session (migrating a legacy PN-addressed one first if needed) without mutating it further. See the [Signal API reference](/api/signal) for both. + + +### Signal address resolution + +`Client::resolve_encryption_jid()` mirrors WA Web's `SignalAddress.toString()` (`WAWeb/Signal/Address.js`). It upgrades the JID's `server` to its LID counterpart when a mapping is known, and otherwise returns the input unchanged: + +| Input `Server` | Resolved `Server` (mapping known) | No mapping | +|----------------|------------------------------------|------------| +| `Pn` | `Lid` | `Pn` (preserved) | +| `Hosted` | `HostedLid` | `Hosted` (preserved) | +| Any other | unchanged | unchanged | + +The `device`, `agent`, and `integrator` fields always round-trip — only the `user` (replaced with the LID user) and `server` change. This keeps Cloud API / Meta Business hosted devices on a hosted-flavored LID address rather than collapsing them into the standard `@lid` server, matching WA Web's per-device session keying. + + + `resolve_encryption_jid()` upgrades PN → LID unconditionally whenever a mapping is known — it governs Signal **session** addressing only, matching WA Web's `SignalAddress.toString()`. The outbound DM **wire** namespace (the stanza `to`, ``, and `DeviceSentMessage` destination) is a separate, account-level decision — see [DM wire namespace vs. Signal session addressing](#dm-wire-namespace-vs-signal-session-addressing) below. + + +### DM wire namespace vs. Signal session addressing + +Since v0.6.x (fix for [#941](https://github.com/oxidezap/whatsapp-rust/issues/941)), a DM's outer `` / `` addressing is no longer derived directly from `resolve_encryption_jid()`. Some accounts are not yet **1:1-LID-migrated** on WhatsApp's servers, and those accounts get every LID-addressed DM rejected with `ack error="400"` even though the underlying Signal session is correctly LID-keyed. + +`Client::resolve_dm_wire_jid()` (`src/client/lid_pn.rs`) makes this account-level decision, mirroring WA Web's `Lid1X1MigrationUtils.isLidMigrated()` / `WAWebMessageDestinationChat`: + +```rust +pub(crate) async fn resolve_dm_wire_jid(&self, to: &Jid) -> Jid { + if self.is_lid_migrated().await { + return self.resolve_encryption_jid(to).await.into_non_ad(); + } + let bare = to.to_non_ad(); + if bare.is_lid() { + self.swap_pn_lid_namespace(&bare).await.unwrap_or(bare) + } else { + bare + } +} +``` + +- **Migrated account** (`Client::is_lid_migrated()` is `true`): behaves exactly like before — the wire namespace upgrades PN → LID whenever a mapping is known. +- **Unmigrated account**: DMs stay on PN even with a cached LID mapping. A caller-supplied LID with a known PN mapping is mapped back to the PN chat; a LID with no cached mapping is sent as-is (there is no reverse LID→PN network resolution, matching WA Web). + +`Client::is_lid_migrated()` is `true` when either is true: + +1. The persisted `Device.lid_migrated` flag (see [Storage — DeviceStore](/concepts/storage#devicestore)), set once from the primary's `pair-success` `` (`isChatDbLidMigrated`) or from a `lid_migration_mapping_sync_message` protocol message pushed to the primary's own companions (self-only — see [Authentication — one-to-one LID migration state](/concepts/authentication#one-to-one-lid-migration-state)). +2. The `lid_one_on_one_migration_enabled` ab prop, as a fallback for sessions paired before the flag existed. The first observation of this prop being on also latches the persisted flag, so the account doesn't flap back to PN addressing before the next props fetch. + +Once set, `lid_migrated` never reverts for the same account — only pairing a *different* account onto the same store resets it. Signal session addressing (`resolve_encryption_jid`) and inbound decrypt are unaffected by any of this; only the outbound DM wire namespace is gated. + + + This gate applies to 1:1 DMs only. Group sends, which already address everything by the group's own `AddressingMode`, are untouched. + + +### Why migration is needed + +After pairing, the primary phone may initially establish sessions under a PN address. Once the LID mapping becomes known (from usync, incoming messages, or device notifications), the phone begins sending from the LID address. Without migration, the client holds a session under the PN address but receives messages addressed to the LID — causing `SessionNotFound` decryption failures. + +### Proactive migration at LID discovery + +When a new LID-PN mapping is learned (via `add_lid_pn_mapping`), the client scans devices 0–99 for PN-keyed sessions and migrates them. All reads and writes go through the `SignalStoreCache` rather than the backend directly — this prevents reading stale data when the cache has unflushed mutations (e.g., after SKDM encryption ratcheted the session). The migrated state is flushed to the backend at the end so it survives restarts. + +```rust +// src/client/lid_pn.rs +pub(crate) async fn migrate_signal_sessions_on_lid_discovery(&self, pn: &str, lid: &str) { + for device_id in 0..=99u16 { + // Read from signal_cache (authoritative over backend) + // If PN session exists and no LID session → move session to LID via cache + // If both exist → delete the stale PN session from cache + // Identity keys are migrated independently of sessions + } + // Flush migrated state to backend so it survives restarts + self.signal_cache.flush(backend.as_ref()).await; +} +``` + +**Migration rules per device:** + +| PN session | LID session | Action | +|-----------|-------------|--------| +| Exists | Does not exist | Move session and identity from PN→LID address | +| Exists | Exists | Delete stale PN session (LID takes precedence) | +| Does not exist | Any | No action | + +Identity keys are migrated independently of sessions — they can outlive deleted sessions and survive session re-establishment. + + + The migration reads through the cache because the backend may contain stale session data when unflushed cache mutations exist. Reading directly from the backend could skip in-flight ratchet advances, causing the migrated session to decrypt with an outdated chain key. + + + + `add_lid_pn_mapping` also has a batch form, `Client::add_lid_pn_mappings(mappings, source)`, which durably records many LID↔PN pairs in one call and runs the same per-mapping migration as the single-entry path. It returns how many mappings were actually written, deduplicated against existing records. + + +### On-the-fly migration during decryption + +If a message arrives from a LID address and decryption fails with `SessionNotFound` or `InvalidPreKeyId`, the client attempts PN→LID migration as a fallback before requesting a retry: + +1. Look up the PN for the sender's LID +2. Attempt to migrate PN sessions to LID via the signal cache (same cache-first logic as proactive migration) +3. Retry decryption with the migrated session (already in the cache — no reload needed) +4. If `DuplicateMessage` occurs during post-migration retry, it is silently ignored +5. Fall back to retry receipt only if migration does not resolve the issue + +The `InvalidPreKeyId` case occurs when a `PreKeyMessage` references a consumed one-time prekey, but the session actually exists under a PN address (legacy migration). Migrating the session lets Signal use the existing ratchet state instead of looking up the consumed prekey. This migration is attempted in both the identity-change retry path and the initial decryption path. + +This ensures existing databases are fixed without requiring re-pairing. + +### Login-time session check + +At login, the client checks the session state of own device 0 (primary phone): + +- **LID session exists** — no action needed +- **PN session only** — logged; migration deferred to first message via on-the-fly path +- **No session** — will be established on first message exchange + +```rust +// src/client/sessions.rs +pub(crate) async fn establish_primary_phone_session_immediate(&self) -> Result<()> { + // Checks LID session → logs PN-only state → defers migration to message path +} +``` + + + Both migration paths route through the `SignalStoreCache`, ensuring they see the latest in-memory state. The proactive migration runs when a LID mapping is first discovered and flushes to the backend afterward. The on-the-fly migration handles the case where the database already contains stale PN sessions from before the mapping was known. + + +Location: `src/client/lid_pn.rs`, `src/client/sessions.rs`, `src/message.rs` + +## Sender keys (group encryption) + +Groups use the Sender Key protocol for efficient multi-recipient encryption. + +### Sender key address normalization + +Sender key records are keyed by a composite `SenderKeyName` containing the group JID and a sender protocol address string. WhatsApp delivers group stanzas with **inconsistent sender addressing** — the `pkmsg` (which carries the SKDM) arrives with a device-qualified participant JID (e.g., `100000000000001.1:75@lid`), while the `skmsg` (the actual encrypted group message) arrives with a bare participant JID (e.g., `100000000000001.1@lid`). + +Without normalization, the sender key would be stored under the device-qualified address during SKDM processing but looked up under the bare address during `skmsg` decryption, causing `NoSenderKeyState` failures. + +The client normalizes the sender JID to its bare form using `to_non_ad()` (which strips the device component, setting `device = 0, agent = 0`) at every point where a `SenderKeyName` is constructed. The `SenderKeyName::from_jid()` convenience method handles the `to_string()` conversion automatically: + +```rust +// Decryption path (src/message.rs) — normalize before group_decrypt +let sender_for_sk = info.source.sender.to_non_ad(); +let sender_address = sender_for_sk.to_protocol_address(); +let sender_key_name = SenderKeyName::from_jid(&info.source.chat, &sender_address); + +// SKDM storage path (src/message.rs) — normalize before process_sender_key_distribution_message +let sender_bare = sender_jid.to_non_ad(); +let sender_address = sender_bare.to_protocol_address(); +let sender_key_name = SenderKeyName::from_jid(&group_jid, &sender_address); +``` + +`SenderKeyName::from_jid()` is equivalent to `SenderKeyName::new(group_jid.to_string(), sender_address.to_string())` but avoids the manual `to_string()` calls and is the preferred constructor. + +This ensures the cache key is always in the form `"{group}:{bare_user}@{server}.0"`, regardless of whether the original stanza used a device-qualified or bare JID. + + + Custom implementations that construct `SenderKeyName` directly must also normalize the sender JID to its bare form. Failing to do so will cause sender key lookup mismatches and decryption failures for group messages. + + +Location: `src/message.rs`, `wacore/libsignal/src/store/sender_key_name.rs`, `wacore/binary/src/jid.rs` (`to_non_ad()`) + +### Sender key distribution + +Each participant generates and distributes a sender key: + +```rust +// From wacore/libsignal/src/protocol/group_cipher.rs:283-336 +pub async fn create_sender_key_distribution_message( + sender_key_name: &SenderKeyName, + sender_key_store: &mut dyn SenderKeyStore, + csprng: &mut R, +) -> Result +``` + +**Structure:** + +- **Chain ID**: Random 31-bit identifier for this sender key session +- **Iteration**: Message counter (starts at 0) +- **Chain Key**: 32-byte seed for deriving message keys +- **Signing Key**: Ed25519 public key for message authentication + +### Group Encryption + +Messages are encrypted with the sender's current chain key: + +```rust +// From wacore/libsignal/src/protocol/group_cipher.rs:53-116 +pub async fn group_encrypt( + sender_key_store: &mut dyn SenderKeyStore, + sender_key_name: &SenderKeyName, + plaintext: &[u8], + csprng: &mut R, +) -> Result +``` + +**Process:** + +1. Load sender key state for the group +2. Derive message keys from current chain key +3. Encrypt with AES-256-CBC +4. Sign message with Ed25519 private key +5. Advance chain key + +### Group Decryption + +Recipients decrypt using the sender's distributed key: + +```rust +// From wacore/libsignal/src/protocol/group_cipher.rs:162-250 +pub async fn group_decrypt( + skm_bytes: &[u8], + sender_key_store: &dyn SenderKeyStore, + sender_key_name: &SenderKeyName, +) -> Result> +``` + +**Process:** + +1. Parse SenderKeyMessage +2. Look up sender key state by chain ID +3. Verify Ed25519 signature +4. Derive message keys for iteration (handling out-of-order) +5. Decrypt with AES-256-CBC + + + Group decryption maintains up to MAX_FORWARD_JUMPS (2,000) cached message keys per sender. This prevents resource exhaustion attacks but limits tolerance for extreme out-of-order delivery. + + +### Unknown device detection + +During group message decryption, the client checks whether the sender's device is present in the local device registry via `is_from_known_device()`. This detection triggers in two places within the group message processing path: + +1. **After successful `skmsg` decrypt** — if the sender device is not in the registry, the decrypted message is still **processed and delivered normally**. Signal decryption success already proves the sender holds a valid session key, so discarding the message would only add latency via an unnecessary retry round-trip. A background device sync is triggered to update the local device registry. +2. **After a `NoSenderKeyState` error** — if the sender device is unknown, the retry reason is upgraded from `NoSession` to `UnknownCompanionNoPrekey` + +In both cases, the client queues a device list synchronization for the sender's user JID. The behavior depends on the connection state: + +- **Online**: the client immediately invalidates the cached device registry for the user and fires a background usync request to refresh the device list +- **Offline** (during offline sync): the unknown device's user JID is batched into a `PendingDeviceSync` set, which is flushed after offline sync completes (see [Deferred device sync](/concepts/architecture#deferred-device-sync)) + +Primary devices (device ID 0) are always treated as known — the check only applies to companion devices. + +This mechanism ensures that group messages from newly-paired companion devices are delivered immediately without waiting for a retry round-trip. The background device sync updates the local registry so future messages from the same device are recognized directly. + +```rust +// src/message.rs — simplified flow +async fn handle_unknown_device_sync(&self, info: &Arc) { + let user_jid = info.source.sender.to_non_ad(); + if !self.pending_device_sync.add(user_jid.clone()).await { + return; // already queued, dedup + } + if info.is_offline { + return; // batched for deferred flush + } + // Online: immediate sync + self.invalidate_device_cache(&user_jid.user).await; + self.get_user_devices(&[user_jid]).await.ok(); +} +``` + +Location: `src/message.rs`, `src/client/device_registry.rs`, `src/pending_device_sync.rs` + +### Retry receipt from unknown group device + +When the client receives a retry receipt, `handle_retry_receipt` checks whether the requesting device is present in the local device registry. Previously the handler dropped all retries from unregistered devices — this was safe for WA Web because WA Web keeps participant device lists fresh via a pre-send sync, so any legitimate requester is already known before the send. + +For a library client, a participant device can legitimately be absent from the local registry: if the device joined between the last device-list sync and the group send, it will have received the `skmsg` from the server but never obtained a sender key, causing it to retry indefinitely. The retry receipt may carry a `` bundle — the ADV-signed `device-identity`, the identity key, a one-time prekey, and the signed prekey — which is everything needed to establish a Signal session and resend. But a newly-linked device that has no bundle still retries forever if the client only drops it: the reconciliation that fires when a prekey fetch returns 406 never triggers for that device because it was never in the send set. + +Whenever a retry arrives from an unknown device, `handle_retry_receipt` now calls `schedule_unknown_device_sync` **before** consulting `should_drop_unknown_device_retry`. This treats the retry as a staleness signal: the requester's user JID is enqueued for a device-list resync (deduplicated via `PendingDeviceSync`, so a retry storm from a single device cannot fan out into a usync storm). Once the resync completes, the device appears in the registry and future sends include it in the sender-key distribution — the retries stop. This mirrors WA Web's `syncDeviceListJob` trigger on the retry path. + +The drop predicate still controls whether the *current* retry is recovered or dropped: + +```rust +// wacore/src/protocol/retry.rs +pub fn should_drop_unknown_device_retry(keys_present: bool, device_known: bool) -> bool { + !keys_present && !device_known +} +``` + +| `keys_present` | `device_known` | Result | +|---|---|---| +| `true` | `false` | **Recover** — build a session from the embedded bundle and resend; resync also triggered | +| `false` | `false` | **Drop** — no bundle to recover this message; device-list resync triggered so device is learned for the next send | +| any | `true` | **Resend** — device is in registry, proceed normally | + +When the bundle includes a ``, `process_retry_key_bundle` validates the ADV chain against the requester's account key (using the stored primary identity as a fallback when the server omits `account_signature_key`). A present-but-invalid ADV result is a hard error; the session is not built. If `` is absent from the bundle, or if no account key can be found, the check is skipped with a warning and the session is built anyway — matching the behaviour of the regular prekey-fetch path. The drop predicate only gates on syntactic `` presence, so the ADV guarantee is conditional on the bundle including a well-formed ``. This mirrors whatsmeow's approach of building the prekey session directly from the receipt bundle without a device-registry gate. + +Location: `src/retry.rs`, `wacore/src/protocol/retry.rs`, `src/pending_device_sync.rs` + +### Immutable sender key loading + +The `SenderKeyStore` trait's `load_sender_key` method takes `&self` (not `&mut self`), allowing sender key lookups to proceed under a **read lock**. This is safe because loading a sender key is a pure read operation — no state is mutated. The `store_sender_key` method still requires `&mut self` since it modifies state. + +This means concurrent group decryptions for different senders can load sender keys in parallel without contention, while writes (SKDM processing) still serialize correctly. + + +If you implement `SenderKeyStore` for a custom backend, `load_sender_key` must use `&self` (immutable reference). Implementations that previously required `&mut self` for internal caching should use interior mutability (e.g., `Mutex` or `RwLock`) instead. + + +### Sender key existence check + +Before distributing sender keys, the group message path checks whether the local sender key already exists. This check uses the `SignalStoreCache` with a **read lock** (`get_sender_key()`), matching the status broadcast path. This avoids acquiring a write lock and prevents unnecessary SKDM re-distribution on every group send. + +### Per-device sender key tracking + +To avoid resending Sender Key Distribution Messages on every group message, the client tracks sender key distribution status **per device** for each group. This uses a unified `sender_key_devices` table (see [Storage - ProtocolStore](/concepts/storage#protocolstore)) that matches WhatsApp Web's `participant.senderKey Map` model — a single boolean per device per group indicating whether that device has a valid sender key (`true`) or needs fresh SKDM distribution (`false`). + +The tracking update is **deferred until after the server acknowledges** the message stanza. This matches WhatsApp Web's behavior where `markHasSenderKey()` is only called after the server confirms receipt. + +**Why deferred?** If the tracking were updated immediately after building the stanza (but before sending), a network failure between stanza build and send would leave stale entries — devices would be marked as having the sender key when they never actually received it. Subsequent messages would skip SKDM for those devices, causing decryption failures. + +**`PreparedGroupStanza` return value:** + +`prepare_group_stanza` returns a `PreparedGroupStanza` struct containing the stanza `node` and a `skdm_devices: Vec` field listing exactly which devices received SKDM in this stanza. This eliminates the need for callers to re-resolve devices after sending, closing a race window where the device list could change between stanza preparation and post-ACK tracking update. + +```rust +pub struct PreparedGroupStanza { + pub node: Node, + /// Devices that received SKDM in this stanza. Empty when no SKDM was distributed. + pub skdm_devices: Vec, +} +``` + +**Implementation:** + +- **Group path:** After `send_node()` succeeds, the caller uses the `skdm_devices` list from `PreparedGroupStanza` to call `set_sender_key_status(group, devices, true)`. No re-resolution needed. +- **Status path:** A late-init boolean tracks whether full distribution occurred. The sender key tracking is only updated after the status stanza is successfully sent. +- **Error recovery:** If `prepare_group_stanza` fails with `NoSenderKeyState`, all sender key device tracking for that group is cleared and the send is retried with full distribution. +- **Sender key rotation:** On `rotateKey`, the Signal sender key is also deleted for forward secrecy (matching WhatsApp Web's `deleteGroupSenderKeyInfo`), and all device tracking is cleared via `reset_sender_key_device_tracking` — a DB-first clear with a cold-mark fallback (see below). +- **Group `` notification (number/LID migration):** A `w:gp2` `` notification (a participant's number or LID changed) unconditionally force-rotates the own group sender key and invalidates both the persisted and in-memory group metadata cache, matching WhatsApp Web's `modifyParticipantInfo` (`rotateKey: true`). The next send regenerates and redistributes a fresh sender key against the current participant list instead of risking a stale entry for the migrated device. See `Client::force_rotate_own_sender_key`, `src/handlers/notification/groups.rs`. + +**Incremental targeting:** + +Rather than distributing the sender key to all group devices on every message, the client: +1. Loads the per-device sender key map — first checking the in-memory cache, falling back to the database via `get_sender_key_devices` +2. Resolves all current group participant devices +3. Computes the diff — only devices with `has_key=false` or not yet tracked receive the SKDM +4. Passes the targeted device list to `prepare_group_stanza` via the `skdm_target_devices` parameter + + +On the **first** group send (or any send where the cached map is empty), the filter still runs unconditionally — every resolved participant device is treated as `has_key=false` and receives the SKDM. This matches WhatsApp Web, which iterates an empty `senderKey` Map as `false` per participant. There is no early-exit for an empty cache; otherwise the very first message after a fresh start would skip distribution entirely. + + + +**Own devices are never marked `has_key=true` ([#999](https://github.com/oxidezap/whatsapp-rust/pull/999)).** The post-ACK warm mark excludes the account's own companion devices, matching WhatsApp Web's `!isMeDevice` guard on `markHasSenderKey`. They therefore never leave the "not yet tracked" bucket above and are re-included as SKDM targets on every send — see the follow-up note under ["Parallelized group encrypt fan-out"](#parallelized-group-encrypt-fan-out) for why. External group members are unaffected: a successful distribution still marks them warm. + + +Location: `src/send.rs`, `src/client/sender_keys.rs`, `wacore/src/send.rs` + +### Parallelized group encrypt fan-out + +The group send path no longer serializes encryption behind a client-level lock. `prepare_group_stanza` and `encrypt_for_devices` now take an explicit `&runtime` handle (`&*self.runtime`) so per-device encryption can run on `runtime::blocking()` tasks concurrently. Combined with the move to `update_device_lists` (batched device-registry writes) and a no-lock `IdentityAdapter::is_trusted_identity` stub, group fan-out scales with the runtime's worker count rather than with a single critical section. + +This is an internal performance change — no public method on `Client::send_message` was renamed, and the order of `` children in the resulting stanza is unchanged. If you implemented a custom `SignalStore`, note that `update_device_lists(records: Vec)` is now part of the trait so the fan-out can batch its writes. + + +While *per-device* encryption runs concurrently, the sender-key chain is protected by **two separate locks per `(group, sender)` pair**: + +1. **Session-setup lock** (`SenderKeyStore::session_setup_lock`) — held only across `ensure_sessions_for_devices` (prekey fetch + X3DH). May span network I/O. Warm sends (no SKDM needed) never take it, so they are never blocked by a cold send's network round-trip. + +2. **Chain lock** (`SenderKeyStore::sender_key_lock`) — held across SKDM creation + pairwise encrypt fan-out + `skmsg` encrypt. Pure CPU; never spans network I/O. This is the invariant that prevents two concurrent sends from splitting the key between the SKDM and the `skmsg`. + +Prior to [#807](https://github.com/oxidezap/whatsapp-rust/pull/807), a single chain lock covered both phases, causing concurrent group sends to serialize behind a server round-trip whenever a new session needed to be established. Now only the CPU phase is in the critical section. Different groups (or different senders) encrypt fully in parallel, unchanged. + +`encrypt_for_devices` is composed of two public halves: `ensure_sessions_for_devices` (network, returns `SessionPlan`) and `encrypt_for_devices_with_sessions` (CPU, consumes `SessionPlan`). The DM path calls `encrypt_for_devices` unchanged; the group path calls them separately with the chain lock taken only around the second. + + + +**Per-device session lock around the SKDM fan-out (v0.6).** The chain lock above only serializes the sender-key chain — it does not cover the *pairwise* Signal sessions that `encrypt_for_devices_with_sessions` mutates for each SKDM target device. Those are the same pairwise sessions the DM path locks (see "DM per-device locking" under [Single-allocation session lock keys](#single-allocation-session-lock-keys) below) via `session_lock_for()` / `session_guards_for()`. Before [#990](https://github.com/oxidezap/whatsapp-rust/pull/990), the group fan-out held only the chain lock, a disjoint key, so a concurrent DM (or another group send) sharing a device could race that device's pairwise ratchet — both sides load chain index *N* and both store *N+1*, silently dropping one advance. When the lost advance carried the SKDM, that member never received the sender key and every subsequent `skmsg` stayed undecryptable for it until a retry re-distributed. + +`prepare_group_stanza` now acquires the SKDM targets' per-device session locks through `SendContextResolver::lock_device_sessions()` before taking the chain lock, and releases them right after the fan-out — the `skmsg` chain encrypt that follows only touches the sender-key chain, never a pairwise session. The `Client` implementation of this hook reuses `build_session_lock_keys()` + `session_guards_for()`, so the group and DM paths serialize on the exact same mutexes, in the same sorted order, and always acquire session locks before the chain lock — no path takes the reverse order, so this cannot deadlock. The hook defaults to a no-op, so a custom `SendContextResolver` (as used in tests and benches) is unaffected unless it opts in. + + + +**Session-setup failures are isolated per device (v0.6).** `ensure_sessions_for_devices` used to abort with `Err` the moment `process_prekey_bundle` failed for *any* one target device. Since `prepare_group_stanza` gates the entire SKDM fan-out on `session_plan.is_some()`, one device's X3DH failure nulled the plan and **every** device in the cohort — not just the failing one — got no SKDM, even though the `skmsg` still shipped and the phash covered the full set. An external member recovers via a retry receipt, but an own companion's retry hits `mark_forget_sender_key` with `exclude_own_devices=true`, which filters own-user JIDs and returns early — so that companion stayed `has_key=true` forever and couldn't decrypt the group from that device until an unrelated full rotation (participant removal or PN↔LID migration). + +As of [#996](https://github.com/oxidezap/whatsapp-rust/pull/996), a device whose session setup fails is logged and skipped rather than aborting the plan — matching WhatsApp Web's `GroupKeyDistributionMsg`, which wraps each device's `ensureE2ESessions` in its own try/catch and drops only the failing one. The sessionless device is then naturally excluded by the encrypt fan-out (which already skips devices without a session), so every other device still receives its pairwise SKDM. + +[#996](https://github.com/oxidezap/whatsapp-rust/pull/996) closed the primary harm — an *unrelated* device's setup failure no longer suppresses the whole cohort's SKDM. A narrower window remained: the **warm mark** (`update_sender_key_devices`, called after the server ACK) recorded the *full* distribution target as `has_key=true`, including our own companion devices, regardless of whether each one's pairwise SKDM encryption actually succeeded. Since the forget path (`mark_forget_sender_key`) excludes own devices for the reason above, an own companion whose one SKDM encryption failed — or that was warm-marked without ever receiving a node — was marked warm and could **never** be un-marked: a permanent orphan until an unrelated full rotation. External devices didn't have this problem; they recover through the retry-receipt forget path. + +[#999](https://github.com/oxidezap/whatsapp-rust/pull/999) closes this residual by excluding own devices from the warm mark too (`exclude_own_devices=true`), mirroring WhatsApp Web's `ParticipantStore` helper, which guards *both* `markHasSenderKey` and `markForgetSenderKey` with the same `!isMeDevice` check. Own companions are therefore never memoized as `has_key=true` — `filter_skdm_targets` (["Per-device sender key tracking → Incremental targeting"](#per-device-sender-key-tracking) above) always re-includes them, so they get a fresh SKDM on every group send. This is a deliberate trade-off (a few extra pairwise SKDM nodes per send when the account has companions) in exchange for making the orphan impossible. External devices are unaffected: a successful distribution still marks them warm, and the retry-receipt path still repairs any that go stale. + + + +**The group distribution lane now guards the full audit-reset-redistribute sequence, not just the SKDM fan-out ([#1043](https://github.com/oxidezap/whatsapp-rust/pull/1043)).** Previously `Client::group_distribution_lock()` (see ["Parallelized group encrypt fan-out"](#parallelized-group-encrypt-fan-out) above) was taken only around the cold SKDM send itself. Sender-key deletion (participant-removal rotation, forced own-key rotation), per-device tracker resets, and the status-broadcast distribution path could run concurrently with that lock held elsewhere, letting an encrypt racing a rotation restore a retired key after deletion, or a tracker reset race stale delivery marks back onto a new chain. + +`rotate_sender_key_on_participant_remove`, `force_rotate_own_sender_key` (now taking `&Jid` instead of a pre-stringified group ID), warm group sends, status sends, phash-mismatch recovery, and periodic sender-key rotation all now hold the same per-group lane across their own-key delete/reset and the following redistribution. A rotation that arrives while a send is mid-fan-out waits for the lane instead of deleting the chain state out from under it; a send that arrives mid-rotation waits for the rotation to finish before re-auditing device state. Held lanes are never capacity-evicted, so a live rotation or fan-out cannot be silently dropped from the map mid-operation (see `group_distribution_locks_capacity` in the [Cache Configuration reference](/api/bot#cache-configuration-reference)). + + + +**Sender-key tracker resets are DB-first ([#1043](https://github.com/oxidezap/whatsapp-rust/pull/1043)).** `reset_sender_key_device_tracking` replaces the old direct `clear_sender_key_devices` + cache-invalidate call at every rotation and redistribution site. It clears the per-device tracking row-by-row in the database first, and only invalidates the in-memory `SenderKeyDeviceCache` after that durable clear succeeds. If the DB clear fails, every existing tracked row is instead marked cold (`has_key=false`) as a fallback so the next send still re-distributes; if that fallback write also fails, the operation returns an error and the send stays fail-closed rather than risking a stale `has_key=true` row surviving onto a freshly rotated chain. + +The unknown-participant rotation in [retry receipt handling](/advanced/retry-admission) is a special case: `handle_retry_receipt` deletes the own sender key and resets tracking for a `` from an unrecognized group participant, then must still fall through to the per-chat resend rate limiter and other throttles further down the same function. The signal cache is now explicitly flushed right after the rotation — before any later throttle can return early — so a rotation is never left un-persisted by an unrelated early exit later in the same call. + + + +**Observability: distribution-lane pressure is exposed on `memory_report()` ([#1043](https://github.com/oxidezap/whatsapp-rust/pull/1043)).** `Client::memory_report()` now reports `group_distribution_locks` (live lane count), `group_distribution_lock_evictions` (cumulative cold evictions), and `group_distribution_lock_eviction_blocks` (cumulative evictions skipped because the lane was live) — see [`memory_report()`](/api/client#memory_report). These update only under capacity pressure and add no allocation or per-message cost below the soft cap. + + +### In-memory sender key device cache + +The `SenderKeyDeviceCache` provides an in-memory caching layer over the per-device sender key tracking data stored in the database. Without this cache, every group send would require a database round-trip to load the sender key device map — the cache eliminates that overhead after the first load for each group. + +```rust +pub(crate) struct SenderKeyDeviceCache { + inner: Cache>, +} +``` + +**Key design decisions:** + +- **Time-to-idle eviction:** The cache uses TTI semantics (default: 1 hour, 500 entries), so entries for inactive groups are automatically evicted while frequently-used groups stay cached +- **Pre-parsed, pre-indexed maps:** Database rows are parsed into a `SenderKeyDeviceMap` struct that provides O(1) lookups by user and device ID, avoiding per-query string parsing +- **Single-flight initialization:** The `get_or_init` method uses `PortableCache`'s single-flight `get_with` — if multiple concurrent group sends for the same group trigger a cache miss simultaneously, only one database read executes and all callers share the result +- **Explicit invalidation:** The cache is invalidated when sender key state changes (rotation, error recovery, retry failures) so stale data is never served + +```rust +// Atomic get-or-init: concurrent callers for the same group +// share the single database read result +let cached_map = self + .sender_key_device_cache + .get_or_init(group_jid, async { + let db_rows = pm.get_sender_key_devices(group_jid).await.unwrap_or_default(); + Arc::new(SenderKeyDeviceMap::from_db_rows(&db_rows)) + }) + .await; +``` + +**`SenderKeyDeviceMap` structure:** + +The `SenderKeyDeviceMap` pre-parses JID strings from the database into a user-to-devices HashMap for efficient lookup: + +```rust +pub(crate) struct SenderKeyDeviceMap { + /// user → (device_id → has_key) + devices: HashMap, HashMap>, + /// Users with at least one has_key=false device + forgotten_users: HashSet>, +} +``` + +**Cache invalidation points:** + +| Event | Action | +|-------|--------| +| Sender key rotation (`rotateKey`) | Invalidate group entry | +| `NoSenderKeyState` error during send | Invalidate group entry | +| Retry failure for a group message | Invalidate group entry | +| Server rejects group stanza | Invalidate group entry | +| New device added (`patch_device_add`) | Invalidate all entries | +| Device removed (`patch_device_remove`) | Invalidate all entries | +| Identity change (`clear_device_record`) | No global tracker wipe — per-device SKDM redistribution is driven by retry receipts (`markForgetSenderKey`), matching WhatsApp Web's `WAWebUpdateLocalSignalSession`. The `status@broadcast` sender key is still deleted for forward secrecy. | + +You can tune the cache capacity and TTI via the `sender_key_devices_cache` field in [`CacheConfig`](/api/bot#cache-configuration-reference). + +Location: `src/sender_key_device_cache.rs`, `src/send.rs` + +### Phash validation for stale device list detection + +When sending group, status, or DM messages, the library validates the participant hash (`phash`) returned in the server's acknowledgment against the locally computed `phash`. A mismatch indicates that the server's view of participant devices differs from the client's — meaning the local device list is stale. + +**How it works:** + +1. Before sending, the client obtains the locally computed `phash` — from the stanza `phash` attribute for group/status messages, or from `PreparedDmStanza.phash` for DMs +2. A `PhashWaiter` (expected hash, target JID, whether to also invalidate the group cache) is registered for the message ID via `register_phash_waiter` — a map entry, not a channel or a task +3. The message stanza is sent to the server +4. When the server's ack for that message ID arrives, the read loop compares its `phash` attribute against the expected value inline, with no task involved +5. On a match the entry is just dropped; on a mismatch the client spawns a task to invalidate caches — so a send only pays for a task in the uncommon case, not on every send ([#1116](https://github.com/oxidezap/whatsapp-rust/pull/1116)) + +**On mismatch, the following caches are invalidated:** + +| Send path | Sender key device cache | Group info cache | Device registry | +|-----------|------------------------|-----------------|----------------| +| Group messages | Invalidated | Invalidated | — | +| Status messages | Invalidated | Not invalidated | — | +| DM messages | — | — | Recipient + own PN devices invalidated | + +For DM messages, the phash covers both recipient and own devices (matching WA Web's `syncDeviceListJob([recipient, me])`). On mismatch, the client invalidates the device registry cache for both the recipient's user JID and your own phone number (PN) JID, ensuring the next send re-fetches the current device list for both parties. + +```rust +// src/send.rs — simplified phash validation flow + +// Group/status path: phash from stanza attribute +let our_phash = stanza.attrs().optional_string("phash").map(|s| s.into_owned()); + +// DM path: phash from PreparedDmStanza (not on the wire) +let dm_phash = prepared.phash; + +// On DM phash mismatch: +if !jid.is_group() && !jid.is_status_broadcast() { + client.invalidate_device_cache(&jid.user).await; + if let Some(own_pn) = &client.persistence_manager.get_device_snapshot().pn { + client.invalidate_device_cache(&own_pn.user).await; + } +} +``` + + +The phash check never blocks the send path. If the server's ack never arrives, nothing polls the waiter directly — it is swept out on the keepalive tick, and is guaranteed to survive the sweep immediately following its registration (so a waiter is never dropped mid-flight) but is removed on the sweep after that. Since each keepalive tick lands 15–30 seconds after the last, the actual time-to-live is roughly one to two tick intervals — about 15 to 60 seconds, depending on where registration falls relative to the sweep cycle — rather than the old fixed 10-second timeout. The sweep runs before keepalive's own idle early-return, so a connection with steady inbound traffic — which skips sending pings — still gets its stale waiters cleared; a stranded waiter would otherwise read as an outstanding IQ and suppress pings for the life of the connection. This matches WhatsApp Web's approach of using phash as a best-effort staleness detector rather than a hard requirement. + + +#### WA Web phash parity (v0.6) + +Two corrections aligned the group phash with WA Web's `phashV2`: + +- **Full device set, every send.** The group phash is now computed over the *complete* resolved participant device set plus the sending device on every send — not just the devices that received an SKDM in that stanza. Warm sends (which distribute no new SKDM) now pass the full resolved set via the `all_devices_for_phash` parameter to `prepare_group_stanza`, so the phash matches the server's view even when the SKDM target set is empty. Status broadcasts keep their prior phash behavior. +- **Standard base64 alphabet.** The phash now encodes with the standard base64 alphabet (`+` / `/`) instead of URL-safe (`-` / `_`), matching WA Web and whatsmeow. + +The client also now **persists group metadata** locally after a query and sends the stored participant phash on the next group query, letting the server answer `not-modified` (304) when membership is unchanged — saving a full metadata round-trip. + +Location: `src/send.rs`, `src/client.rs` + +## Cryptographic Primitives + +### AES-256-CBC (message content) + +Used for encrypting message bodies in both 1:1 and group messages: + +```rust +pub fn aes_256_cbc_encrypt_into( + plaintext: &[u8], + key: &[u8], // 32 bytes + iv: &[u8], // 16 bytes + output: &mut Vec, +) -> Result<()> +``` + +Location: `wacore/libsignal/src/crypto/aes_cbc.rs` + +### Thread-Local Buffers + +The implementation uses thread-local buffers to reduce allocations: + +```rust +thread_local! { + static ENCRYPTION_BUFFER: RefCell = ...; + static DECRYPTION_BUFFER: RefCell = ...; +} + +// Usage in session_cipher.rs:99-111 +let ctext = ENCRYPTION_BUFFER.with(|buffer| { + let mut buf_wrapper = buffer.borrow_mut(); + let buf = buf_wrapper.get_buffer(); + aes_256_cbc_encrypt_into(ptext, message_keys.cipher_key(), + message_keys.iv(), buf)?; + let result = std::mem::take(buf); + buf.reserve(EncryptionBuffer::INITIAL_CAPACITY); + Ok::, SignalProtocolError>(result) +})?; +``` + +Location: `wacore/libsignal/src/protocol/session_cipher.rs:14-54` + +### HKDF-SHA256 + +Used for key derivation in session initialization: + +```rust +pub fn derive_keys(secret_input: &[u8]) -> (RootKey, ChainKey, InitialPQRKey) { + let mut secrets = [0; 96]; + hkdf::Hkdf::::new(None, secret_input) + .expand(b"WhisperText", &mut secrets) + .expect("valid length"); + // Split into RootKey[32], ChainKey[32], PQRKey[32] +} +``` + +Location: `wacore/libsignal/src/protocol/ratchet.rs:18-39` + +## PreKey Management + +Pre-keys enable asynchronous session establishment in the Signal Protocol. whatsapp-rust manages pre-key generation and upload to match WhatsApp Web's behavior. + +### Configuration + +The per-batch upload count is configurable through the builder/factory API (default `812`, matching WhatsApp Web's `UPLOAD_KEYS_COUNT`). The upload-trigger threshold is a private constant. + +| Setting | Default | Description | +|---------|---------|-------------| +| [`BotBuilder::with_wanted_pre_key_count`](/api/bot#with-wanted-pre-key-count) / [`Client::set_wanted_pre_key_count`](/api/client#set-wanted-pre-key-count) | 812 | Number of pre-keys generated and uploaded per batch. Clamped to `5..=65_535` at upload time. | +| `MIN_PRE_KEY_COUNT` (private const) | 5 | Minimum server-side pre-key count before triggering an upload. | + +```rust +// Via the Bot builder +Bot::builder() + .with_wanted_pre_key_count(256) // smaller batches for embedded consumers + // ... + +// Or directly on a Client constructed by hand (before connect) +client.set_wanted_pre_key_count(256); +``` + +Values outside `[5, 65_535]` are clamped at upload time (an out-of-range value logs a `warn!`). The floor avoids an empty-but-flagged pool or a re-upload loop. The ceiling is the wire-format limit: the upload IQ encodes the pre-key list length as a `u16`, so a larger batch would generate keys locally and then fail to encode. + +Per-key X25519 generation and protobuf encoding for the batch are offloaded via `wacore::runtime::blocking` (runtime-agnostic; runs inline on wasm) since the caller-controlled batch size can be large. + +### Pre-key ID counter and wrap-around + +Pre-key IDs use a persistent monotonic counter (`Device::next_pre_key_id`) that only increases, matching WhatsApp Web's `NEXT_PK_ID` pattern: + +```rust +// Determine starting ID using both the persistent counter AND the store max +let max_id = backend.get_max_prekey_id().await?; +let start_id = if device_snapshot.next_pre_key_id > 0 { + std::cmp::max(device_snapshot.next_pre_key_id, max_id + 1) +} else { + // Migration: start from MAX(key_id) + 1 + max_id + 1 +}; +``` + +This approach prevents ID collisions when pre-keys are consumed non-sequentially from the store. + +**24-bit wrap-around:** + +WhatsApp Web uses 24-bit pre-key IDs on the wire (3-byte big-endian), so valid IDs range from 1 to 16,777,215 (2^24 − 1). When the persistent counter grows past this boundary, modular arithmetic wraps IDs back into the valid range: + +```rust +const MAX_PREKEY_ID: u32 = 16777215; // 2^24 - 1 + +// Wrap start ID into valid [1, MAX_PREKEY_ID] range +let start_id = ((raw_start as u64 - 1) % MAX_PREKEY_ID as u64) as u32 + 1; + +// Each key ID in the batch is also wrapped +let pre_key_id = (((start_id as u64 - 1) + i as u64) % (MAX_PREKEY_ID as u64)) as u32 + 1; + +// After upload, the persisted next_pre_key_id wraps too +let next_id = (((start_id as u64 - 1) + key_pairs_to_upload.len() as u64) + % (MAX_PREKEY_ID as u64)) as u32 + 1; +``` + + + If the counter wraps while unconsumed high-ID pre-keys still exist in the store, the database upsert (`ON CONFLICT DO UPDATE`) silently overwrites them. This is an accepted trade-off because the server consumes keys well before a full 16M cycle completes. + + +Location: `src/prekeys.rs` + +### Retry-receipt prekey marking + +When building a retry receipt that includes keys (`should_include_keys`), the one-time prekey handed directly to the peer is now also marked uploaded via `mark_single_prekey_uploaded`, matching WhatsApp Web's `markKeyAsUploaded`. Without this, the same prekey ID could be re-offered to the server pool in the next batch upload — a third party fetching the bundle could then consume the identical one-time ID and fail to decrypt. + +`mark_single_prekey_uploaded` requires a held `prekey_upload_lock` guard (a compile-time proof, not just a runtime convention) so the get-or-gen and the watermark write are atomic against the batch upload path. It only advances `first_unupload_pre_key_id` when the id being marked is still the current window head (idempotent no-op otherwise), and collapses `next_pre_key_id` onto the wrapped low watermark only when marking the terminal id at the 24-bit edge — a non-terminal high-end head keeps its surviving window key. + +The device account is validated **before** the prekey is reserved/marked, so a missing account fails the retry-receipt build without silently abandoning a one-time prekey from the upload window. + +Location: `src/prekeys.rs`, `src/retry.rs` + +### Force-refreshing pre-keys for device migration + +When migrating a device from an external source (e.g., a Baileys session into an `InMemoryBackend`), the server may still hold pre-key IDs whose private key material you cannot reconstruct. Any `pkmsg` referencing those IDs will fail permanently with `InvalidPreKeyId`. + +The public `refresh_pre_keys()` method force-uploads a fresh batch of `Client::wanted_pre_key_count()` pre-keys (default 812; tunable via [`with_wanted_pre_key_count`](/api/bot#with-wanted-pre-key-count) / [`set_wanted_pre_key_count`](/api/client#set-wanted-pre-key-count)), giving the server new IDs the caller has locally. Old unmatched IDs drain naturally as peers consume them. + +```rust +// After restoring a session from another library +client.refresh_pre_keys().await?; +``` + +Internally, this acquires `prekey_upload_lock` to prevent races with the count-based and digest-repair upload paths, then calls `upload_pre_keys_with_retry(force: true)` which uses Fibonacci backoff (1s, 2s, 3s, 5s, 8s, ... capped at 610s). + +Two related public methods build on the same `prekey_upload_lock`-guarded path: + +- `Client::refresh_pre_keys_with_count(count)` — same force-upload as `refresh_pre_keys()`, but with a caller-chosen batch size instead of the configured [`wanted_pre_key_count`](#configuration). +- `Client::ensure_pre_keys()` — a non-forced check-and-top-up: uploads only if the server-side pool is below the low-water mark, rather than unconditionally replacing it. + +Location: `src/prekeys.rs:263-266` + +### Digest key validation + +After connection, the client validates that the server's copy of the key bundle matches local keys. This matches WhatsApp Web's `WAWebDigestKeyJob.digestKey()` flow. + +**Wire format:** + +```xml + + + + + + + + + [4-byte BE registration ID] + [1-byte: 5] + [32-byte identity public key] + + [3-byte BE signed pre-key ID] + [32-byte signed pre-key public] + [64-byte signature] + + + [3-byte BE prekey ID] + ... + + [20-byte SHA-1 hash] + + +``` + +**Validation process:** + +1. Query the server for the key bundle digest via `DigestKeyBundleSpec` +2. If the server returns **404** (no record), trigger a full pre-key re-upload +3. If the server returns **406/503** or other errors, log and skip +4. On success, compare registration IDs +5. Load each pre-key referenced by the server and extract its public key +6. Compute a local SHA-1 digest over: identity public key + signed pre-key public + signed pre-key signature + all pre-key public keys +7. Compare the local hash against the server-provided hash + + + The `` node contains `` children (not `` children). The parser iterates all children of `` without tag filtering, matching WhatsApp Web's `mapChildren` behavior which does not filter by tag name. + + + + Hash mismatches or missing local pre-keys are logged but do **not** trigger a re-upload. Only a 404 response (server has no record) triggers re-upload. This matches WhatsApp Web's behavior where `validateLocalKeyBundle` exceptions are caught without re-uploading — the normal [`RotateKeyJob`](#signed-pre-key-rotation-rotatekeyjob) eventually refreshes the signed pre-key. + + + + `Client::validate_digest_key()` is a public method — callers can trigger this validation pass on demand instead of only relying on the automatic post-connection check. + + +Location: `src/prekeys.rs:218-344`, `wacore/src/iq/prekeys.rs:170-302` + +### Signed pre-key rotation (RotateKeyJob) + +The signed pre-key minted at pairing was otherwise **permanent** — a forward-secrecy gap. whatsapp-rust mirrors WhatsApp Web's `RotateKeyJob`: on a cadence, generate a fresh signed pre-key, upload it, and retain the previous keys so prekey messages already in flight against a rotated-out key still decrypt. + +**Cadence:** + +- Checked once per connection. Spawned right after the startup pre-key upload during post-login init, so a slow or failing rotation IQ never delays the rest of login. +- Rotates once `now - last_signed_pre_key_rotation_ms >= SIGNED_PRE_KEY_ROTATION_INTERVAL_MS` (7 days / weekly). This interval is the one value **not** grounded in the captured WA Web bundle — there `RotateKeyJob` is a persisted, server-tuned background job — so it's a tunable policy default. +- A device upgraded in with the field at `0` gets a one-time baseline stamp (`DeviceCommand::SetSignedPreKeyRotationBaseline`) instead of rotating immediately, so its first rotation lands a full interval out. +- Single-flighted via `Client::signed_pre_key_rotation_lock` so overlapping post-login tasks (from reconnect churn) can't run the rotate/upload/prune sequence concurrently; a losing task just no-ops for that check. + +**Wire format** (`RotateSignedPreKeySpec`, reuses the upload path's `` encoder so the two can never drift): + +```xml + + + + [3-byte BE signed pre-key ID] + [32-byte signed pre-key public] + [64-byte signature] + + + +``` + +**Rotation sequence** (`Client::rotate_signed_pre_key`): + +1. Compute `new_id = current_id + 1`, wrapping to `1` at the 24-bit border (same scheme as one-time pre-key IDs). +2. Stage the new key pair in the `signed_prekeys` backend table **before** upload — an already-staged candidate for that id is reused verbatim, so a retry after an ambiguous failure re-uploads the exact key the server may have already accepted instead of minting a different one under the same id. +3. Retain the outgoing (current) key in the backend table **before** upload, so once the server accepts the new key the old id's decrypt window is already durable — no post-acceptance write can strand it. +4. Upload via the `` IQ above. +5. On success, `DeviceCommand::SetSignedPreKey` atomically installs the new key pair, id, signature, and rotation timestamp. The now-redundant staged copy is dropped, and retained signed pre-keys are pruned (newest-id-first) to `SIGNED_PRE_KEY_RETENTION` (3 total: the current key + the 2 most recent rotated-out keys). + +**Error handling** (mirrors WA Web's `RotateKeyJob` ladder; a rotation failure never fails login): + +| Response | Action | +|----------|--------| +| `406` / `409` (deterministic rejection of this key) | Drop the staged candidate and remint a fresh one on a later connect — reusing it would wedge rotation forever. | +| Other server error (rate limits, `>=500`, ...) | Keep the staged candidate and retry it as-is on a later connect. | +| Transport failure (ambiguous — the server may have accepted it) | Keep the staged candidate and retry it as-is on a later connect. | + +**Backend fallback for rotated-out ids:** + +Before this feature, `Device::load_signed_prekey` (`src/store/signal.rs`) returned a record only when the requested id matched the *current* `signed_pre_key_id` field — the `signed_prekeys` backend table (which already existed, with full CRUD) was never consulted for other ids. Rotating the key in place would therefore make any in-flight prekey message naming the old id fail with `InvalidSignedPreKeyId`. `load_signed_prekey` and `contains_signed_prekey` now fall back to the backend table for non-current ids, which is what makes rotation safe to ship. + +**Retry, not NACK, once the id ages past retention:** A sender's `PreKeySignalMessage` can still name a signed pre-key id that has since aged past `SIGNED_PRE_KEY_RETENTION` (3 total: current + 2 rotated-out) — the backend fallback above has nothing left to return, and `InvalidSignedPreKeyId` is the correct, permanent answer. On the 1:1 decrypt path (`src/message/receive.rs`), this now routes to a retry receipt (`RetryReason::InvalidKeyId`) carrying the current bundle, mirroring the sibling `InvalidPreKeyId` arm — instead of falling through to the catch-all `UnhandledError` nack, which would drop the stanza from the offline queue and lose the 1:1 message permanently and silently. + + + `Client::rotate_signed_pre_key()` is a public method — callers can force an out-of-cadence rotation directly instead of waiting for the weekly check. It shares `signed_pre_key_rotation_lock` with the automatic path (so a manual call can't race a background rotation) and propagates upload failures to the caller rather than swallowing them. + + +Location: `src/features/rotate_key.rs`, `src/store/signal.rs`, `src/message/receive.rs`, `wacore/src/iq/prekeys.rs`, `wacore/src/store/commands.rs` + +### Re-pair pre-key healing (v0.6) + +If the user re-pairs the device (for example by re-scanning the QR code), the server discards its copy of our pre-key bundle even though `Device::server_has_prekeys` may still read `true` from the previous pairing. v0.6 resets `server_has_prekeys = false` immediately after a successful re-pair so the next connect uploads a fresh batch instead of trusting the stale flag. + +The lock-acquisition for the digest-key validator also moved into `validate_digest_key` itself. Previously the caller held `prekey_upload_lock` before calling the validator, which would deadlock when validation hit a 404 and tried to acquire the same lock to perform the re-upload. The lock now wraps only the re-upload path, so the 404→re-upload transition completes without contention. + +Location: `src/handlers/notification.rs`, `src/pair.rs`, `src/prekeys.rs` + +### ADV companion identity validation + +When fetching a pre-key bundle for a contact's companion device (WhatsApp Web / Desktop), the bundle's `` element is validated to confirm that the fetched identity key is cryptographically bound to the account. This guards against a relay substituting a forged identity key, matching WA Web's `SessionApi.createSignalSession`. + +**Account key resolution** mirrors WA Web's `validateADVwithIdentityKey` (`e.accountSignatureKey || t`): + +1. **In-blob key**: If `ADVSignedDeviceIdentity.account_signature_key` is present and non-empty, it is used directly. +2. **Stored identity fallback**: The server legitimately omits this field for a contact's companion because the client already holds the contact's primary (device 0) identity in the Signal identity store. When the field is absent, `Client::load_account_identity` loads it — reading through the `SignalStoreCache` so any unflushed mutations from the current session are visible. `PreKeyFetchSpec::with_account_identities` threads the pre-loaded map into `wacore`'s stateless prekey parser, keeping store access in the `whatsapp-rust` crate. + +**Validation results** (`wacore::adv::AdvValidation`): + +| Variant | Condition | Action | +|---------|-----------|--------| +| `Valid` | Both account and device signatures verified | Session is established normally | +| `Invalid` | Blob is malformed, or signatures fail against the available key | Bundle is rejected — a relay swapping in a forged identity lands here | +| `NoAccountKey` | Neither the blob nor the store has the key | Bundle is kept, ADV check skipped (logged as `warn!`) | + +`NoAccountKey` does not weaken security beyond the pre-existing "device-identity absent" path: a relay could already strip the entire `` element to bypass the check. It exists so brand-new contacts whose primary identity has never been seen are not silently dropped. + +The same three-state validation applies in the retry-receipt handler (`src/retry.rs`) when a companion device requests a re-send. + +Location: `wacore/src/adv.rs`, `wacore/src/iq/prekeys.rs`, `src/prekeys.rs` + +## Storage Integration + +whatsapp-rust integrates Signal Protocol storage through a layered architecture: + +``` +src/store/ +├── signal.rs # SignalStore trait impl for Device (identity, session, prekey, sender key) +├── signal_adapter.rs # SignalProtocolStoreAdapter — cache-backed adapter bridging wacore traits to libsignal traits +└── signal_cache.rs # Re-export of wacore::store::signal_cache::SignalStoreCache +``` + +The `Device` struct implements the libsignal `SessionStore`, `IdentityKeyStore`, and other traits. These are wrapped by `SignalProtocolStoreAdapter`, which adds the `SignalStoreCache` layer — sessions are cached as `SessionRecord` objects (not bytes), with serialization deferred to `flush()`. + +Each store (sessions, identities, sender keys) is flushed **independently** under its own lock. Only one store is locked during its I/O — the other two remain free for concurrent encrypt/decrypt operations. The lock is held from snapshot through write through clear, so mutations to the same store are blocked until flush completes, preventing dirty-set races: + +```rust +// SignalProtocolStoreAdapter reads/writes through the cache +#[async_trait] +impl SessionStore for SessionAdapter { + async fn load_session( + &self, + address: &ProtocolAddress, + ) -> Result, SignalProtocolError> { + // Returns cached SessionRecord object directly (no deserialization) + // Cold load deserializes from backend bytes once and caches the object + self.cache.get_session(&addr_str, &*device.backend).await + } + + async fn store_session( + &mut self, + address: &ProtocolAddress, + record: SessionRecord, // Takes ownership — zero-cost move + ) -> Result<(), SignalProtocolError> { + // Stores the SessionRecord object in cache, marks dirty + // Serialization happens only during flush() + self.cache.put_session(&addr_str, record).await; + Ok(()) + } +} +``` + +### Flush scheduling: send vs. receive + +*When* the dirty Signal cache reaches the backend differs by direction, because the two directions have different recovery properties: + +- **Send (DM/1:1 sessions)** persists through a batched **counter lease**. `SessionRecord` reserves its outbound sender-chain counter `SENDER_CHAIN_RESERVATION_BATCH` (64) values at a time, via `SessionRecord::reserve_sender_chain_counters`. A send covered by an unexhausted lease is already durable — it only schedules the same coalesced write-behind as the receive path below. The send that exhausts the lease, roughly 1 in 64, raises the ceiling and flushes **synchronously, before the stanza reaches the wire**. If that flush fails, the send aborts instead of transmitting an advance it couldn't save. Reusing an outbound counter reuses its message key and IV, so no counter can ever be used before its lease is durable. The lease field is local-only: it's field 100 in the encoded `SessionRecord`, outside the vendored `whatsapp.proto`. By default (`SessionRecord::deserialize`), every load fast-forwards the sender chain to the lease ceiling, so a crash mid-lease can never re-derive a possibly-spent counter — the store-backed load path relaxes this only for a *trusted* reload, via the incarnation marker described below. +- **Send (group and status sends)** follows the same lease pattern as DMs. `SenderKeyRecord` reserves its outbound chain iteration `SENDER_CHAIN_RESERVATION_BATCH` (64) at a time via `SenderKeyRecord::reserve_iterations`, using the same field-100 local-only encoding and the same fast-forward-on-load recovery as `SessionRecord`. A send within an unexhausted lease rides the coalesced write-behind; only the send that raises the ceiling, roughly 1 in 64, flushes synchronously before the stanza reaches the wire. Status posts go through the same group-encrypt path and inherit this behavior; status *reactions* are a DM-branch send and always followed the DM lease instead. Production encryption (`wacore::send::encrypt_group_message`) delegates to the same `group_encrypt` primitive these guarantees live in, so there is exactly one sender-key encrypt/advance/store implementation — earlier, a second unguarded copy on the production path meant most warm group sends skipped the pre-wire flush entirely. +- **Receive** (live traffic, outside the offline-drain batcher) routes through a single-flight coalescing scheduler (`src/signal_flush.rs`) instead of flushing per stanza: a burst of receives folds into one flush per ~25ms window, retried with exponential backoff (up to a 5s cap) on backend failure. This is safe because a lost receive-side advance simply re-derives forward on the next message (the receiving chain derives `CK_n → CK_n+1`), and a consumed one-time prekey stays buffered until its session is durable — a crash inside the window is recoverable. + +**Every direct Signal-mutating call site now shares the same gate ([#1048](https://github.com/oxidezap/whatsapp-rust/pull/1048)).** The public [`Signal::encrypt_group_message` and `Signal::create_participant_nodes`](/api/signal) accessors, plus VoIP's outbound call-key fanout (`place_call` in `src/voip/facade.rs`), used to call `Client::flush_signal_cache_batch_safe()` — an unconditional flush — after releasing their session or sender-key chain locks, regardless of whether the mutation actually raised a lease's durable ceiling. They now call the same `Client::persist_signal_state_pre_wire()` gate the primary `wacore::send` path uses: a warm call already covered by an existing lease rides the coalesced write-behind and returns immediately, while only the roughly-1-in-64 call that raises the ceiling still flushes synchronously before its ciphertext is used or sent. Measured against the same real-SQLite harness: warm group encryption dropped from 84.69 ms to 23.98 ms per 1,024-call sample (-71.7%), and warm 4-recipient participant fanout dropped from 49.16 ms to 8.69 ms per 256-call sample (-82.3%). Safety is unchanged — the per-entry durability check that guards what gets written is identical in both cases; the difference is that `flush_signal_cache_batch_safe()` always entered the flush path even on a warm lease, while `persist_signal_state_pre_wire()` skips it entirely once the raised lease is already durable. This removes redundant synchronous I/O, not any safety guarantee. + + +Downgrading to a version that predates the counter lease after running a leased version: the older version ignores the lease field(s) and could reuse counters/iterations that were only reserved (not yet actually sent) by the lease. This applies to `SenderKeyRecord` (group/status sends) as well as `SessionRecord` (DM sends). Avoid downgrading a device's local state across this boundary. + + + +The pre-wire gate is a point-in-time check, not a lock held across the flush, and this specifically affects **DM sessions**: `needs_pre_wire_flush()` inspects pending reservations once, and `flush()` skips any session entry that is currently checked out by a concurrent operation (`SessionEntry::CheckedOut`), still returning `Ok` for the entries it did persist. If another task checks out the same session between this send's lock release and its flush, that session's reservation can remain pending even though the flush "succeeded" — the caller proceeds to write its stanza regardless. This is a property of `SignalStoreCache::flush` itself, not specific to retries; it affects any pre-wire-gated DM send that races a concurrent, *still in-progress* operation on the same session. Sender-key entries have no analogous checked-out state — `get_sender_key` clones an `Arc` without removing the cached record, so every dirty sender-key entry is included in a flush's batch — so group/status sends are not exposed to this race. This is unrelated to what happens if that concurrent operation is then *cancelled*: see [Cancellation-safe session checkouts](#cancellation-safe-session-checkouts) — a dropped checkout restores its entry synchronously rather than leaving it stranded, but the deferred entry still isn't included in a flush that already ran before the restore. + + +Deleting a session or sender-key record — an identity change, a session reset, a rotated sender key — creates a tombstone rather than clearing the record's pending gate immediately. If a durability gate was open on the record at delete time, `SignalStoreCache` keeps it open until the backend's `delete_session`/`delete_sender_key` call actually succeeds; a failed delete leaves `needs_pre_wire_flush()` returning `true` and is retried on the next flush. Earlier, the gate was released as soon as the tombstone was applied to the in-memory cache, so a failed delete or a crash in that window could let ciphertext reach the wire while the pre-delete chain state was still loadable from the backend, re-deriving already-used key material on reload. A lossy `clear()` still drops a pending tombstone gate — the tombstone is discarded from the in-memory cache without issuing the backend delete, so the old chain state may remain in the backend. + +### Clean reload vs. crash recovery + +Fast-forwarding past a lease's reserved ceiling on every reload is the safe default, but it's also overly conservative for the common case: a clean reconnect or a same-process store re-creation never actually risked losing an in-flight send, yet unconditionally fast-forwarding still burned a full unused batch every time. 32 clean reconnects could push a sender-key chain 2,048 iterations ahead and get rejected once a peer who missed the intervening messages hit `MAX_FORWARD_JUMPS` (2,000). + +`SignalStoreCache`, and the direct, non-cached `Device` store, now tag a durably-reserved record with a random 128-bit **store incarnation**, carried in a second local-only field (101, alongside the lease's field 100) on both `SessionRecord` and `SenderKeyRecord`: + +- A live `SignalStoreCache` generates one incarnation marker when it's constructed. A reload that observes a matching marker proves the record came from *this same live cache* and was never lost to a crash, so it's exact: the chain resumes at the next iteration instead of fast-forwarding. +- A reload with no marker, a different marker, or a malformed/duplicated marker is untrusted and keeps the conservative fast-forward — this covers process restarts, a freshly constructed cache, and any genuinely lossy discard. +- [`SignalStoreCache::clear_after_flush()`](/concepts/architecture#disconnect-cleanup), used by connection teardown, only evicts a store once it's fully settled — no dirty, deleted, checked-out, or pending-wire-gate entries. Anything a concurrent write installs after the flush stays resident (and the incarnation stays put) until a later flush settles it; only an actual lossy discard rotates the marker. +- Direct `Device` stores hold one process-level incarnation in an `OnceLock` instead of a per-cache one. Since these stores synchronously await the backend write before returning ciphertext, a new `Device` wrapping the same backend in the same process is a clean, trusted reload; a process restart gets a fresh marker and stays conservative. + +This adds no field to the public `Device` struct and no synchronous I/O — a marker is generated only when a cache or process starts, or when a lossy boundary invalidates trust. + +The scheduler is generation-scoped (embeds the connection generation in its atomic state), so a reconnect during an in-flight flush needs no explicit reset: a stale worker from the previous connection cannot mutate the new generation's state, and stands down when it observes a foreign generation. + +The offline drain, identity-change recovery, and teardown all keep their own **synchronous** flushes — they gate acks, receipts, or follow-up reads on durability and are not routed through the receive coalescer. (Teardown's flush is itself conditional: `teardown_inbound_commits_bounded` only flushes on a durable drain with no outstanding batch entries; on a timeout or non-durable/pending entries, it clears the cache without flushing instead, relying on server redelivery rather than risking a persisted-but-incomplete advance.) See [Inbound Durability Hook](/advanced/inbound-durability) for the drain-batch commit ordering, which this coalescing does not change. + +Retry-receipt recovery (`handle_retry_receipt` resending to a DM or group requester, `src/retry.rs`) instead shares the DM/group send durability rule above through one helper, `send_retry_stanza`: the session lock is released first, then `persist_signal_state_pre_wire()` runs — flushing synchronously if the retry's Signal advance crossed an unpersisted lease boundary — before the stanza is written to the wire. An `Err` from that flush aborts the retry instead of transmitting an advance it couldn't save; for a DM retry specifically, see the point-in-time caveat above for the narrower case where the flush reports success without having actually persisted this retry's own checked-out session (group retries are not exposed to that race — see the same caveat). This replaced an earlier unconditional full flush that ran only *after* the retry stanza had already reached the wire — a crash or persistence failure in that window could reload the old chain state after the ciphertext was already sent. + +Call [`Client::flush_pending_signal_state()`](/api/client#flush_pending_signal_state) to force a deterministic settle — e.g. before reading persisted Signal state directly, or ahead of a non-graceful shutdown. Never call it from inside an `InboundDurabilityHook` or a synchronous, inline `EventHandler::handle_event` implementation, since settling re-enters the processing permit those run under and would deadlock during an offline-sync drain. Ordinary `Bot` closure handlers are unaffected — both default delivery modes run the callback in a detached task off the permit. + +### DH ratchet resets rebase the lease + +A DH ratchet doesn't extend the current sender chain — it replaces it in place. The ratchet installs fresh key material from a new random ephemeral at counter zero and drops the retired chain instead of archiving it. The counter lease described above is a **record-level** ceiling, but the chain it bounds is **per-ratchet-epoch**. Without a matching lease rebase, the ceiling keeps describing a chain that no longer exists. + +For ping-pong traffic, that gap is one batch and you'd never notice it. It's different for a peer you only ever monologue at — say, your own other device, which gets a copy of every message you send but rarely replies. The chain climbs past `MAX_RESERVATION_FAST_FORWARD` before one reply triggers the ratchet, and the ceiling ends up stranded thousands of counters above a chain that just restarted at zero. + +A live reload never surfaces this, since a trusted-incarnation reload (above) skips the fast-forward entirely. The gap only shows up on **recovery** — a restart, or any lossy cache reset. There, the reload has to fast-forward across a span no send ever created. It refuses past `MAX_RESERVATION_FAST_FORWARD` and fails the whole record load. From that point the address is stranded: every path that could repair the session — inbound decrypt, the group-send fan-out, the retry-receipt handler — has to load the unloadable record first. + +`SessionRecord::rebase_lease_after_sender_chain_reset()` closes this gap. As part of the same mutation that swaps in the fresh chain, it lowers the ceiling to at most one `SENDER_CHAIN_RESERVATION_BATCH`. It only ever lowers, never raises, so you can never publish a counter under a ceiling that isn't yet durable. The lowering happens atomically with the chain swap, so no snapshot can pair the retired chain with a ceiling rebased for it, or vice versa. Rebasing to one batch instead of zero keeps the fresh chain's first counters lease-covered, so steady-state ping-pong keeps its write-behind send path instead of paying a synchronous flush on the very next send. A chain that's *archived* rather than discarded keeps its claim on the lease instead: `promote_fresh_state` burns the outgoing state to the ceiling before resetting it. + +If a record already has a stranded ceiling — written by a build that predates this fix — it recovers on its own the next time you use that address; you don't need to delete the row by hand. See [undecodable session rows](/concepts/storage#signalstorecache). + +## Record components + +`wacore-libsignal` exposes owned, validated projections of `SessionRecord` and `SenderKeyRecord` called **components**. Use them when you need to interchange or inspect session and sender-key record state without depending on the generated protobuf schema directly — for example in custom store implementations, migration tooling, or offline debugging. This API is purely additive: the protobuf-backed `serialize()`/`deserialize()` path is unchanged. A record does not round-trip through `into_components()` → `from_components()` → `serialize()` byte-for-byte — the conversion applies the validated, normalized export rules described below (counter-lease advancement, stale-chain removal, and bounded truncation), so treat it as a safe normalized re-encoding rather than a lossless copy. + +```rust +// wacore/libsignal/src/protocol/record_components.rs, re-exported from +// wacore/libsignal/src/protocol/mod.rs +pub use record_components::{ + PendingKeyExchangeComponents, PendingPreKeyComponents, SenderChainKeyComponents, + SenderKeyRecordComponents, SenderKeyStateComponents, SenderMessageKeyComponents, + SenderSigningKeyComponents, SessionChainComponents, SessionChainKeyComponents, + SessionComponents, SessionMessageKeyComponents, SessionMessageKeyMaterial, + SessionRecordComponents, +}; +``` + +### Session and sender-key shapes + +`SessionRecordComponents` mirrors the `current_session` / archived `previous_sessions` split already described in [Arc previous sessions](#arc-previous-sessions); `SenderKeyRecordComponents` mirrors a `SenderKeyRecord`'s state list: + +```rust +pub struct SessionRecordComponents { + pub current_session: Option, + pub previous_sessions: Vec, +} + +pub struct SessionComponents { + pub session_version: Option, + pub local_identity_public: Option>, + pub remote_identity_public: Option>, + pub root_key: Option>, + pub previous_counter: Option, + pub sender_chain: Option, + pub receiver_chains: Vec, + pub pending_key_exchange: Option, + pub pending_pre_key: Option, + pub remote_registration_id: Option, + pub local_registration_id: Option, + pub needs_refresh: Option, + pub alice_base_key: Option>, +} + +pub struct SessionChainComponents { + pub sender_ratchet_key: Option>, + pub sender_ratchet_key_private: Option>, + pub chain_key: Option, + pub message_keys: Vec, +} + +pub struct SenderKeyRecordComponents { + pub states: Vec, +} + +pub struct SenderKeyStateComponents { + pub key_id: u32, + pub chain_key: SenderChainKeyComponents, // { iteration: u32, seed: Vec } + pub signing_key: SenderSigningKeyComponents, // { public: Vec, private: Option> } + pub message_keys: Vec, // { iteration: u32, seed: Vec } +} +``` + +`SessionMessageKeyComponents` and `SenderMessageKeyComponents` hold skipped out-of-order message keys, keyed by chain index/iteration. A session message key's secret material is `SessionMessageKeyMaterial`: + +```rust +#[derive(Clone, Copy, PartialEq, Eq)] +pub enum SessionMessageKeyMaterial { + Seed([u8; 32]), + Derived { + cipher_key: [u8; 32], + mac_key: [u8; 32], + iv: [u8; 16], + }, +} +``` + +As of [#1210](https://github.com/oxidezap/whatsapp-rust/pull/1210), `Seed` is no longer just a compact *import* form expanded through the same canonical derivation used elsewhere (see [`MessageKeyGenerator`](#chain-key-ratcheting)) — the seed is now persisted, so a skipped key round-trips back out as `Seed` too. `session_structure::chain::MessageKey` carries an additive `seed` field (local field 100) alongside the pre-existing `cipher_key`/`mac_key`/`iv`; `from_structure` prefers it when present, re-deriving from it and rejecting the record if the result disagrees with the stored triple (the seed supersedes the triple on export, so a mismatch means a corrupt record). `Derived` is now what comes back only for a key persisted *before* this change, which never wrote a seed. Both variants are fixed-width and `Copy`, so `into_structure()` — the reverse direction — is infallible where it used to validate a `Vec` length. + + +**Breaking for direct consumers of these types.** `SessionMessageKeyMaterial::Seed`/`Derived` switched from `Vec` fields to fixed-width arrays — match both variants rather than assuming `Derived` is the only exported form. `MessageKeyGenerator::Keys(MessageKeys)` was removed in the same change (nothing in the workspace constructed it); build a seeded key with `MessageKeyGenerator::new_from_seed` instead. Downgrading to a build predating #1210 and writing a record back out drops the seed silently — decrypt is unaffected, but the key becomes unexportable again. + + +Conversions: + +```rust +impl SessionRecord { + pub fn from_components(value: SessionRecordComponents) -> Result; + pub fn into_components(mut self) -> Result; +} +impl SenderKeyRecord { + pub fn from_components(value: SenderKeyRecordComponents) -> Result; + pub fn into_components(mut self) -> Result; +} +``` + +### Import validation + +`from_components` enforces the same structural invariants the canonical protobuf reader relies on elsewhere in this codebase, rather than accepting whatever shape the caller hands it: + +- A **sender chain** (the local sending ratchet chain within a pairwise session — not to be confused with a group sender-key chain) must be structurally complete: ratchet public key present, a 32-byte ratchet private key, and a chain key with both its index and 32-byte secret set. An incomplete sender chain fails with `SignalProtocolError::InvalidArgument`; a session with *no* sender chain at all (e.g. one just received and not yet replied to) is fine and imports as `sender_chain: None`. +- A **receiver chain** must never carry `sender_ratchet_key_private` — receiver chains never own the remote party's private key, so import fails if one is set. Symmetrically, projecting a persisted record into components always reports a receiver chain's `sender_ratchet_key_private` as `None`, silently dropping any non-canonical private material a legacy record might contain, matching how the canonical reader already treats that field. +- Raw 32-byte public keys and canonically-serialized public keys (type byte + 32 bytes) are both accepted on import; whichever form was imported, `into_components()` always exports canonical serialization. + +### Export normalization + +`into_components()` never hands back a session or sender-key chain whose counter could be replayed on re-import: + +- Any durably reserved sender-chain counter range — see the counter-lease mechanics in [Flush scheduling: send vs. receive](#flush-scheduling-send-vs-receive) — is advanced to its exclusive ceiling before export. Re-importing the exported components can't reuse a counter value that was only reserved, not yet actually sent. +- A sender chain too stale to fast-forward past its reservation is dropped from the exported chain rather than aborting the whole export — the rest of the record (receiver chains, other archived sessions) still exports normally. +- `SessionRecordComponents.previous_sessions` is truncated to `ARCHIVED_STATES_MAX_LENGTH` (40) and `SenderKeyRecordComponents.states` is truncated to `MAX_SENDER_KEY_STATES` (5) — the same bounds the records themselves already enforce. See [Protocol safety limits](#protocol-safety-limits). + +### `has_usable_sender_chain` + +```rust +impl SessionState { + pub fn has_usable_sender_chain(&self) -> Result; +} +impl SessionRecord { + pub fn has_usable_sender_chain(&self) -> Result; +} +``` + +Checks whether a session has a sender chain it could actually encrypt with, rather than assuming one exists. Previously this was effectively best-effort/always-true; the current implementation returns `Ok(false)` (not an error) when no sender chain is set at all, and otherwise structurally validates the ratchet public key, the ratchet private key, and the chain key are all present before returning `Ok(true)` — the same completeness check `from_components` applies on import. `SessionRecord::has_usable_sender_chain` delegates to the current session's check, returning `Ok(false)` when there is no current session. + +Location: `wacore/libsignal/src/protocol/state/session.rs` + +### Debug output redacts secrets + +`Debug` on every `*Components` type — `SessionComponents`, `SessionChainComponents`, `SessionChainKeyComponents`, `PendingKeyExchangeComponents`, `PendingPreKeyComponents`, `SenderChainKeyComponents`, `SenderSigningKeyComponents`, `SenderMessageKeyComponents`, and `SessionMessageKeyMaterial` — prints private keys, chain/root keys, seeds, and cipher/mac/IV material as ``, while structural fields (indices, counters, iteration numbers, key presence) print plainly: + +```rust +println!("{chain:?}"); +// SessionChainKeyComponents { index: Some(7), key: } +``` + +This makes it safe to log or assert against a `*Components` value in application code without writing a custom `Debug` impl to avoid leaking key material. + +**Example — inspecting whether a session can currently send, without touching protobuf types:** + +```rust +// `record: SessionRecord` loaded from your store +if record.has_usable_sender_chain()? { + let components = record.into_components()?; + println!("{:?}", components.current_session); // secrets redacted +} +``` + +Location: `wacore/libsignal/src/protocol/record_components.rs` + +### Legacy session v1 interop + +Behind the opt-in `legacy-session-interop` Cargo feature (default off, forwarded through `wacore` and the root `whatsapp-rust` crate), `wacore-libsignal` exposes a typed, transport-agnostic model of the **decoded** legacy libsignal `SessionRecord` v1 layout — the format this project's stores used before the canonical shapes above. It exists for migration tooling importing an externally produced v1 store into the canonical `SessionRecord`, or projecting a canonical record back into v1 terms; ordinary clients never enable it, so the model compiles out of native builds entirely. + +Container decoding — turning a legacy store's transport bytes into these typed fields — is explicitly out of scope; callers own that step and hand this module owned values (`Bytes`, integers, enums). The module owns everything downstream: chain-role selection, counter translation, lifecycle ordering, pruning, ratchet reconstruction, and skipped-key derivation. + +```rust +// wacore/libsignal/src/protocol/legacy_session.rs, re-exported from +// wacore/libsignal/src/protocol/mod.rs behind `legacy-session-interop` +pub use legacy_session::{ + LegacyIndexedSessionV1, LegacySessionBaseKeyRoleV1, LegacySessionChainCounterV1, + LegacySessionChainKeyV1, LegacySessionChainRoleV1, LegacySessionChainV1, + LegacySessionDispositionV1, LegacySessionFieldV1, LegacySessionIndexV1, + LegacySessionInteropError, LegacySessionKeyPairV1, LegacySessionLocalContext, + LegacySessionMessageKeyV1, LegacySessionPendingPreKeyV1, LegacySessionRatchetV1, + LegacySessionRecordV1, LegacySessionUnrepresentableFieldV1, LegacySessionV1, +}; +``` + +**Import — v1 into canonical:** + +```rust +impl LegacySessionRecordV1 { + pub fn from_indexed_sessions( + sessions: Vec, + ) -> Result; + + pub fn into_session_record( + self, + context: LegacySessionLocalContext, + ) -> Result; +} +``` + +`from_indexed_sessions` validates that each entry's outer map key matches its own session's base key and rejects duplicate base keys or more than one `Current` session. `into_session_record` then validates every session — chain roles, key lengths, counters, skipped-key indexes, pending pre-keys, and the same canonical limits enforced in [Import validation](#import-validation) — retains archived sessions by close time, reorders them by last use to match the v1 decrypt search, truncates to `ARCHIVED_STATES_MAX_LENGTH`, and reuses `SessionRecord::from_components` to build the canonical record. `LegacySessionLocalContext` supplies the local identity key and registration ID, since v1 sessions never persisted them. + +A v1 sending chain seeds `previous_counter` at `-1` for a ratchet step over a chain that has never sent; `LegacySessionChainCounterV1` floors that at zero on import instead of treating it as an error. Every other out-of-range counter fails typed as `InvalidChainCounter`. + +**Export — canonical into v1 (operational, not byte-exact):** + +```rust +impl SessionRecord { + pub fn into_legacy_session_v1_operational( + self, + ) -> Result; +} +``` + +This is a deterministic operational projection, not a round trip: v1 lifecycle timestamps and base-key lookup roles are reconstructed from canonical search/eviction order rather than recovered verbatim, since the canonical record never persisted them in the first place. State the v1 format genuinely cannot represent is rejected with a typed error instead of being silently dropped or inferred — a session with no sender chain, a non-current `session_version`, a pending key exchange, or a `needs_refresh` flag all fail as `NotRepresentable`; a pending pre-key whose base key doesn't match the session's own base key fails as `PendingPreKeyBaseMismatch` rather than producing v1 state the importer would reject on the way back in. + +A receiver chain holding a *derived* (seedless) skipped-message key, with no inverse to a v1 seed, fails as `ChainNotRepresentable::DerivedMessageKey`. Before [#1210](https://github.com/oxidezap/whatsapp-rust/pull/1210) this was every skipped key without exception — the projection was lossy from the first cycle, since import expanded a v1 seed into derived keys and export could never recover it, so a v1 record with a skipped key failed to round-trip even immediately. Now that the seed rides along with the keys it derives (see [`SessionMessageKeyMaterial`](#session-and-sender-key-shapes) above), only a key persisted *before* that change — which never had a seed to retain — still hits this error; a skipped key imported or received after #1210 carries its seed through export and projects normally, permanently, until it is consumed or evicted. + + +CI runs a dedicated `cargo nextest run --features legacy-session-interop` job as of #1210. The feature was previously off in every job, so this entire module's test suite — including the regression test for the bug above — compiled away and never ran. + + +Every type in the module redacts key material from `Debug` — root keys, chain keys, ratchet key pairs, skipped-message seeds, and identity keys all print as ``; only structural fields (roles, counters, indexes, session/chain counts) print plainly, the same convention as the canonical [`*Components` types](#debug-output-redacts-secrets). + +Location: `wacore/libsignal/src/protocol/legacy_session.rs` + +## Security Considerations + +### Identity key trust + +The implementation verifies identity keys before encryption/decryption: + +```rust +if !crate::protocol::storage::is_trusted_identity( + identity_store, + remote_address, + &their_identity_key, + Direction::Sending, +) +.await? +{ + return Err(SignalProtocolError::UntrustedIdentity( + remote_address.clone(), + )); +} +``` + +As of [#1124](https://github.com/oxidezap/whatsapp-rust/pull/1124), this calls the free `is_trusted_identity` resolver (see [Unboxed identity and session hooks](#unboxed-identity-and-session-hooks)) rather than the trait method directly, so a store's `try_is_trusted_identity` hook gets a chance to answer first. + +Location: `wacore/libsignal/src/protocol/session_cipher.rs:309-325` + +### Self-only protocol message gating + +`app_state_sync_key_share`, `app_state_sync_key_request`, and `history_sync_notification` are protocol messages WhatsApp Web treats as "self-only": they only carry meaning when delivered from your own account to another of your linked devices. v0.6 hardens `handle_decrypted_plaintext` so that incoming copies of these messages are dropped unless `MessageInfo.source.is_from_me` is `true`, matching WA Web's `WAWebKeyManagementHandleKeyShareApi` and whatsmeow's gating. + +The consequences if the gate is missing: + +- A spoofed `app_state_sync_key_share` from a peer would let an attacker inject an app-state encryption key, leading to attacker-controlled mutations of your contacts, blocklist, archive state, etc. +- A spoofed `app_state_sync_key_request` from a peer would make the client share its app-state encryption keys with an attacker instead of only with the account's own companion devices. +- A spoofed `history_sync_notification` would point the client at attacker-supplied media for ingestion as your own history. + +If you implement a custom message dispatcher, replicate this `is_from_me` check before honoring any of these protocol messages. Other protocol-message types (`REVOKE`, `EPHEMERAL_SETTING`, `MESSAGE_EDIT`, …) keep their existing semantics. + +Location: `src/message.rs` (`handle_decrypted_plaintext`) + +### Duplicate message detection + +The protocol detects and rejects duplicate messages: + +```rust +if chain_index > counter { + return match state.get_message_keys(their_ephemeral, counter)? { + Some(keys) => Ok(keys), + None => Err(SignalProtocolError::DuplicateMessage(chain_index, counter)), + }; +} +``` + +Location: `wacore/libsignal/src/protocol/session_cipher.rs:822-827` + +As of [#1072](https://github.com/oxidezap/whatsapp-rust/pull/1072), a `DuplicatedMessage` from one candidate session is not terminal — the search keeps trying the remaining current and archived sessions, including a **closed** receiver chain (one with no chain-key seed left, only leftover skipped keys). This covers re-initiations, which reuse the peer's signed pre-key as a ratchet key: a delayed message whose skipped key survives only in an archived session still decrypts, and a closed chain still recognizes a replay of an already-consumed counter instead of falling through to a generic failure. + +Once every session has been tried, a recognized `DuplicatedMessage` outranks `BadMac` in the final classification: a sibling session that happens to share the same ratchet key derives different message keys for the same counter and fails its MAC as expected noise, but the decrypting client already knows this counter was consumed elsewhere. Classifying that as `BadMac` would trigger a retry receipt for a message the peer already delivered; the duplicate verdict wins instead, so the replay is acknowledged and silently dropped. + +### Log level discipline + +The protocol layer follows strict rules about what cryptographic material appears in logs and at which level: + +- **No private keys or secrets are ever logged** — `ChainKey`, `MessageKeys`, and `RootKey` types do not expose their key bytes through logging +- **Public keys appear only at `warn`/`error` levels** — and only when something has gone wrong (untrusted identity, MAC failure) +- **MAC key fingerprints are truncated** — only the first 4 bytes (8 hex chars) are logged during MAC verification failures, not the full key: + ```rust + let mac_key_fingerprint: String = hex::encode(mac_key_bytes).chars().take(8).collect(); + ``` +- **Ratchet keys in debug logs** — successful decryptions log the sender ratchet public key (never private) at `debug` level for diagnostics +- **Pre-key operations** use `debug` for routine operations and `warn`/`info` for exceptional conditions + + + The Signal protocol layer (`wacore/libsignal/src/protocol/`) uses no `trace!`-level logging. Sensitive operations stay at `debug` or above to avoid leaking material in verbose log configurations. + + +### Session state corruption + +Detailed logging helps diagnose crypto failures: + +```rust +fn create_decryption_failure_log( + remote_address: &ProtocolAddress, + errs: &[SignalProtocolError], + record: &SessionRecord, + ciphertext: &SignalMessage, +) -> Result +``` + +This generates comprehensive error logs showing: +- All attempted session states +- Receiver chain information +- Message metadata (sender ratchet key, counter) + +Location: `wacore/libsignal/src/protocol/session_cipher.rs:365-454` + +### Protocol safety limits + +The implementation enforces several hard limits to prevent resource exhaustion and cryptographic failures: + +| Constant | Value | Purpose | +|---|---|---| +| `MAX_PREKEY_ID` | 16,777,215 (2^24 − 1) | Maximum valid pre-key ID (24-bit wire format) | +| `MAX_FORWARD_JUMPS` | 2,000 | Maximum message skip in a ratchet chain — peer sessions and group sender keys | +| `MAX_FORWARD_JUMPS_SELF` | 25,000 | Maximum message skip for a session with your own other devices (wider, still bounded) | +| `MAX_MESSAGE_KEYS` | 2,000 | Maximum cached out-of-order message keys per chain | +| `MAX_RECEIVER_CHAINS` | 5 | Maximum receiver chains per session | +| `ARCHIVED_STATES_MAX_LENGTH` | 40 | Maximum archived session states | +| `MAX_SENDER_KEY_STATES` | 5 | Maximum sender key states per group | +| `MESSAGE_KEY_PRUNE_THRESHOLD` | 50 | Amortized eviction trigger for old message keys | +| Chain key index | u32::MAX | Overflow returns `InvalidState` error (not silent wrap) | + +Location: `wacore/libsignal/src/protocol/consts.rs` + +### Self-DM / sibling decryption recovery + +When a message from your own primary phone or another linked companion fails to decrypt, the v0.6 client distinguishes the failure mode and applies the matching recovery strategy: + +| Internal `RetryReason` | Trigger | Recovery | +|---|---|---| +| `NoSession` | `SessionNotFound` (no Signal session yet for the device) | Request a fresh prekey bundle via retry receipt; install the new session before retrying decryption. | +| `BadMac` | Ratchet desync (`InvalidMessage` / mac failure) on an existing session | Mark the session for re-creation, throttled per peer via the `session_recreate_history` cache so repeated BadMacs don't loop, and re-send via a peer-addressed `pkmsg` carrying our identity. | + +The throttle is a per-peer cooldown (1-hour TTL after the last recreate). In v0.6 the implementation moved from a `Mutex>` to a bounded TTL cache (`PortableCache`, ~256 entries): the per-peer check-and-stamp is now atomic (serialized by the existing per-peer session lock) and lock-free at the map level, so concurrent retry-receipt spawns from the same peer can't trigger duplicate recreates. The behavior is unchanged — if a peer is already in cooldown, the client skips re-creation and falls back to a normal retry receipt rather than thrashing the session. Under more than ~256 distinct peers retrying within the window, the cache may evict a recent entry, costing at most one extra recreate (bounded and self-healing). Peer-addressed `pkmsg` carries the protocol identity so the receiver can verify ownership before installing the new session, blocking spoofed sibling recoveries. + +This closed a deadlock where self-DM fan-out to a sibling device produced repeated BadMac decrypt failures: the recipient would request a retry, the sender would re-encrypt against the same broken session, and the cycle would continue until the user manually relogged. With the throttled re-creation plus identity-validated pkmsg, the second receipt installs a fresh session and decryption resumes. + +Self-DM fan-out also gained WA Web parity for the BadMac case: when our own primary phone reports BadMac, the client now treats it as a session-level recovery rather than dropping the message, matching `WAWebDecryptOrThrow`'s branch on session divergence. + +Location: `src/client.rs`, `src/retry.rs`, `wacore/libsignal/src/protocol/session_cipher.rs`, `wacore/src/send.rs` + +## Performance optimizations + +### Session object cache + +The `SignalStoreCache` stores sessions and sender keys as deserialized objects (`SessionRecord` and `SenderKeyRecord`) rather than serialized bytes, matching WhatsApp Web's architecture where the JS object IS the cache. Serialization only happens during `flush()` to the database — not on every `store_session` or `put_sender_key` call. + +```rust +// wacore/src/store/signal_cache.rs +enum SessionEntry { + /// Arc so peek_session (retry / LID-migration checks) bumps a refcount + /// instead of deep-cloning the record (KBs with archived states). + Present(Arc), + Absent, + /// Taken by a destructive-update load; has_session treats as present. + /// Carries enough identity to reject a stale owner — see + /// "Cancellation-safe session checkouts" below. + CheckedOut { + had_session: bool, + token: NonZeroU64, + }, +} + +struct SessionStoreState { + cache: HashMap, SessionEntry>, // Objects, not bytes + dirty: HashSet>, + deleted: HashSet>, + incarnation: StoreIncarnation, + checkout_generation: u64, + next_checkout_token: u64, +} + +// Sender keys use the same object-caching pattern +struct SenderKeyStoreState { + cache: HashMap, Option>, // Objects, not bytes + dirty: HashSet>, +} +``` + +This eliminates protobuf encode (on store) and decode (on load) from the per-message hot path for both 1:1 and group messages. The `store_session` method takes `SessionRecord` by value, enabling zero-cost moves from the protocol layer: + +```rust +// wacore/libsignal/src/protocol/storage/traits.rs +pub trait SessionStore { + async fn store_session( + &mut self, + address: &ProtocolAddress, + record: SessionRecord, // Owned — zero-cost move, no clone + ) -> Result<()>; +} +``` + +All four protocol-layer call sites (`message_encrypt`, `message_decrypt_signal`, `message_decrypt_prekey`, `process_prekey_bundle`) — plus PreKey setup and group preflight mutations — take ownership of the record and drop it immediately after storing. Taking ownership eliminates the `.clone()` in the adapter and the compiler enforces no use-after-store. As of [#1044](https://github.com/oxidezap/whatsapp-rust/pull/1044), these call sites obtain the record through the `SessionCheckout` guard (below) rather than a bare `load_session`/`store_session` pair, so the zero-cost move still applies but the checkout is now cancellation-safe. + +**Per-message hot path impact:** + +| Operation | Before | After | +|-----------|--------|-------| +| `store_session` | clone all fields + protobuf encode | move (zero-cost) | +| `load_session` | protobuf decode + construct | clone current session only (`previous_sessions` O(1) via Arc) | +| `peek_session` | deep-clone record (1–2 KB) | `Arc` refcount bump; returns `Option>` | +| `store_sender_key` | serialize to bytes + store bytes | store `SenderKeyRecord` object directly | +| `load_sender_key` (`&self`) | load bytes + deserialize | return cached `SenderKeyRecord` object (read lock only) | +| `flush` (batched) | write bytes to DB | serialize sessions + sender keys + write bytes to DB | + +### Cancellation-safe session checkouts + +As of [#1044](https://github.com/oxidezap/whatsapp-rust/pull/1044), the record backing a `CheckedOut` entry is owned by a `SessionCheckout<'a>` guard (`wacore/libsignal/src/protocol/storage/traits.rs`) instead of being a bare marker. This closes a gap where cancelling the future mid-mutation — a Tokio task abort, a `select!` losing a race, a timeout — could leave only the marker behind: the ratchet advance the future was computing was lost, and a decrypt's ratchet advance could persist without its matching identity write, or vice versa. + +`SessionCheckout` is obtained via `SessionCheckout::load` (existing session) or `load_or_create` (synthesizes `SessionRecord::new_fresh()` if absent, tracked via `had_session`), exposes `record()`/`record_mut()` for the protocol layer to mutate in place, and is consumed by either: +- `commit(self)` — stores the mutated record back through `try_store_session_from_checkout`, or +- `discard(self)` — releases a deliberately-rejected *fresh* (never-had-a-session) checkout without storing. + +If neither runs because the guard is dropped first (the cancellation case), `Drop` synchronously puts the record back itself — no caller code has to remember to handle cancellation. If the sessions mutex is uncontended, restoration is immediate; if a concurrent `flush()` holds the lock, the restore is queued as a `PendingSessionRestore` and replayed the next time any operation acquires the sessions lock, so an aborted task's state is never dropped even though it can't wait for the lock itself. The queue itself (`SyncMutex>`) is unbounded — every lock acquisition drains it first, so it only grows if cancellations under lock contention arrive faster than anything else touches the session store, which does not happen under normal load. + +Every checkout carries a `SessionCheckoutKey { generation, token }`: `checkout_generation` increments on every lossy `clear()`, and `token` is assigned monotonically per checkout of a given address. A restore is rejected (rather than silently applied) unless both match the cache's current state — this stops a checkout issued before a session reset from resurrecting stale state, and stops a stale owner from clobbering a newer checkout of the same address. + +`flush()` and cache eviction (`clear_after_flush`) now preserve live `CheckedOut` entries — and any PreKey deletion buffered against that address — instead of dropping them; the deferred prekey delete and the session's own persistence both wait for a later flush once the checkout completes, rather than being lost. + +`SessionStore` gained five `#[doc(hidden)]` methods with pass-through default bodies (`load_session_for_update`, `try_load_session_for_update`, `try_store_session_from_checkout`, `cancel_session_checkout`, `complete_session_checkout`), so existing custom `SessionStore` implementations keep compiling unchanged — only a backend that wants the destructive-update fast path needs to implement them. + +### Unboxed identity and session hooks + +As of [#1124](https://github.com/oxidezap/whatsapp-rust/pull/1124), the same "sync hook first, boxed async fallback second" pattern above extends to three more per-message checks: `IdentityKeyStore::try_is_trusted_identity`, `IdentityKeyStore::try_save_identity`, and `SessionStore::try_has_session`. `#[async_trait]` boxes every store method into a `Pin>`, regardless of whether the answer is already known. The bundled `IdentityAdapter`'s async `is_trusted_identity` body is an unconditional `Ok(true)` — WA Web `isTrustedIdentity` parity, since identity *changes* surface through `save_identity` rather than through trust checks. That box was therefore pure overhead, paid once per encrypt and once per decrypt. + +Each hook defaults to `None`, and `None` means "I cannot answer synchronously" — never "the answer is the default": +- `try_save_identity` declines when nothing is cached for the address, so the caller reads the backend. Answering from an empty cache would report every identity as new. +- `try_has_session` declines when the cache cannot answer, rather than reporting "no session" and forcing a needless session rebuild. +- `try_is_trusted_identity` can always answer **for the bundled adapter specifically**, because its async `is_trusted_identity` is already the unconditional `Ok(true)` above. This is not a general license to short-circuit trust checks: a custom `IdentityKeyStore` whose async `is_trusted_identity` enforces a real policy must not copy this hook verbatim — returning `Some(Ok(true))` unconditionally would let it bypass that policy. Such a store should return `None` unless it can decide synchronously without skipping any of its own trust logic. + +Three free functions in `wacore::libsignal::protocol` — `is_trusted_identity`, `save_identity`, `has_session` — express "ask the hook, then await the fallback" once instead of at each of the eight call sites that previously called the trait methods directly: + +```rust +// wacore/libsignal/src/protocol/storage/traits.rs +pub async fn is_trusted_identity( + store: &S, + address: &ProtocolAddress, + identity: &IdentityKey, + direction: Direction, +) -> Result { + match store.try_is_trusted_identity(address, identity, direction) { + Some(answer) => answer, + None => store.is_trusted_identity(address, identity, direction).await, + } +} +``` + +`message_encrypt`, `message_decrypt_signal`, and `ensure_sessions_for_devices` now call through these resolvers instead of the trait methods directly. The hooks are purely additive: a custom `IdentityKeyStore`/`SessionStore` that implements only the async methods keeps compiling and behaving unchanged, since declining both hooks falls through to exactly the async path it always ran. + +### Arc previous sessions + +`SessionRecord.previous_sessions` is wrapped in `Arc>`, making clone O(1) for the ~40 archived previous sessions that previously accounted for ~40% of the serialize cost: + +```rust +// wacore/libsignal/src/protocol/state/session.rs +pub struct SessionRecord { + current_session: Option, + /// Wrapped in Arc so cloning is O(1). Only mutated on rare paths via Arc::make_mut. + previous_sessions: Arc>, +} +``` + +Only rare operations (archive current session, promote previous session, take/restore during session setup) trigger `Arc::make_mut` and a deep copy. + +### Chain key buffer reuse + +As of [#1137](https://github.com/oxidezap/whatsapp-rust/pull/1137), advancing a chain key no longer allocates a fresh buffer for its persisted 32-byte key material on every step. `SessionState` stores each chain key's bytes as `Option` on the underlying protobuf `ChainKey` field. `Bytes` is immutable, so writing the ratcheted key used to be an unconditional `Bytes::copy_from_slice(..)` on every send or receive that advances a session's chain key. In the harness benchmark that motivated this change — a single-device 1:1 pingpong session — a full message round trip advances chain keys three times (twice sending, once receiving); a real send can touch more sessions than that, since [DM device fanout](#dm-device-fanout) encrypts separately for every resolved recipient and own-device session. + +`write_chain_key` (`wacore/libsignal/src/protocol/state/session.rs`) instead reuses the existing buffer in place when it safely can: + +```rust +fn write_chain_key(field: &mut Option, key: &[u8]) { + if let Some(existing) = field.take() + && existing.len() == key.len() + && let Ok(mut owned) = existing.try_into_mut() + { + owned.copy_from_slice(key); + *field = Some(owned.freeze()); + return; + } + *field = Some(bytes::Bytes::copy_from_slice(key)); +} +``` + +Reuse only happens when both guards pass: `try_into_mut()` succeeds solely when the `Bytes` is uniquely owned (no other clone observing the old key), and the length check keeps a differently-sized buffer (e.g. from a legacy record) from reaching `copy_from_slice` at all — like the slice method it resolves to via `DerefMut`, a length mismatch there panics rather than writing anything. Either guard failing falls back to the original allocating behavior. In steady state a checked-out session record is uniquely owned — the cache takes it out of its `Arc` via `try_unwrap` (see [Session object cache](#session-object-cache) above) — so the fallback is rare. + +### Redundant signal store write elimination + +The `SignalStoreCache` uses targeted deduplication strategies per store type. For identities (which rarely change), `put_dedup()` compares incoming bytes against the cached value and skips if identical: + +```rust +// wacore/src/store/signal_cache.rs — ByteStoreState +fn put_dedup(&mut self, address: &str, data: &[u8]) { + if let Some(Some(existing)) = self.cache.get(address) + && existing.as_ref() == data + { + return; // Skip — data unchanged, no dirty mark + } + self.put(address, data); +} +``` + +Sessions and sender keys use unconditional `put()` since they change with every message — dedup would always fail and waste CPU cycles. This split avoids unnecessary database writes during `flush()` while not adding overhead where it provides no benefit. + +### Key reuse in cache + +The `key_for()` method on `SessionStoreState`, `SenderKeyStoreState`, and `ByteStoreState` reuses existing `Arc` keys from the HashMap via `get_key_value()`, avoiding a heap allocation on every cache operation: + +```rust +fn key_for(&self, address: &str) -> Arc { + match self.cache.get_key_value(address) { + Some((existing, _)) => existing.clone(), // O(1) refcount bump + None => Arc::from(address), // Only on first insert + } +} +``` + +On the hot path (put/delete for addresses already in the cache), this is always a refcount bump instead of a heap allocation. + +### Single-allocation session lock keys + +Session lock keys use the full Signal protocol address string (e.g., `5511999887766@c.us.0`). The `JidExt` trait provides methods for generating these strings, defined in `wacore/src/types/jid.rs`: + +```rust +pub trait JidExt { + /// Construct a fresh ProtocolAddress for this JID. + fn to_protocol_address(&self) -> ProtocolAddress; + + /// Signal address string: `{user}[:device]@{server}` + /// Device part only included when device != 0. + fn to_signal_address_string(&self) -> String; + + /// Full protocol address string: `{signal_address_string}.0` + /// Equivalent to `to_protocol_address().to_string()` but avoids the + /// intermediate ProtocolAddress allocation — one String instead of two. + fn to_protocol_address_string(&self) -> String; + + /// Rewrite a reusable ProtocolAddress in place for this JID. + /// See [Reusable hot-loop address construction](#reusable-hot-loop-address-construction). + fn reset_protocol_address(&self, addr: &mut ProtocolAddress); +} +``` + +`to_protocol_address_string()` is used on hot paths (message encryption and decryption) as the key for `session_locks`. It pre-sizes the output buffer and builds the `String` in a single allocation. Constructing a `ProtocolAddress` itself no longer allocates for addresses that fit inline (see [Single-buffer ProtocolAddress](#single-buffer-protocoladdress) below), but `.to_string()` on top of it still does, so `to_protocol_address_string()` remains the cheaper path when only the string is needed. + +The `write_protocol_address_to()` free function provides the same formatting but writes into a caller-supplied `&mut String` buffer, enabling buffer reuse across multiple JIDs. + +**Format examples:** + +| JID | Signal address | Protocol address string | +|-----|---------------|------------------------| +| `5511999887766@s.whatsapp.net` | `5511999887766@c.us` | `5511999887766@c.us.0` | +| `5511999887766:33@s.whatsapp.net` | `5511999887766:33@c.us` | `5511999887766:33@c.us.0` | +| `123456789@lid` | `123456789@lid` | `123456789@lid.0` | +| `123456789:33@lid` | `123456789:33@lid` | `123456789:33@lid.0` | + + + The server `s.whatsapp.net` is mapped to `c.us` in address strings, matching WhatsApp Web's internal format. The trailing `.0` is the Signal device_id (always 0 in WhatsApp's usage). + + +**Usage in message processing:** + +```rust +// In message decryption (src/message.rs) — single lock per sender device +let signal_addr_str = sender_encryption_jid.to_protocol_address_string(); +let session_mutex = self.session_locks + .get_with(signal_addr_str.clone(), async { + Arc::new(async_lock::Mutex::new(())) + }).await; +let _session_guard = session_mutex.lock().await; + +// In peer message encryption (src/send.rs) — single lock +let signal_addr_str = encryption_jid.to_protocol_address_string(); + +// In DM message encryption (src/send.rs) — per-device locks for all devices +let lock_jids = self.build_session_lock_keys(&all_dm_devices).await; +let session_guards = self.session_guards_for(&lock_jids).await; +// Each lock taken as its mutex is resolved, in sorted order, to prevent deadlocks +``` + +**DM multi-device fanout:** + +The DM send path resolves all known recipient devices and own companion devices, encrypting per-device for each. This matches WA Web's `WAWebSendUserMsgJob` behavior where the local device table is read on the send path, and `WAWebDBDeviceListFanout` filters out hosted devices. The client checks the local device registry first (via `get_devices_from_registry()`); a network fetch is only triggered on a cache miss to avoid unnecessary LID-migration side effects from `get_user_devices`. The sender device is excluded (matching WA Web's `isMeDevice` in `getFanOutList`), and for self-DMs, overlapping device lists are deduplicated using a `HashSet` (matching WA Web's `Map` keyed by `toString`). + +**Own-device namespace alignment (v0.6):** When the recipient is addressed in the LID namespace (`@lid`), the client converts its own companion devices from the PN namespace to LID before fanning out. Without this alignment, a `` mix of `@lid` and `@s.whatsapp.net` participants caused the server to reject the stanza for LID-addressed DMs. Outgoing messages to PN-addressed recipients are unaffected. + + + The recipient's namespace here is decided by [`resolve_dm_wire_jid()`](#dm-wire-namespace-vs-signal-session-addressing), not a raw mapping lookup — on an account that isn't 1:1-LID-migrated, the recipient (and therefore the whole fanout) stays PN even when a LID mapping is cached. + + +Own companion devices (your other linked devices) receive per-device encryption for multi-device self-sync via `DeviceSentMessage`. + + +WA Web has a bare-`` fast path for single primary device (`WAWebSendMsgCreateFanoutStanza`). This is not implemented in whatsapp-rust because `encrypt_for_devices` always wraps in `` nodes. The `` form is accepted by the server regardless. + + + +**Fail-fast on total encrypt failure (v0.6).** If per-device encryption fails for *every* recipient device, the DM send now returns an error instead of emitting a stanza with an empty participant list (which the server would silently swallow, making the message look sent when it wasn't). A partial failure — some devices encrypt, some don't — still sends to the devices that succeeded. + + +**DM per-device locking:** + +To prevent ratchet desync when concurrent sends and receives operate on the same Signal session, the DM path acquires session locks for all devices involved — the bare recipient plus own companion devices. The `build_session_lock_keys()` helper resolves encryption JIDs and sorts them for deadlock-free lock acquisition: + +1. Resolves the recipient to its bare encryption JID via `resolve_encryption_jid().to_non_ad()` (stripping device component) +2. Resolves own companion device JIDs +3. Sorts by `(server, user, device)` using `cmp_for_lock_order()` and deduplicates +4. Returns sorted `Vec` — no intermediate `String` allocations needed for sorting + +The `session_guards_for()` helper (PR [#1131](https://github.com/oxidezap/whatsapp-rust/pull/1131)) then takes each device's lock as its mutex is resolved from the sorted JIDs, reusing a single `ProtocolAddress` via `reset_protocol_address()` to name each lock key without a formatting buffer: + +```rust +// In DM message encryption (src/send.rs) +let lock_jids = self.build_session_lock_keys(&all_dm_devices).await; +let session_guards = self.session_guards_for(&lock_jids).await; +// Each lock taken as its mutex is resolved, in sorted order, to prevent deadlocks +``` + + + Resolving every mutex first and then locking each in a second pass (the previous implementation) is still available as `session_mutexes_for()`, but only under `#[cfg(test)]` — production code no longer builds the intermediate `Vec` of mutex handles. + + +The recipient lock key is always the bare form (e.g., `100000012345678@lid.0`), matching the decrypt path's lock format. This ensures send and receive paths serialize on the exact same lock key. + +Location: `wacore/src/types/jid.rs:4-51`, `src/send.rs:1481-1507` + +### Single-buffer ProtocolAddress + +The `ProtocolAddress` struct stores the full address string `"{name}.{device_id}"` in a single `AddressBuf` buffer, with a `name_len` marker to split name from suffix. Since [#1131](https://github.com/oxidezap/whatsapp-rust/pull/1131), `AddressBuf` is inline-first: addresses up to 47 bytes (`INLINE_CAPACITY`) live inside the value itself with no heap allocation at all — every real WhatsApp address fits, e.g. `"5511987650001:5@c.us.0"` is 22 bytes. A longer address spills to a `String`. + +Which arm holds the bytes is not part of the value: `Eq`, `Ord`, and `Hash` all read the rendered string via `as_str()`, so an inline-built key finds a heap-spilled entry (and vice versa) in the session cache. That equivalence is what makes the optimization safe rather than a silent cache-miss generator. + +```rust +// wacore/libsignal/src/core/address.rs +const INLINE_CAPACITY: usize = 47; + +pub struct ProtocolAddress { + buf: AddressBuf, // inline up to 47 bytes, spills to a `String` beyond that + name_len: usize, // marks where the name ends + device_id: DeviceId, +} + +impl ProtocolAddress { + /// One-shot construction: writes `name` into the buffer and appends the suffix. + pub fn new(name: &str, device_id: DeviceId) -> Self; + + /// An address with no name yet, ready for `reset_with()`. No capacity argument — + /// the buffer starts inline and only allocates if an address ever exceeds it. + pub fn empty(device_id: DeviceId) -> Self; + + /// Rewrite the address in place via closure. Single write pass — no intermediate copy. + pub fn reset_with(&mut self, write_name: impl FnOnce(&mut AddressBuf)); + + /// Zero-cost slice of the name portion. + pub fn name(&self) -> &str; + + /// Zero-cost slice of the full buffer ("{name}.{device_id}"). + pub fn as_str(&self) -> &str; +} +``` + +Both `name()` and `as_str()` are zero-cost slices into the same buffer — no allocations on access, whether inline or spilled. + + +**API change (PR [#1131](https://github.com/oxidezap/whatsapp-rust/pull/1131)):** `ProtocolAddress::new` now takes `name: &str` instead of an owned `String`. `with_capacity(capacity, device_id)` was removed in favor of `empty(device_id)` — the buffer no longer needs a capacity hint, since it starts inline and only allocates on overflow. `reset_with()`'s closure now receives `&mut AddressBuf` instead of `&mut String`; `AddressBuf` implements `push_str`, `push`, and `std::fmt::Write`, which covers existing call sites. + + + +`Debug` on `AddressBuf` (and `ProtocolAddress`) must format `as_str()`, never the backing byte array: clearing an inline buffer only rewinds its length, so the unused tail still holds whichever address occupied it before. A derived `Debug` would print the whole array and could leak an unrelated peer's JID into a log line or error that formats a reused address. + + +### Reusable hot-loop address construction + +When iterating over many devices (e.g., during group stanza preparation or session resolution), allocating a fresh `ProtocolAddress` per device is wasteful. The `JidExt` trait provides `reset_protocol_address()` to rewrite a pre-allocated address in place, and `make_reusable_protocol_address()` creates the initial buffer: + +```rust +// wacore/src/types/jid.rs +pub fn make_reusable_protocol_address() -> ProtocolAddress { + ProtocolAddress::empty(SIGNAL_DEVICE_ID) +} + +pub trait JidExt { + /// Rewrite a reusable ProtocolAddress in place for this JID. + fn reset_protocol_address(&self, addr: &mut ProtocolAddress); +} +``` + +**Usage in group stanza preparation:** + +```rust +// wacore/src/send.rs — session resolution loop +let mut reusable_addr = make_reusable_protocol_address(); + +for device_jid in devices { + // Rewrite the same buffer — no new allocation per device + device_jid.reset_protocol_address(&mut reusable_addr); + + if stores.session_store.load_session(&reusable_addr).await?.is_some() { + // Session exists — use it + } +} +``` + +Since [#1131](https://github.com/oxidezap/whatsapp-rust/pull/1131), a fresh `ProtocolAddress` for a typical WhatsApp address (up to 47 bytes) is already allocation-free — inline storage covers it. Reuse still matters for addresses that spill to the heap: the first spill on a reused buffer still allocates its backing `String`, but every reset after that keeps the existing allocation instead of dropping and reallocating one per device — for a group with 100 participant devices past the inline limit sharing a reused buffer, that saves up to 99 heap allocations on the send path. + + + Use `to_protocol_address()` for one-shot address construction (e.g., cache keys, single lookups). Use `make_reusable_protocol_address()` + `reset_protocol_address()` when iterating over multiple JIDs in a tight loop. + + +Location: `wacore/libsignal/src/core/address.rs`, `wacore/src/types/jid.rs`, `wacore/src/send.rs` + +### Zero-Allocation JID Deduplication + +Group stanza preparation needs to deduplicate participant JIDs at two stages: before device resolution (by user identity) and after LID conversion (by device identity). Two utility functions in `wacore/src/types/jid.rs` handle this with in-place sorted dedup instead of `HashSet` allocations: + +```rust +/// Sort and deduplicate by user identity (user + server). +pub fn sort_dedup_by_user(jids: &mut Vec); + +/// Sort and deduplicate by device identity (user + server + device + integrator + identity_agent). +pub fn sort_dedup_by_device(jids: &mut Vec); +``` + +`sort_dedup_by_device` keys on `Jid::identity_agent()` rather than the raw `agent` field, so its notion of "same device" matches exactly what `Jid`'s `PartialEq`/`Hash` already treat as equal (see [Binary Protocol](/advanced/binary-protocol#jid-encoding) for why the two fields differ). That has to hold in both directions: keying on the raw `agent` would let two JIDs that are actually one device — an inert agent byte on `Pn`/`Lid`/`Hosted`/`HostedLid`, same AD-JID, same Signal address — both survive the dedup and pick up two concurrent encryption jobs against one session; dropping `agent` from the key entirely would go too far the other way and silently collapse two genuinely distinct `@bot`/`@interop` devices, which *do* render it, losing a fan-out destination. + +Both use `sort_unstable_by` followed by `dedup_by`, comparing JID fields directly without allocating intermediate strings or hash sets. This is more efficient than the `HashSet<(String, String)>` approach because: + +- No per-JID `String::clone()` for hash keys +- No `HashSet` allocation or hashing overhead +- Stable dedup order (sorted) instead of hash-dependent iteration + +**Usage in group sends (`wacore/src/send.rs`):** + +```rust +// Before device resolution — dedup participants by user identity +sort_dedup_by_user(&mut jids_to_resolve); + +// After LID conversion — dedup devices by full device identity +// Catches duplicates where both phone and LID queries resolve +// to the same device (e.g., 559980000003:33 and 100000037037034:33@lid) +sort_dedup_by_device(&mut resolved_list); +``` + +Location: `wacore/src/types/jid.rs:33-51` + +### Take/Restore Pattern + +Avoids cloning session states during decryption attempts: + +```rust +// Take ownership instead of cloning +if let Some(mut current_state) = record.take_session_state() { + let result = decrypt_message_with_state(&mut current_state, ...); + match result { + Ok(ptext) => { + record.set_session_state(current_state); + return Ok(ptext); + } + Err(e) => { + record.set_session_state(current_state); // Restore + } + } +} +``` + +Location: `wacore/libsignal/src/protocol/session_cipher.rs:495-564` + +### Buffer Reuse + +Thread-local buffers eliminate per-message allocations: + +```rust +struct EncryptionBuffer { + buffer: Vec, + usage_count: usize, +} + +const INITIAL_CAPACITY: usize = 1024; +const MAX_CAPACITY: usize = 16 * 1024; +const SHRINK_THRESHOLD: usize = 100; + +fn get_buffer(&mut self) -> &mut Vec { + self.usage_count += 1; + if self.usage_count.is_multiple_of(SHRINK_THRESHOLD) { + if self.buffer.capacity() > MAX_CAPACITY { + self.buffer = Vec::with_capacity(INITIAL_CAPACITY); + } + } + &mut self.buffer +} +``` + +Location: `wacore/libsignal/src/protocol/session_cipher.rs:20-54` + +## Public API + +The `client.signal()` accessor exposes low-level Signal protocol operations for direct use. This includes 1:1 and group encryption/decryption, session validation, session deletion, participant node creation, and device resolution. + +See [Signal API reference](/api/signal) for full method documentation and examples. + +## Related Components + +- [Binary Protocol](/advanced/binary-protocol) - How encrypted messages are serialized +- [State Management](/advanced/state-management) - How session state is persisted +- [WebSocket Handling](/advanced/websocket-handling) - Transport layer for encrypted messages +- [Signal API](/api/signal) - Public API for Signal protocol operations + +## References + +- [Signal Protocol Specification](https://signal.org/docs/) +- [libsignal Repository](https://github.com/signalapp/libsignal) +- Source: `wacore/libsignal/src/protocol/` +- Storage: `src/store/signal.rs`, `src/store/signal_adapter.rs`, `wacore/src/store/signal_cache.rs` \ No newline at end of file From 0ba7b05bd1c5043ee3ffdb32eb8a7478f98e6cb1 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 5 Aug 2026 17:29:06 +0000 Subject: [PATCH 3/3] docs(signal-protocol): tighten prose per style guide Split two dense multi-clause passages (persisted-seed behavior and the legacy v1 export error) into shorter, one-idea sentences, per this repo's active-voice/second-person style guideline. Addresses CodeRabbit review comment on PR #474. Co-Authored-By: Claude Sonnet 5 --- advanced/signal-protocol.mdx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/advanced/signal-protocol.mdx b/advanced/signal-protocol.mdx index 2dac8bbc..baa053e3 100644 --- a/advanced/signal-protocol.mdx +++ b/advanced/signal-protocol.mdx @@ -1389,10 +1389,10 @@ pub enum SessionMessageKeyMaterial { } ``` -As of [#1210](https://github.com/oxidezap/whatsapp-rust/pull/1210), `Seed` is no longer just a compact *import* form expanded through the same canonical derivation used elsewhere (see [`MessageKeyGenerator`](#chain-key-ratcheting)) — the seed is now persisted, so a skipped key round-trips back out as `Seed` too. `session_structure::chain::MessageKey` carries an additive `seed` field (local field 100) alongside the pre-existing `cipher_key`/`mac_key`/`iv`; `from_structure` prefers it when present, re-deriving from it and rejecting the record if the result disagrees with the stored triple (the seed supersedes the triple on export, so a mismatch means a corrupt record). `Derived` is now what comes back only for a key persisted *before* this change, which never wrote a seed. Both variants are fixed-width and `Copy`, so `into_structure()` — the reverse direction — is infallible where it used to validate a `Vec` length. +As of [#1210](https://github.com/oxidezap/whatsapp-rust/pull/1210), the seed is persisted alongside the keys it derives, not just accepted on import. Previously `Seed` was only a compact import form, expanded through the same canonical derivation used elsewhere (see [`MessageKeyGenerator`](#chain-key-ratcheting)). Now a skipped key's seed round-trips back out on export too. `session_structure::chain::MessageKey` carries an additive `seed` field (local field 100) alongside the pre-existing `cipher_key`/`mac_key`/`iv`. `from_structure` prefers the seed when present: it re-derives from the seed and rejects the record if the result disagrees with the stored triple, since the seed supersedes the triple on export and a mismatch means a corrupt record. `Derived` now comes back only for a key persisted *before* this change — one that never wrote a seed. Both variants are fixed-width and `Copy`, so `into_structure()`, the reverse direction, is now infallible; it used to validate a `Vec` length. -**Breaking for direct consumers of these types.** `SessionMessageKeyMaterial::Seed`/`Derived` switched from `Vec` fields to fixed-width arrays — match both variants rather than assuming `Derived` is the only exported form. `MessageKeyGenerator::Keys(MessageKeys)` was removed in the same change (nothing in the workspace constructed it); build a seeded key with `MessageKeyGenerator::new_from_seed` instead. Downgrading to a build predating #1210 and writing a record back out drops the seed silently — decrypt is unaffected, but the key becomes unexportable again. +**Breaking for direct consumers of these types.** `SessionMessageKeyMaterial::Seed` and `Derived` switched from `Vec` fields to fixed-width arrays. Match both variants — don't assume `Derived` is the only exported form. `MessageKeyGenerator::Keys(MessageKeys)` was also removed; nothing in the workspace constructed it. Build a seeded key with `MessageKeyGenerator::new_from_seed` instead. If you downgrade to a build predating #1210 and write a record back out, the seed drops silently — decrypt is unaffected, but the key becomes unexportable again. Conversions: @@ -1512,10 +1512,10 @@ impl SessionRecord { This is a deterministic operational projection, not a round trip: v1 lifecycle timestamps and base-key lookup roles are reconstructed from canonical search/eviction order rather than recovered verbatim, since the canonical record never persisted them in the first place. State the v1 format genuinely cannot represent is rejected with a typed error instead of being silently dropped or inferred — a session with no sender chain, a non-current `session_version`, a pending key exchange, or a `needs_refresh` flag all fail as `NotRepresentable`; a pending pre-key whose base key doesn't match the session's own base key fails as `PendingPreKeyBaseMismatch` rather than producing v1 state the importer would reject on the way back in. -A receiver chain holding a *derived* (seedless) skipped-message key, with no inverse to a v1 seed, fails as `ChainNotRepresentable::DerivedMessageKey`. Before [#1210](https://github.com/oxidezap/whatsapp-rust/pull/1210) this was every skipped key without exception — the projection was lossy from the first cycle, since import expanded a v1 seed into derived keys and export could never recover it, so a v1 record with a skipped key failed to round-trip even immediately. Now that the seed rides along with the keys it derives (see [`SessionMessageKeyMaterial`](#session-and-sender-key-shapes) above), only a key persisted *before* that change — which never had a seed to retain — still hits this error; a skipped key imported or received after #1210 carries its seed through export and projects normally, permanently, until it is consumed or evicted. +A receiver chain holding a *derived* (seedless) skipped-message key fails as `ChainNotRepresentable::DerivedMessageKey` — it has no inverse to a v1 seed. Before [#1210](https://github.com/oxidezap/whatsapp-rust/pull/1210), this was every skipped key without exception. Import expanded a v1 seed into derived keys, and export could never recover it, so a v1 record with a skipped key failed to round-trip even on the very first cycle. Now that the seed rides along with the keys it derives (see [`SessionMessageKeyMaterial`](#session-and-sender-key-shapes) above), only a key persisted *before* that change still hits this error — one that never had a seed to retain. A skipped key imported or received after #1210 carries its seed through export and projects normally. That holds permanently, until the key is consumed or evicted. -CI runs a dedicated `cargo nextest run --features legacy-session-interop` job as of #1210. The feature was previously off in every job, so this entire module's test suite — including the regression test for the bug above — compiled away and never ran. +CI runs a dedicated `cargo nextest run --features legacy-session-interop` job as of #1210. The feature was previously off in every job. This entire module's test suite — including the regression test for the bug above — compiled away and never ran. Every type in the module redacts key material from `Debug` — root keys, chain keys, ratchet key pairs, skipped-message seeds, and identity keys all print as ``; only structural fields (roles, counters, indexes, session/chain counts) print plainly, the same convention as the canonical [`*Components` types](#debug-output-redacts-secrets).