Skip to content

refactor: targeted single-device DM retry (matches WA Web) - #549

Merged
jlucaso1 merged 9 commits into
mainfrom
copilot/fix-dm-retry-handler-sessions-again
Apr 15, 2026
Merged

refactor: targeted single-device DM retry (matches WA Web)#549
jlucaso1 merged 9 commits into
mainfrom
copilot/fix-dm-retry-handler-sessions-again

Conversation

Copilot AI commented Apr 15, 2026

Copy link
Copy Markdown
Contributor

Replace the full DM fanout resend with a WA Web-aligned targeted retry that encrypts only for the requesting device.

What changed

wacore/src/send.rs

  • New prepare_dm_retry_stanza: builds a single-device encrypted stanza with <participants><to jid="device"> and count="N" on the <enc> node, matching WAWebSendMsgCreateDeviceStanza's retry structure.

src/retry.rs

  • DM retries now delete the session and re-establish it for only the requesting device, not all known devices. This matches WAWebUpdateLocalSignalSession which operates on the single requester.
  • Resend uses prepare_dm_retry_stanza + send_node instead of the full send_message_impl fanout path.
  • Per-device session locks acquired before both group and DM retry encryption (fixes pre-existing gap).
  • Retry dedupe key now always includes the participant device, so multi-device DM retries are handled independently instead of the second device being silently dropped.
  • take_recent_message re-adds unconditionally (not just for groups), so companion devices can still retry the same message.

tests/e2e/tests/retry_dm_multidevice.rs

  • E2E test: establishes sessions, deletes B's session with A, sends a message triggering retry, then asserts the retry stanza has exactly one <to> target with count="1" and verifies bidirectional messaging still works.

Unit tests

  • Stanza structure tests for prepare_dm_retry_stanza (attributes, device-identity conditional)
  • dm_retry_deletes_only_requested_session: verifies only the requesting device's session is deleted
  • retry_dedupe_key_per_participant: verifies DM and status dedupe keys differentiate by device
  • recent_message_cache_readd_after_take: covers both DM and status re-add scenarios

@jlucaso1 jlucaso1 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: Verified against WhatsApp Web captured JS

WhatsApp Web behavior (WAWebHandleRetryRequest + WAWebUpdateLocalSignalSession)

The WA Web retry flow works differently from ours — worth noting for context:

WA Web approach (targeted resend):

  1. handleRetryRequest extracts the requester: var g = e.isUser() ? r : u (DM: from, group: participant)
  2. updateLocalSignalSession(chat, receipt) deletes session for the single requester device (_ = participant || from)
  3. ensureE2ESessions([g]) — ensures session for only the requester device, not all devices
  4. Resend goes through WAWebSendRetryMsgJob.sendRetry() — a targeted resend to the specific recipient, NOT through the full DM fanout

Our approach (broad session cleanup + full fanout resend):

  1. delete_dm_retry_sessions deletes sessions for all known devices of the requester
  2. Resend goes through send_message_impl — the full DM fanout path that encrypts for all devices

Why this PR is correct

The architectural difference (WA Web uses targeted resend, we use full fanout resend) is the root cause of the bug. Since send_message_impl encrypts for ALL devices via ensure_e2e_sessions, any device with a stale session gets reused as-is (the has_session check returns true and skips prekey fetch).

The PR solves this correctly: by deleting sessions for all known devices before the full-fanout resend, ensure_e2e_sessions sees has_session = false for every device and fetches fresh prekeys for all of them. This is slightly broader than WA Web's approach but safe because:

  • Fresh sessions are always re-established during ensure_e2e_sessions
  • No stale PreKeyMessages with consumed prekey IDs can leak through
  • Base key collision tracking per-device is safe (keyed by (address, msg_id))

Code review

  1. dm_retry_session_targets — Clean. to_non_ad() strips device info for registry lookup, fallback to just the resolved JID when no registry entry exists. ✓

  2. delete_dm_retry_sessions — Base key collision logic correctly applied per-device, flush_signal_cache() once at the end. ✓

  3. Tests — Both the happy path (known devices) and fallback (no registry) are covered. ✓

Optional future improvement

A more WA-Web-aligned approach would be to add a targeted send_retry_message that re-encrypts for the specific requester device only (like WAWebSendRetryMsgJob.sendRetry), avoiding the need to delete all device sessions. But that's a bigger refactor and this fix is correct for now.

LGTM — fixes the regression from the per-device DM fanout change.

Copilot AI and others added 3 commits April 15, 2026 01:02
@jlucaso1

Copy link
Copy Markdown
Collaborator

The "future improvement" from the review should be implemented in this PR.

The current approach (delete all device sessions + full fanout resend) works around the architectural mismatch rather than fixing it. WA Web does a targeted single-device retry, and we should match that. Here's how:

WA Web DM Retry Flow (WAWebHandleRetryRequest + WAWebSendRetryMsgJob)

1. handleRetryRequest → resolves single requester device: g = e.isUser() ? r : u
2. updateLocalSignalSession(chat, receipt) → deletes session for ONLY device g
3. ensureE2ESessions([g]) → ensures session for ONLY device g
4. sendRetry → createUserDeviceMsgStanza → encrypts for ONLY device g with count=N

Key: WA Web never touches other devices' sessions and never re-fans-out to all devices. The retry is a surgical re-encrypt for the single failing device.

Implementation Plan

1. Add prepare_dm_retry_stanza in wacore/src/send.rs

Model it after prepare_group_retry_stanza (line 870). Differences from the group version:

  • No participant attribute on the <message> (that's group-only)
  • No addressing_mode attribute
  • Wrap the <enc> in <participants><to jid="device_jid">...</to></participants> (DM stanza structure) instead of putting <enc> directly as a child of <message> (group retry structure)
  • Same count="N" attribute on <enc>, same <device-identity> if prekey message
pub async fn prepare_dm_retry_stanza<S, I>(
    session_store: &mut S,
    identity_store: &mut I,
    to_jid: Jid,           // user-level JID (chat target)
    requester_jid: Jid,    // device-specific JID to encrypt for
    encryption_jid: Jid,   // resolved encryption JID (after LID mapping)
    message: &wa::Message,
    message_id: String,
    retry_count: u8,
    account: Option<&wa::AdvSignedDeviceIdentity>,
) -> Result<Node>

The stanza structure:

<message to="user@lid" id="MSG_ID" type="text">
  <participants>
    <to jid="user:device@lid.0">
      <enc v="2" type="msg|pkmsg" count="N" mediatype="...">CIPHERTEXT</enc>
    </to>
  </participants>
  <device-identity>...</device-identity>  <!-- only if prekey message -->
</message>

2. Change src/retry.rs DM retry path

Revert to single-device session deletion (the original behavior was correct for this), then use the targeted stanza instead of send_message_impl:

} else {
    // DM retry: delete session + base key tracking for ONLY the requesting device
    // (matches WAWebUpdateLocalSignalSession which operates on single requester)
    self.delete_dm_session(&resolved_jid, &message_id, retry_count).await?;

    // Ensure session with only the requesting device
    self.ensure_e2e_sessions(std::slice::from_ref(&resolved_jid)).await?;

    let device_snapshot = self.persistence_manager.get_device_snapshot().await;
    let mut store_adapter = self.signal_adapter().await;

    let stanza = wacore::send::prepare_dm_retry_stanza(
        &mut store_adapter.session_store,
        &mut store_adapter.identity_store,
        receipt.source.chat.clone(),
        participant_jid,
        resolved_jid.clone(),
        &original_msg,
        message_id,
        retry_count,
        device_snapshot.account.as_ref(),
    ).await?;

    self.send_node(stanza).await?;
    self.flush_signal_cache().await?;
}

3. Remove delete_dm_retry_sessions / dm_retry_session_targets — no longer needed since we only target one device.

E2E Test

The test should validate the full retry recovery cycle and verify the stanza structure is targeted (not a full fanout). Using the existing wait_for_sent_node infrastructure:

#[tokio::test]
async fn test_dm_retry_sends_targeted_stanza() -> anyhow::Result<()> {
    let mut client_a = TestClient::connect("e2e_retry_targeted_a").await?;
    let mut client_b = TestClient::connect("e2e_retry_targeted_b").await?;

    let jid_a = client_a.jid().await;
    let jid_b = client_b.jid().await;

    // Establish sessions with a full roundtrip
    send_and_expect_text(&client_a.client, &mut client_b, &jid_b, "setup-a2b", 30).await?;
    send_and_expect_text(&client_b.client, &mut client_a, &jid_a, "setup-b2a", 30).await?;

    // Delete B's session with A to force a retry on next message from A
    client_b.client.signal().delete_sessions(std::slice::from_ref(&jid_a)).await?;

    // Set up stanza waiters BEFORE the send that triggers the retry.
    // First waiter catches the initial send, second catches A's retry resend.
    let _initial_waiter = client_a.next_sent_message_waiter();
    let retry_waiter = client_a.next_sent_message_waiter();

    // This send triggers: A encrypts → B can't decrypt → B sends retry → A resends
    send_and_expect_text(&client_a.client, &mut client_b, &jid_b, "trigger-retry", 30).await?;

    // Capture and verify the retry stanza that A sent
    if let Ok(retry_node) = retry_waiter.await {
        // Verify targeted: <participants> should have exactly ONE <to> child
        // Verify retry indicator: <enc> should have count="..." attribute
    }

    // Verify bidirectional messaging still works after retry recovery
    send_and_expect_text(&client_a.client, &mut client_b, &jid_b, "post-retry-a2b", 15).await?;
    send_and_expect_text(&client_b.client, &mut client_a, &jid_a, "post-retry-b2a", 15).await?;

    client_a.disconnect().await;
    client_b.disconnect().await;
    Ok(())
}

Key stanza assertions on the retry node:

  • <participants> has exactly one <to> child (targeted, not fanout)
  • The <enc> has a count attribute (retry indicator, absent on normal sends)
  • to attribute on the <message> matches B's user JID

Copilot stopped work on behalf of jlucaso1 due to an error April 15, 2026 01:11
Copilot AI requested a review from jlucaso1 April 15, 2026 01:11
@jlucaso1

Copy link
Copy Markdown
Collaborator

@copilot check my comments and start implementing these things. (all the "future improvement" related things)

Copilot AI and others added 2 commits April 15, 2026 01:24

Copilot AI commented Apr 15, 2026

Copy link
Copy Markdown
Contributor Author

The "future improvement" from the review should be implemented in this PR.

The current approach (delete all device sessions + full fanout resend) works around the architectural mismatch rather than f...

Implemented the targeted DM retry path in bd12256: DM retries now delete and re-establish only the requesting device session, build a single-device retry stanza via prepare_dm_retry_stanza, and the e2e test now asserts the retry resend has exactly one <to> target with count="1".

@copilot check my comments and start implementing these things. (all the "future improvement" related things)

Started and finished the targeted-retry refactor in bd12256: the DM retry flow now uses a single-device resend instead of send_message_impl, with unit coverage in wacore and an e2e assertion that the retry stanza targets one device only.

@jlucaso1

Copy link
Copy Markdown
Collaborator

@claude review

@jlucaso1
jlucaso1 marked this pull request as ready for review April 15, 2026 01:29
@claude

claude Bot commented Apr 15, 2026

Copy link
Copy Markdown

Claude finished @jlucaso1's task in 4m 51s —— View job


Code Review: Targeted DM Retry Handler

  • Read PR diff and all changed files
  • Review wacore/src/send.rsprepare_dm_retry_stanza
  • Review src/retry.rs — DM retry path changes
  • Review tests/e2e/tests/retry_dm_multidevice.rs — E2E test
  • Post review findings

Overall this is a solid refactor that correctly aligns the DM retry path with WA Web's targeted single-device behavior. The stanza structure, session management, and test coverage all look good. A few things to flag:

Bug / Correctness

Missing flush_signal_cache after session deletion in delete_dm_retry_session_targetsrc/retry.rs:546

The old inline code called self.flush_signal_cache().await? immediately after self.signal_cache.delete_session(...). The new extracted method does not:

// delete_dm_retry_session_target (line 546)
self.signal_cache.delete_session(&signal_address).await;
info!("Deleted session for {signal_address} due to retry receipt");
Ok(())

The cache deletion happens in-memory, and the flush only occurs at the end of the resend path (line 460). This works because ensure_e2e_sessions (line 440) reads through the signal cache (which already has the deletion), so it correctly sees "no session" and fetches fresh prekeys. However, if the process crashes between the delete and the eventual flush, the stale session survives on disk. The old code was more crash-resilient here.

This is low-risk in practice (the retry would just fire again), but worth noting. Consider either:

  • Adding self.flush_signal_cache().await? back into delete_dm_retry_session_target, or
  • Adding a brief comment explaining why the deferred flush is intentional

Group retry: missing ensure_e2e_sessions?

The DM retry path correctly calls ensure_e2e_sessions before encryption (line 440), matching WA Web's ensureE2ESessions([g]). The group retry path (line 410-437) does not call it — it relies on the pairwise session already existing (either from the key bundle processed earlier at line 228, or from a pre-existing session). This is fine when the key bundle is present, but if process_retry_key_bundle fails (line 232-233 logs a warning but continues), the group retry would attempt to encrypt with whatever stale session remains. Not a regression from this PR, just an observation.

Stanza structure — verified correct

prepare_dm_retry_stanza (wacore/src/send.rs:869-927) produces the right structure:

<message to="user@s.whatsapp.net" id="MSG_ID" type="text">
  <participants>
    <to jid="user:device@lid.0">
      <enc v="2" type="pkmsg" count="1" ...>CIPHERTEXT</enc>
    </to>
  </participants>
  <device-identity>...</device-identity>  
</message>

Key differences from prepare_group_retry_stanza are correctly implemented:

  • No participant attribute on <message>
  • No addressing_mode attribute ✓
  • <enc> wrapped in <participants><to> (DM structure) instead of direct child (group structure) ✓
  • count="N" retry indicator present ✓
  • device-identity conditional on prekey message ✓

Nits

  1. Redundant importssrc/retry.rs:934-935: Both use std::sync::Arc and use wacore::types::jid::JidExt as _ are already available via use super::* (from lines 9 and 16). Not harmful, but unnecessary. Fix this →

  2. E2E test race windowtests/e2e/tests/retry_dm_multidevice.rs:86-88: The retry_waiter is registered after the initial send completes. There's a theoretical race where A could process B's retry receipt and resend before the waiter is registered. In practice this requires multiple network round trips so it's extremely unlikely, but for robustness you could register both waiters before the initial send (like the reviewer's pseudocode suggested).

  3. #[allow(clippy::too_many_arguments)] on prepare_dm_retry_stanza — matches the existing pattern on prepare_group_retry_stanza, so it's consistent. Future refactor could introduce a shared struct, but not needed now.

Tests

Unit tests (dm_retry_deletes_only_requested_session and dm_retry_deletes_resolved_session_without_registry_devices) correctly verify single-device targeting and fallback behavior. The wacore stanza tests verify structure, attributes, and device-identity inclusion. E2E test covers the full retry recovery cycle with stanza assertions. Good coverage.

Verdict

The architectural change from full-fanout resend to targeted single-device retry is correct and well-implemented. The code follows existing patterns, the stanza matches WA Web's structure, and the old multi-device deletion methods are properly cleaned up. LGTM with the minor notes above.


Agent-Logs-Url: https://github.com/jlucaso1/whatsapp-rust/sessions/159b014c-04f1-48f9-90a5-0ffa202f5a54

Co-authored-by: jlucaso1 <55464917+jlucaso1@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix DM retry handler to delete sessions for all known devices [WIP] Refactor DM retry handler to use targeted single-device resend Apr 15, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bd122568bf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/retry.rs
Comment on lines +439 to +440
// DM retry: pairwise resend to the requesting device only.
self.ensure_e2e_sessions(std::slice::from_ref(&resolved_jid))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Include device in DM retry dedupe key

This branch now resends a DM retry only to the requesting device, but DM deduplication is still keyed only by (chat, msg_id) (src/retry.rs:131-145). In multi-device chats, once one device’s retry is handled, subsequent retry receipts from the user’s other devices are treated as duplicates and skipped, so those devices never get their targeted resend and remain undecryptable. Deduping needs to include the sender device when DM retries are device-targeted.

Useful? React with 👍 / 👎.

Comment thread src/retry.rs
Comment on lines +446 to 449
let stanza = wacore::send::prepare_dm_retry_stanza(
&mut store_adapter.session_store,
&mut store_adapter.identity_store,
receipt.source.chat.clone(),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Acquire session lock before DM retry encryption

The new DM retry flow encrypts directly with prepare_dm_retry_stanza without taking the per-device session lock, unlike the normal DM send path (src/send.rs:1137-1142). If this retry runs concurrently with another send/decrypt for the same Signal address, both paths can mutate the same ratchet state at once and cause additional undecryptable messages or retry loops. Wrap retry encryption with the same session-locking discipline used by regular DM sends.

Useful? React with 👍 / 👎.

@jlucaso1

Copy link
Copy Markdown
Collaborator

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Apr 15, 2026

Copy link
Copy Markdown
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai

coderabbitai Bot commented Apr 15, 2026

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

This PR introduces a targeted DM retry mechanism enabling resend of encrypted messages to specific requesting devices after session collisions. It adds session-deletion tracking via delete_dm_retry_session_target, integrates a new prepare_dm_retry_stanza builder for constructing retry payloads, and includes comprehensive e2e and unit tests validating recovery behavior.

Changes

Cohort / File(s) Summary
DM Retry Handler
src/retry.rs
Added delete_dm_retry_session_target async function that encapsulates base-key collision tracking and targeted session deletion via signal cache. Refactored DM retry resend path to use ensure_e2e_sessions and prepare_dm_retry_stanza instead of send_message_impl. Updated imports and added 2 unit tests validating session deletion for requesting device only and resolved sessions.
Retry Stanza Builder
wacore/src/send.rs
Added public async function prepare_dm_retry_stanza that constructs WA Web DM retry message stanzas with encrypted payloads targeting a single failing recipient device. Includes conditional <device-identity> insertion for prekey messages and 2 unit tests validating single-device targeting and identity inclusion.
E2E Integration Tests
tests/e2e/tests/retry_dm_multidevice.rs
Added test_dm_retry_recovers_after_session_deletion e2e test verifying DM recovery workflow across session deletion, including verification of retry targeting, payload attributes, and continued messaging capability. Included helper functions to inspect retry participant counts and encryption attributes.

Sequence Diagram(s)

sequenceDiagram
    participant Client as Client (retry)
    participant SessionMgr as ensure_e2e_sessions
    participant StanzaBuilder as prepare_dm_retry_stanza
    participant Backend as Backend/Cache
    participant Transport as Transport

    Client->>Backend: Detect session collision<br/>(base-key mismatch)
    Client->>Backend: Peek cached base-key state
    Backend-->>Client: Current base-key info
    
    Client->>SessionMgr: ensure_e2e_sessions<br/>(requesting device only)
    SessionMgr->>Backend: Establish/refresh session<br/>for target device
    Backend-->>SessionMgr: Session ready
    SessionMgr-->>Client: Session confirmed
    
    Client->>StanzaBuilder: prepare_dm_retry_stanza<br/>(message, device_jid,<br/>retry_count)
    StanzaBuilder->>StanzaBuilder: Encrypt for target device
    StanzaBuilder->>StanzaBuilder: Build <participants><to/></to><br/>with enc attributes
    StanzaBuilder->>Backend: Check account identity<br/>(for device-identity)
    Backend-->>StanzaBuilder: Identity (if available)
    StanzaBuilder-->>Client: Retry stanza

    Client->>Backend: Delete stale session
    Backend->>Backend: Clear signal cache<br/>entry for device
    Backend-->>Client: Deletion confirmed

    Client->>Transport: Send retry stanza
    Transport->>Transport: Flush signal cache
    Transport-->>Client: Sent
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related issues

Possibly related PRs

Poem

🐰 A hop, skip, and session deletion bliss,
Retry stanzas dance to the rhythm of JIDs,
One device at a time, we mend the rift—
No more collisions, just a cryptographic gift! ✨

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 80.00% which is sufficient. The required threshold is 80.00%.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title accurately reflects the main change: implementing a targeted single-device DM retry approach that aligns with WhatsApp Web's behavior, replacing the previous broad session cleanup and fanout resend.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch copilot/fix-dm-retry-handler-sessions-again

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/retry.rs`:
- Around line 466-549: The delete_dm_retry_session_target function deletes a
session without acquiring the per-sender session lock; before calling
self.signal_cache.delete_session(&signal_address).await (and before the
subsequent info!("Deleted session...")), acquire the per-sender lock via let
lock = self.session_lock_for(signal_address.as_str()).await; then await
lock.lock() and hold the guard while calling delete_session so session deletion
is serialized with concurrent encrypt/decrypt operations (mirrors the pattern
used in process_retry_key_bundle and the registration ID mismatch path).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 3657bf4d-34cf-4563-be22-9626399f95dd

📥 Commits

Reviewing files that changed from the base of the PR and between 77716a7 and 74b4956.

📒 Files selected for processing (3)
  • src/retry.rs
  • tests/e2e/tests/retry_dm_multidevice.rs
  • wacore/src/send.rs

Comment thread src/retry.rs
Comment on lines +466 to 549
async fn delete_dm_retry_session_target(
&self,
device_jid: &Jid,
message_id: &str,
retry_count: u8,
) -> Result<(), anyhow::Error> {
// Base key collision detection prevents stale-session retry loops.
let signal_address = device_jid.to_protocol_address();
let device_store = self.persistence_manager.get_device_arc().await;

// Check for base key collision before deleting the session.
// Read session through cache for consistent state.
{
let device_guard = device_store.read().await;
let session = self
.signal_cache
.peek_session(&signal_address, &*device_guard.backend)
.await
.ok()
.flatten();

if let Some(session) = session
&& let Ok(current_base_key) = session.alice_base_key()
{
let addr_str = signal_address.as_str();
if retry_count == MIN_RETRY_FOR_BASE_KEY_CHECK {
// Save retry #2's base key so later retries can prove regeneration happened.
if let Err(e) = device_guard
.backend
.save_base_key(addr_str, message_id, current_base_key)
.await
{
warn!("Failed to save base key for {}: {}", signal_address, e);
} else {
info!(
"Saved base key for {} at retry #{} for collision detection",
signal_address, retry_count
);
}
} else if retry_count > MIN_RETRY_FOR_BASE_KEY_CHECK {
// An unchanged base key means we never rebuilt the session.
match device_guard
.backend
.has_same_base_key(addr_str, message_id, current_base_key)
.await
{
Ok(true) => {
// Collision detected! We haven't regenerated our session.
warn!(
"Base key collision detected for {} at retry #{}. \
Session hasn't been regenerated. Forcing fresh session.",
signal_address, retry_count
);
// Clean up base key entry since we're deleting the session
let _ = device_guard
.backend
.delete_base_key(addr_str, message_id)
.await;
}
Ok(false) => {
// Base key changed, session was regenerated - good!
info!(
"Base key changed for {} at retry #{} - session regenerated",
signal_address, retry_count
);
// Clean up old base key entry
let _ = device_guard
.backend
.delete_base_key(addr_str, message_id)
.await;
}
Err(e) => {
warn!("Failed to check base key for {}: {}", signal_address, e);
}
}
}
}
}

// Delete through the cache so resend can't revive a stale in-memory session.
self.signal_cache.delete_session(&signal_address).await;
info!("Deleted session for {signal_address} due to retry receipt");
Ok(())
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Missing session lock before session deletion.

The delete_dm_retry_session_target method reads and deletes the session without acquiring the per-sender session lock. This could race with concurrent decrypt operations that also read/modify session state.

The existing code in process_retry_key_bundle (lines 672-673) and the registration ID mismatch path (lines 265-268) both acquire the session lock before session deletion:

let lock = self.session_lock_for(signal_address.as_str()).await;
let _guard = lock.lock().await;
self.signal_cache.delete_session(&signal_address).await;

Consider adding the same locking pattern here:

Proposed fix
     async fn delete_dm_retry_session_target(
         &self,
         device_jid: &Jid,
         message_id: &str,
         retry_count: u8,
     ) -> Result<(), anyhow::Error> {
         // Base key collision detection prevents stale-session retry loops.
         let signal_address = device_jid.to_protocol_address();
+        let session_mutex = self.session_lock_for(signal_address.as_str()).await;
+        let _session_guard = session_mutex.lock().await;
+
         let device_store = self.persistence_manager.get_device_arc().await;

As per coding guidelines: "Use session_locks to serialize per-sender Signal encrypt/decrypt operations".

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/retry.rs` around lines 466 - 549, The delete_dm_retry_session_target
function deletes a session without acquiring the per-sender session lock; before
calling self.signal_cache.delete_session(&signal_address).await (and before the
subsequent info!("Deleted session...")), acquire the per-sender lock via let
lock = self.session_lock_for(signal_address.as_str()).await; then await
lock.lock() and hold the guard while calling delete_session so session deletion
is serialized with concurrent encrypt/decrypt operations (mirrors the pattern
used in process_retry_key_bundle and the registration ID mismatch path).

@jlucaso1

Copy link
Copy Markdown
Collaborator

Review: Targeted DM Retry (latest commits)

The refactor from full-fanout to targeted single-device retry is the right architectural call and matches WA Web. The prepare_dm_retry_stanza stanza structure, base key collision detection scoping, and unit/e2e test coverage are all solid. A few things need fixing before merge:

1. DM retry dedupe key must include device (Bug -- agrees with Codex P1)

Now that DM retries are per-device, the dedupe key at src/retry.rs:131-144 only uses (chat, msg_id) for DMs. If user B has two devices and both send retry receipts for the same message, the second device's retry is silently dropped by both dedup layers:

  • retried_group_messages cache: "chat:msg_id" already present from first device
  • take_recent_message: message already consumed by first device (DMs don't re-add, unlike groups at line 194)

Fix: include the participant device in the DM dedupe key (same as the group/status path), AND re-add the message to cache after taking for DMs too:

// Dedupe key should always include the participant device:
let dedupe_key = {
    let mut key = String::with_capacity(64);
    receipt.source.chat.push_to(&mut key);
    key.push(':');
    key.push_str(&message_id);
    key.push(':');
    participant_jid.push_to(&mut key);
    key
};

// After take_recent_message, re-add for DMs too (not just groups):
if is_group_or_status || true { // or just always re-add
    self.add_recent_message(&receipt.source.chat, &message_id, &original_msg).await;
}

WA Web doesn't have this problem because getMsgIfAuthorized reads from persistent storage (not a consume-once cache), and the retry queue serializes per-chat via sendMsgQueueMap.enqueue.

2. Missing session lock during DM retry encryption (Bug -- agrees with Codex P2)

The normal DM send path acquires per-device session locks before encryption (send.rs:1137-1142). The new DM retry path calls prepare_dm_retry_stanza without one. Although ensure_e2e_sessions acquires locks internally during process_prekey_bundle, those locks are released before the encryption call. A concurrent decrypt for the same device could advance the ratchet in between.

Fix:

let signal_addr = resolved_jid.to_protocol_address();
let session_mutex = self.session_lock_for(signal_addr.as_str()).await;
let _session_guard = session_mutex.lock().await;

let mut store_adapter = self.signal_adapter().await;
let stanza = wacore::send::prepare_dm_retry_stanza(...).await?;

Note: the group retry path (retry.rs:498-512) has the same pre-existing gap, worth fixing in the same PR.

3. Stanza structure -- verified correct against WA Web

prepare_dm_retry_stanza produces the right structure (verified against WAWebSendMsgCreateDeviceStanza.js g() function):

  • <participants><to jid="device"> wrapping (DM structure) -- not bare <enc> (group structure) ✓
  • No participant or addressing_mode attributes (those are group-only) ✓
  • count="N" on <enc> (retry indicator) ✓
  • <device-identity> conditional on prekey message ✓

4. Assessment of other AI reviews

Codex P1 (dedupe key): Valid -- see #1 above.

Codex P2 (session lock): Valid -- see #2 above.

Claude[bot] review:

  • Missing flush in delete_dm_retry_session_target: Valid observation, but low risk. The deferred flush at the end of the retry path works because ensure_e2e_sessions and signal_adapter both read through the cache. A crash between delete and flush is self-healing (old session reloaded, next retry triggers fresh session). A short comment explaining the intentional deferral would be sufficient.
  • Redundant imports nit: Correct -- use super::* at line 912 already brings in Arc and JidExt from the parent module.
  • E2E test race window: Extremely unlikely given the multiple network round-trips between initial send and retry resend, but registering both waiters before the send would be strictly more robust.

Summary

Fix #1 and #2, the rest is solid. The targeted retry approach is correct and well-tested.

@jlucaso1

Copy link
Copy Markdown
Collaborator

@copilot check my last comment, we need to solve this issues and make sure not regret anymore. Also check if our changes are DRY and allocation less

Agent-Logs-Url: https://github.com/jlucaso1/whatsapp-rust/sessions/ccc77905-54cb-4d7f-be26-824b408f1068

Co-authored-by: jlucaso1 <55464917+jlucaso1@users.noreply.github.com>

Copilot AI commented Apr 15, 2026

Copy link
Copy Markdown
Contributor Author

Review: Targeted DM Retry (latest commits)

The refactor from full-fanout to targeted single-device retry is the right architectural call and matches WA Web. The prepare_dm_retry_stanza stanza st...

Fixed in c9a7585: retry dedupe now includes the requesting device for DMs too, consumed recent messages are re-added so another device can still retry the same DM, and both DM/group targeted retry encryption now take the per-device session lock before stanza preparation. I also reran the focused retry tests plus the full local cargo test -p e2e-tests, and they all passed.

@copilot check my last comment, we need to solve this issues and make sure not regret anymore. Also check if our changes are DRY and allocation less

Handled in c9a7585: I fixed the two regressions from the last review, kept the dedupe-key construction DRY with a shared helper, avoided extra retry-path allocations beyond the single key buffer, and verified the branch with the full local E2E suite (cargo test -p e2e-tests), which passed cleanly.

@jlucaso1

Copy link
Copy Markdown
Collaborator

Review: Latest commit (c9a7585 — dedupe and lock fixes)

Both bugs from my previous review are fixed. Quick verification:

Dedupe key — Fixed

build_retry_dedupe_key now always includes the participant device, unconditionally. DM retries from companion devices get distinct keys. The take_recent_message / add_recent_message is now unconditional too, so the second device can still find the message. Test updated to cover DM per-device dedupe alongside status participants.

Session locks — Fixed

Both group retry (line ~415) and DM retry (line ~441) paths now acquire session_lock_for before encryption, matching the discipline in the normal DM send path (send.rs:1137-1142). Lock scope covers encryption + send_node + flush, which is consistent with the normal send path.

Remaining minor observations (non-blocking)

  1. delete_dm_retry_session_target doesn't hold session lock during deletion — The coderabbit review flagged this. However, the original code didn't hold it either (pre-existing), and the session lock IS acquired later before encryption. The deletion without lock is benign: a concurrent decrypt would either see the old session or not, and we're about to rebuild it anyway.

  2. Redundant test importsuse std::sync::Arc and use wacore::types::jid::JidExt as _ in the test module are redundant with use super::*. Cosmetic only.

LGTM — ready to drop the WIP tag.

@jlucaso1 jlucaso1 changed the title [WIP] Refactor DM retry handler to use targeted single-device resend refactor: targeted single-device DM retry (matches WA Web) Apr 15, 2026

@jlucaso1 jlucaso1 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review: This is now the proper fix

This version is a significant improvement — it aligns with WA Web's actual retry architecture instead of working around it.

What changed from v1

Aspect v1 (broad cleanup) v2 (targeted resend)
Session deletion All known devices Single requesting device
Resend path send_message_impl (full DM fanout) prepare_dm_retry_stanza (single device)
ensure_e2e_sessions All devices (via fanout) [&resolved_jid] only
Dedupe key (DMs) chat:msg_id chat:msg_id:participant
Message re-add Groups/status only Unconditional

Verified against WA Web (WAWebHandleRetryRequest)

WA Web's flow:

// 1. requester = single device
var g = e.isUser() ? r : u;

// 2. Session update for single device  
yield updateLocalSignalSession(e, t);

// 3. Ensure session for single device
yield ensureE2ESessions([g]);

// 4. Targeted resend
yield sendRetry({to, participant, msgRecord, retryCount});

The new PR now mirrors this exactly:

  1. participant_jid = receipt.source.sender.clone() → single device
  2. delete_dm_retry_session_target(&resolved_jid, ...) → single device session
  3. ensure_e2e_sessions(std::slice::from_ref(&resolved_jid)) → single device ✅
  4. prepare_dm_retry_stanza(...) → targeted enc with count attr ✅

Why this fixes the bug

The stale session for device 33 is never used because the targeted resend only encrypts for the resolved device (device 0). The server routes device 0's fresh enc to the companion. No stale PreKeyMessage with consumed prekeys ever reaches the recipient.

Code quality

  • prepare_dm_retry_stanza — Clean, mirrors the existing prepare_group_retry_stanza pattern. Includes count attr and conditional device-identity for pkmsg. ✓
  • build_retry_dedupe_key — Extracted + always includes participant. Fixes a subtle DM bug where two companion devices retrying the same message would get deduplicated. ✓
  • Unconditional message re-add — Correct. DMs can also have multi-device retries (companion 33, 34, etc.). ✓
  • Group retry session lock — Good safety addition, prevents concurrent encrypt races. ✓
  • E2E test inspects the actual sent node (participant count, retry count attr) — much stronger than just checking message delivery. ✓

LGTM — this is the correct fix matching WA Web behavior.

@jlucaso1

Copy link
Copy Markdown
Collaborator

WA Web Compliance Audit — PR #549 (Final)

Systematic comparison of every behavioral aspect against captured WhatsApp Web JavaScript. Each item cites the specific WA Web module and line.


Flow Comparison: WA Web vs PR

Step WA Web PR Match?
1. Max retry check retryCount >= MAX_RETRY (5) → return retry_count > MAX_RETRY_COUNT (5) → return
2. Device validation hasDevice(g, h) → return if not found has_device() → return if not found
3. Queue serialization sendMsgQueueMap.enqueue(chat, ...) pending_retries dedup + session locks ⚠️ Pre-existing
4. Process key bundle processKeyBundle(keyBundle, requester, regId, localRegId, offline) process_retry_key_bundle(node, jid, is_peer)
5. RegId mismatch delete Only if processKeyBundle failed AND localRegId != receivedRegId Same logic (lines 241-271, unchanged)
6. Base key save At retryCount === 2: saveSessionBaseKey At retry_count == 2: save_base_key
7. Base key collision At retryCount > 2: hasSameBaseKeydeleteRemoteSession if collision At retry_count > 2: has_same_base_keydelete_session if collision
8. Session deletion scope Only the requesting device _ Only the requesting device device_jid
9. Ensure sessions ensureE2ESessions([g]) for single device ensure_e2e_sessions(&[resolved_jid])
10. Encrypt target Single device via createUserDeviceMsgStanza Single device via prepare_dm_retry_stanza
11. count attribute Set on <enc> if retryCount > 0, dropped if 0 Always set (but called with count >= 1)
12. Group: sender key forget markForgetSenderKey unconditionally mark_forget_sender_key in group branch ✅ Not changed
13. Group: pairwise encrypt createGroupDeviceMsgStanza targets single participant prepare_group_retry_stanza targets single participant ✅ Not changed

Detailed Findings

✅ Correctly aligned with WA Web (introduced by this PR)

1. Targeted single-device resend

  • WA Web: sendRetry()createUserDeviceMsgStanza(msg, proto, {to: requester, ...}, lidOrigin) encrypts for ONE device (RetryMsgJob.js:54-58)
  • PR: prepare_dm_retry_stanza() encrypts for ONE device with <participants><to jid="device"> structure
  • The old code used send_message_impl (full DM fanout to all devices). This PR fixes the divergence.

2. Session deletion scoped to requesting device

  • WA Web: updateLocalSignalSession operates on _ = participant || from — the single requester (LocalSignalSession.js:19)
  • PR: delete_dm_retry_session_target takes device_jid — the single requester
  • The old code also targeted a single device, this PR preserves that correctly.

3. Per-device retry dedup

  • WA Web: No explicit dedup — message lookup from persistent DB + queue serialization naturally prevents issues (RetryRequest.js:196)
  • PR: build_retry_dedupe_key includes participant device for both DMs and groups, so multi-device retries are independent

4. Base key collision detection

  • WA Web: var y = 2; if (p === y) saveSessionBaseKey; if (p > y) hasSameBaseKey → deleteRemoteSession (LocalSignalSession.js:66-79)
  • PR: MIN_RETRY_FOR_BASE_KEY_CHECK = 2, same save/check/delete logic
  • Exact match.

5. Session locks before encryption

  • WA Web: sendMsgQueueMap.enqueue serializes all sends per-chat (RetryRequest.js:196)
  • PR: session_lock_for acquired before both group and DM retry encryption
  • Different mechanism, same protection goal.

⚠️ Pre-existing differences (NOT introduced by this PR)

1. Unconditional session deletion (pre-existing)

  • WA Web: Deletes session only on regId mismatch OR base key collision (LocalSignalSession.js:51-79)
  • PR (and original code): Always deletes session unconditionally after base key checks
  • Impact: More aggressive session reset than WA Web. Self-healing (burns one prekey per retry) but not strictly matching. Was present before this PR.

2. Message lookup from transient cache vs persistent DB (pre-existing)

  • WA Web: getMessageTable().get(msgKey) — reads from persistent IndexedDB (RetryKeyBundle.js:24-35)
  • PR: take_recent_message — reads from in-memory LRU cache
  • Impact: Messages evicted from cache can't be retried. The PR improves this by unconditional re-add (was group-only before), but the fundamental transient-vs-persistent gap remains.

3. Missing decrypt-fail attribute on retry stanzas (pre-existing)

  • WA Web: Sets decrypt-fail attribute on <enc> via decryptFailAttributeFromProtobuf(proto) (MsgCreateDeviceStanza.js:155-157)
  • PR: Neither prepare_dm_retry_stanza nor prepare_group_retry_stanza sets decrypt-fail
  • Impact: Low — mainly affects infrastructure messages (reactions, pins) which rarely trigger retries. The normal send path (encrypt_for_devices) handles it correctly.

4. Stanza wrapping: <participants> vs bare <enc> (pre-existing, documented)

  • WA Web: Single-device DM sends may use bare <enc> without <participants> wrapper (WAWebSendMsgCreateFanoutStanza)
  • PR: Always uses <participants><to jid="..."> wrapping
  • Impact: None — server accepts both forms. Documented in wacore/src/send.rs:751-755.

5. Bot message secret (pre-existing)

  • WA Web: Generates botMessageSecret from messageSecret for bot messages (RetryMsgJob.js:23-26)
  • PR: Not handled
  • Impact: Only affects CAPI bot accounts.

Review Checklist

Check Status Notes
✅ MAX_RETRY matches 5 in both PostMessageHighRetryCountMetric.js:5
✅ hasDevice early return Both return on unknown device RetryRequest.js:161-195
✅ Base key threshold 2 in both LocalSignalSession.js:66
✅ Single-device session deletion Matches WA Web requester-only scope LocalSignalSession.js:19
✅ Single-device resend Matches WA Web targeted retry RetryMsgJob.js:54-58
count attribute on <enc> Set on retries, matches MsgCreateDeviceStanza.js:153
device-identity conditional Included on prekey messages Both stanza builders
✅ No participant/addressing_mode on DM Correctly omitted (group-only) MsgCreateDeviceStanza.js
✅ Group retry unchanged Pairwise encrypt to single participant Not modified by PR
✅ ensureE2ESessions for single device Called before encryption RetryRequest.js:200, MsgCreateDeviceStanza.js:20
⚠️ Unconditional session deletion Pre-existing, more aggressive than WA Web LocalSignalSession.js:51-79
⚠️ decrypt-fail missing on retry stanzas Pre-existing omission MsgCreateDeviceStanza.js:155

Verdict

The PR moves the codebase significantly closer to WA Web compliance. The main change (full-fanout → targeted single-device retry) eliminates a major behavioral divergence. All pre-existing gaps (unconditional session deletion, transient message cache, missing decrypt-fail) were present before and are not introduced by this PR.

No new WA Web compliance issues found.

@jlucaso1
jlucaso1 merged commit ce33442 into main Apr 15, 2026
8 of 9 checks passed
@jlucaso1
jlucaso1 deleted the copilot/fix-dm-retry-handler-sessions-again branch April 15, 2026 02:12
@jlucaso1

Copy link
Copy Markdown
Collaborator

Remaining issue: stale device 33 session on subsequent normal sends

Tested with the bartender mock server (updated to ce334426). The targeted retry resend works correctly — B receives the retried message via MockPhone relay of device 0's fresh enc.

However, subsequent normal DMs after the retry still fail because the stale device 33 session was never cleaned up:

# Retry resend works (targeted, device 0 only):
[INFO] Resending message 3EB0... to 559980000002@s.whatsapp.net (retry #1)
Message participants: ["to[jid=559980000002@s.whatsapp.net]"]  ← single device ✅
MockPhone relayed message to companion ✅
decrypted PreKey message from 100000012345678@lid.0 with one-time prekey 13 ✅

# But Phase 3 normal DM still fails (full fanout):
Message participants: ["to[100000024691356@lid]", "to[100000024691356:33@lid]", "to[559980000001@s.whatsapp.net]"]
                                                        ^^^^^^^^^^^^^^^^^^^^^^ stale session!
[ERROR] Message from 100000012345678@lid.0 failed to decrypt; message counter 2
    No current session

The issue is that delete_dm_retry_session_target only deletes the session for the resolved JID (device 0). The ensure_e2e_sessions in the normal send path finds the stale device 33 session (has_session = true) and reuses it, producing a PreKeyMessage with a consumed prekey.

Suggestion

The v1 approach (delete sessions for ALL known devices) would fix both the retry AND subsequent sends. Or alternatively, after the targeted retry resend succeeds, also clean up sessions for sibling devices of the same user.

For now, the bartender mock server works around this by always preferring MockPhone relay (device 0 enc → companion) over direct device 33 routing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: DM retry handler should delete sessions for all known devices, not just device 0

2 participants