diff --git a/advanced/signal-protocol.mdx b/advanced/signal-protocol.mdx index 85a1b61..aa9bb71 100644 --- a/advanced/signal-protocol.mdx +++ b/advanced/signal-protocol.mdx @@ -642,7 +642,7 @@ pub struct PreparedGroupStanza { - **Group path:** After `send_node()` succeeds, the caller uses the `skdm_devices` list from `PreparedGroupStanza` to call `set_sender_key_status(group, devices, true)`. No re-resolution needed. - **Status path:** A late-init boolean tracks whether full distribution occurred. The sender key tracking is only updated after the status stanza is successfully sent. - **Error recovery:** If `prepare_group_stanza` fails with `NoSenderKeyState`, all sender key device tracking for that group is cleared and the send is retried with full distribution. -- **Sender key rotation:** On `rotateKey`, the Signal sender key is also deleted for forward secrecy (matching WhatsApp Web's `deleteGroupSenderKeyInfo`), and all device tracking is cleared via `clear_sender_key_devices`. +- **Sender key rotation:** On `rotateKey`, the Signal sender key is also deleted for forward secrecy (matching WhatsApp Web's `deleteGroupSenderKeyInfo`), and all device tracking is cleared via `reset_sender_key_device_tracking` — a DB-first clear with a cold-mark fallback (see below). - **Group `` notification (number/LID migration):** A `w:gp2` `` notification (a participant's number or LID changed) unconditionally force-rotates the own group sender key and invalidates both the persisted and in-memory group metadata cache, matching WhatsApp Web's `modifyParticipantInfo` (`rotateKey: true`). The next send regenerates and redistributes a fresh sender key against the current participant list instead of risking a stale entry for the migrated device. See `Client::force_rotate_own_sender_key`, `src/handlers/notification/groups.rs`. **Incremental targeting:** @@ -697,6 +697,22 @@ As of [#996](https://github.com/oxidezap/whatsapp-rust/pull/996), a device whose [#999](https://github.com/oxidezap/whatsapp-rust/pull/999) closes this residual by excluding own devices from the warm mark too (`exclude_own_devices=true`), mirroring WhatsApp Web's `ParticipantStore` helper, which guards *both* `markHasSenderKey` and `markForgetSenderKey` with the same `!isMeDevice` check. Own companions are therefore never memoized as `has_key=true` — `filter_skdm_targets` (["Per-device sender key tracking → Incremental targeting"](#per-device-sender-key-tracking) above) always re-includes them, so they get a fresh SKDM on every group send. This is a deliberate trade-off (a few extra pairwise SKDM nodes per send when the account has companions) in exchange for making the orphan impossible. External devices are unaffected: a successful distribution still marks them warm, and the retry-receipt path still repairs any that go stale. + +**The group distribution lane now guards the full audit-reset-redistribute sequence, not just the SKDM fan-out ([#1043](https://github.com/oxidezap/whatsapp-rust/pull/1043)).** Previously `Client::group_distribution_lock()` (see ["Parallelized group encrypt fan-out"](#parallelized-group-encrypt-fan-out) above) was taken only around the cold SKDM send itself. Sender-key deletion (participant-removal rotation, forced own-key rotation), per-device tracker resets, and the status-broadcast distribution path could run concurrently with that lock held elsewhere, letting an encrypt racing a rotation restore a retired key after deletion, or a tracker reset race stale delivery marks back onto a new chain. + +`rotate_sender_key_on_participant_remove`, `force_rotate_own_sender_key` (now taking `&Jid` instead of a pre-stringified group ID), warm group sends, status sends, phash-mismatch recovery, and periodic sender-key rotation all now hold the same per-group lane across their own-key delete/reset and the following redistribution. A rotation that arrives while a send is mid-fan-out waits for the lane instead of deleting the chain state out from under it; a send that arrives mid-rotation waits for the rotation to finish before re-auditing device state. Held lanes are never capacity-evicted, so a live rotation or fan-out cannot be silently dropped from the map mid-operation (see `group_distribution_locks_capacity` in the [Cache Configuration reference](/api/bot#cache-configuration-reference)). + + + +**Sender-key tracker resets are DB-first ([#1043](https://github.com/oxidezap/whatsapp-rust/pull/1043)).** `reset_sender_key_device_tracking` replaces the old direct `clear_sender_key_devices` + cache-invalidate call at every rotation and redistribution site. It clears the per-device tracking row-by-row in the database first, and only invalidates the in-memory `SenderKeyDeviceCache` after that durable clear succeeds. If the DB clear fails, every existing tracked row is instead marked cold (`has_key=false`) as a fallback so the next send still re-distributes; if that fallback write also fails, the operation returns an error and the send stays fail-closed rather than risking a stale `has_key=true` row surviving onto a freshly rotated chain. + +The unknown-participant rotation in [retry receipt handling](/advanced/retry-admission) is a special case: `handle_retry_receipt` deletes the own sender key and resets tracking for a `` from an unrecognized group participant, then must still fall through to the per-chat resend rate limiter and other throttles further down the same function. The signal cache is now explicitly flushed right after the rotation — before any later throttle can return early — so a rotation is never left un-persisted by an unrelated early exit later in the same call. + + + +**Observability: distribution-lane pressure is exposed on `memory_report()` ([#1043](https://github.com/oxidezap/whatsapp-rust/pull/1043)).** `Client::memory_report()` now reports `group_distribution_locks` (live lane count), `group_distribution_lock_evictions` (cumulative cold evictions), and `group_distribution_lock_eviction_blocks` (cumulative evictions skipped because the lane was live) — see [`memory_report()`](/api/client#memory_report). These update only under capacity pressure and add no allocation or per-message cost below the soft cap. + + ### In-memory sender key device cache The `SenderKeyDeviceCache` provides an in-memory caching layer over the per-device sender key tracking data stored in the database. Without this cache, every group send would require a database round-trip to load the sender key device map — the cache eliminates that overhead after the first load for each group. diff --git a/api/bot.mdx b/api/bot.mdx index 9eb4c88..0c33ff6 100644 --- a/api/bot.mdx +++ b/api/bot.mdx @@ -1081,7 +1081,11 @@ The `recent_messages` cache is disabled by default (capacity 0), meaning sent me |---------|---------|-------------| | `session_locks_capacity` | 10,000 | Per-device Signal session lock capacity. Soft cap: a lock a task is actively holding is never evicted, so the map can briefly exceed this under heavy concurrent fan-out (bounded by the number of concurrently-held locks) rather than evicting a live lock and letting two writers race the same session | | `chat_lanes_capacity` | 5,000 | Per-chat lane capacity (combined enqueue lock + message queue) | -| `group_distribution_locks_capacity` | 512 | Per-group cold sender-key distribution lock capacity. Single-flights a group's cold SKDM fan-out: a concurrent send that races the winner's redistribution re-resolves against its warm marking instead of redistributing to every member again | +| `group_distribution_locks_capacity` | 512 | Per-group sender-key distribution lane capacity. Soft cap: a live lane (held by an in-flight SKDM fan-out, rotation, or tracker reset) is never evicted, so the map can briefly exceed this under concurrent fan-out rather than breaking tracker ordering | + + +The lane's live entry count plus cumulative capacity evictions and blocked evictions are exposed via [`Client::memory_report()`](/api/client#memory_report) (`group_distribution_locks`, `group_distribution_lock_evictions`, `group_distribution_lock_eviction_blocks`), so operators can derive eviction rates and tune this capacity without guessing. + #### Sent message DB cleanup diff --git a/api/client.mdx b/api/client.mdx index 2c9dc63..d10de12 100644 --- a/api/client.mdx +++ b/api/client.mdx @@ -1951,6 +1951,9 @@ Entry counts plus estimated retained heap bytes for the client's internal collec | `pdo_requested` | `u64` | PDO once-per-message memo cache (count only) | | `session_locks` | `u64` | Per-device session locks (count only) | | `chat_lanes` | `u64` | Per-chat lanes — combined enqueue lock + message queue (count only) | +| `group_distribution_locks` | `u64` | Live per-group sender-key distribution lanes (count only) | +| `group_distribution_lock_evictions` | `u64` | Cumulative capacity evictions of cold (non-live) distribution lanes; poll successive reports to derive a rate | +| `group_distribution_lock_eviction_blocks` | `u64` | Cumulative attempts that kept a live lane and temporarily exceeded `group_distribution_locks_capacity` instead of evicting it | | `resend_rate_limiter_chats` | `u64` | Chats tracked by the per-chat resend rate limiter (count only) | | `response_waiters` | `usize` | Active IQ response waiters | | `node_waiters` | `usize` | Active node waiters |