Skip to content

fix(channels/whatsapp): restore Apr-2026 protocol parity via whatsapp-rust 0.6 + namespace revert (#6246) - #6706

Merged
Audacity88 merged 15 commits into
zeroclaw-labs:masterfrom
Micra-io:feat/wa-rs-revert-rebrand-to-upstream
May 22, 2026
Merged

fix(channels/whatsapp): restore Apr-2026 protocol parity via whatsapp-rust 0.6 + namespace revert (#6246)#6706
Audacity88 merged 15 commits into
zeroclaw-labs:masterfrom
Micra-io:feat/wa-rs-revert-rebrand-to-upstream

Conversation

@alexandme

@alexandme alexandme commented May 16, 2026

Copy link
Copy Markdown
Contributor

Summary

Validation Evidence (required)

cargo fmt --all -- --check:

(no output — clean)

cargo clippy --all-targets --features whatsapp-web -- -D warnings:

error: this `if` can be collapsed into the outer `match`
   --> crates/zeroclaw-providers/src/openai_codex.rs:781:25
    |
781 | /                         if state.fallback_text.is_none() {
782 | |                             state.fallback_text = response_output_text_from_event_item(item);
783 | |                         }
    | |_________________________^
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.95.0/index.html#collapsible_match
    = note: `-D clippy::collapsible-match` implied by `-D warnings`
    = help: to override `-D warnings` add `#[allow(clippy::collapsible_match)]`
help: collapse nested if block
    |
780 ~                     Some("message") if !state.saw_text_delta
781 ~                         && state.fallback_text.is_none() => {
782 |                             state.fallback_text = response_output_text_from_event_item(item);
783 ~                         }
    |

error: could not compile `zeroclaw-providers` (lib) due to 1 previous error

The single clippy error is pre-existing on master, not introduced by this PR. I confirmed via git checkout upstream/master -- crates/zeroclaw-providers/src/openai_codex.rs and re-ran clippy: same error, same line, same source. Introduced in #6117 (commit 6f96122ab, 2026-05-11) when rust-1.95 tightened clippy::collapsible_match. This PR does not touch the file (git diff a090cbce3..HEAD -- crates/zeroclaw-providers/src/openai_codex.rs is empty).

cargo test --features whatsapp-web -p zeroclaw-channels --locked (the crate this PR modifies):

test wati::tests::wati_parse_timestamp_seconds ... ok
test wati::tests::wati_parse_valid_message ... ok
test wati::tests::wati_try_transcribe_returns_none_when_manager_none ... ok
test qq::tests::test_get_token_refreshes_expired_cache ... ok

test result: ok. 1044 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 8.91s

   Doc-tests zeroclaw_channels

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

cargo build --release --features whatsapp-web:

    Finished `release` profile [optimized] target(s) in 4m 02s

Binary size 16,486,704 bytes (was 15,654,384 on PR #55's wa-rs-aliased build, +830 KB). bytes and prost became direct deps, and the lockfile gained cipher 0.5, aes 0.9, ctr 0.10, hkdf 0.13, cbc, and block-padding from wacore's transitive graph.

  • Commands run and tail output: all four commands from the template, above.

  • Beyond CI — what did you manually verify?

    • Deployed the branch binary to a live daemon (macOS, Tailscale-connected) 2026-05-16 11:08 UTC. All three channel supervisor tasks registered cleanly (daemon_state.json: all status: "ok"). Config loaded with Unknown config key ignored warnings for fork-local keys (mention_only, allowed_groups, mention_name), which is correct lenient-deserialization behavior on upstream master. Rolled back to the prior binary immediately; post-rollback state matches pre-deploy.
    • chore(deps): sync wa-rs* fork to oxidezap HEAD (0.5→0.6, 12 commits) Micra-io/zeroclaw#55 (admin-squash-merged 2026-05-16) validated the wire-protocol roundtrip on the same wacore 0.6 source: pair handshake survives zeroclaw service restart, inbound and outbound roundtrip on a live WhatsApp account, zeroclaw channel doctor flips ❌ → ✅. The protocol code is identical between that PR and this one; only the namespace names and lockfile differ.
  • Branch-specific smoke evidence (2026-05-17, in-place swap on the production daemon):

    Built locally from worktree at HEAD 54e82a0be (the git-rev pin commit) with cargo build --release --features whatsapp-web. Binary SHA256: 9b48ba677ad38478223e8f8c5acf09aad1efb222fea5dcbea5bc65fe416e9f28, 16,503,296 bytes. Swapped in over the production binary (old SHA bd9bc8d166beceaac1fdcfdd121e003735a0481795121fc200bf91a8de188a16 preserved at ~/.cargo/bin/zeroclaw.bak-pre6706).

    Note: pre-deploy production was on wa-rs* 0.2 with WhatsApp broken since the 2026-04-24 protocol break (the original bug from [Bug]: WhatsApp Web channel: pair succeeds but messages don't flow after April 2026 server-side protocol bump #6246). The smoke test verifies the new build fixes it.

    Step Result Timestamp (UTC)
    T0: stop service, back up old binary ✅ clean stop 2026-05-17T22:17:37Z
    T1: start service with new binary ✅ all components status: ok in daemon_state.json 2026-05-17T22:19:30Z
    Pair from scratch (Linked Devices on test phone) ✅ scanned QR, session DB populated ~22:25Z
    Inbound message → bot ✅ message received, dispatched to agent loop 22:25:32Z
    Outbound message → user ✅ bot reply delivered to phone 22:25:36Z
    zeroclaw service stop + service start (restart test) ✅ session survives, no fresh QR code emitted after restart (grepped stderr from T_start onward → 0 hits) T_stop 22:29:21Z, T_start 22:29:24Z
    Post-restart inbound + outbound ✅ "hows it going?" received, bot replied 22:32:07Z → 22:32:32Z

    channel doctor reports ❌ WhatsApp unhealthy (auth/config/network) on both runs (before and after restart), despite the bot demonstrably exchanging messages with a live WhatsApp account during the same window. This is a stale-check side-finding — the doctor's WA Web probe doesn't match the new wacore 0.6 connection state. Worth a follow-up in zeroclaw-channels's channel doctor impl, but unrelated to the wire protocol or this PR's correctness.

    Old session DB (paired before 2026-04-24) was not loadable by the new client — the daemon emitted fresh QR codes until paired. This is the same upgrade path consumers of the wa-rs*whatsapp-rust jump should expect.

  • Skipped commands: full-workspace cargo test. The storage and channel changes are confined to zeroclaw-channels; the clean release-build link covered cross-crate behavior.

Security & Privacy Impact (required)

  • New permissions, capabilities, or file system access scope? No. Same SQLite file (~/.zeroclaw/whatsapp-session.db) with two additive tables and one additive column.
  • New external network calls? No. The wire protocol is the same; only the crate names change. No new hosts contacted.
  • Secrets / tokens / credentials handling changed? No. Pairing identity and Signal session state stay in the same SQLite store. Trait methods that handle them now return bytes::Bytes where they used to return Vec<u8>. This is a Rust-side ergonomic shift, not a storage-format change.
  • PII, real identities, or personal data in diff, tests, fixtures, or docs? No. Tests use synthetic device IDs and JIDs. No real phone numbers, push names, or session blobs in the diff.

Compatibility (required)

  • Backward compatible? No. The dep-bump commit carries a ! marking the breaking change.
  • Config / env / CLI surface changed? No. No new keys, no removed keys, no renamed flags.
  • Upgrade steps for existing users:
    1. Pull this PR's master and run cargo update -p whatsapp-rust -p wacore -p waproto. Or just cargo build --release --features whatsapp-web to let Cargo regenerate the lockfile from scratch.
    2. Restart the daemon. The schema migrations apply on next startup via the existing RusqliteStore::ensure_schema path: additive tables sender_key_devices and sent_messages, additive column device_registry.raw_id. No manual SQL required.
    3. If your daemon was paired before the 2026-04-24 protocol bump, the new client picks up the existing session in whatsapp-session.db. If it was paired during the broken window (between 2026-04-24 and this fix), re-pair from WhatsApp → Linked Devices.

Rollback (required for risk: medium and risk: high)

  • Fast rollback command/path: git revert <merge-sha> followed by cargo build --release --features whatsapp-web puts the build back on wa-rs* 0.2. A daemon binary swap reverses the change for an existing install. The schema additions are non-destructive: the new tables and column stay in the DB, unused by the older client.
  • Feature flags or config toggles: None. The whatsapp-web cargo feature is the single gate. Building without it produces a daemon with no WhatsApp Web channel, matching the pre-fix behavior when the feature was off.
  • Observable failure symptoms:

sent_messages retention

The new sent_messages table backs WA Web's retry flow: when we send a message, the serialized payload is stored keyed by (chat_jid, message_id, device_id) so a retry-receipt from the recipient can trigger a re-encrypt + resend without re-rendering the message.

Retention parameters (wacore 0.6 defaults, unchanged):

  • TTL: sent_message_ttl_secs = 300 (5 minutes) at whatsapp-rust/src/cache_config.rs:259.
  • Cleanup cadence: the keepalive loop calls delete_expired_sent_messages(cutoff) every ~12 keepalive ticks (≈ 5 minutes wall-clock) at whatsapp-rust/src/keepalive.rs:151-163.
  • Cutoff: now_secs() - sent_message_ttl_secs.

ZeroClaw SQLite-side implementation:

  • Schema at crates/zeroclaw-channels/src/whatsapp_storage.rs:287-294: (chat_jid, message_id, device_id) primary key, payload BLOB, created_at INTEGER.
  • Cleanup query at crates/zeroclaw-channels/src/whatsapp_storage.rs:1244-1253: DELETE FROM sent_messages WHERE created_at < ?1 AND device_id = ?2.
  • Supporting index idx_sent_messages_device_created ON sent_messages(device_id, created_at) was added by commit a633dc5ea on this branch specifically so the cleanup query stays O(log n). Without it the periodic DELETE would full-scan the table.

Growth bound: at the default 300s TTL and a sustained 100 msgs/sec, the table holds ~30K rows × payload size (typically 200B–2KB serialized), so single-digit MB worst case. Keepalive-driven cleanup keeps the row count bounded; the index keeps the cleanup itself cheap.

Failure modes:

  • If the keepalive loop stops, cleanup stops and the table grows linearly with send rate. The whatsapp_web channel restart paths in daemon re-spawn the keepalive task on reconnect.
  • SQLITE_BUSY during cleanup just retries on the next ~5min tick. Entries are short-lived by design, so no durability impact.

No new config keys: sent_message_ttl_secs comes from CacheConfig's default. Surfacing it on the ZeroClaw [channels.whatsapp] block is out of scope here.


Refs: #6246, Micra-io#52/#53/#55, oxidezap/whatsapp-rust#487/#586/#597/#613/#621/#624/#627/#629/#632.

alexandme added 6 commits May 16, 2026 11:52
Bumps WhatsApp Web crate constraints from `wa-rs*` 0.2 to the
upstream-published `whatsapp-rust`/`wacore`/`waproto` family at
0.6 (oxidezap/whatsapp-rust, on crates.io 2026-05-11). Restores
protocol parity after the WhatsApp Web server-side change
~2026-04-24 documented in zeroclaw-labs#6246.

Mechanical name map (per discussion on zeroclaw-labs#6246):

  wa-rs                 -> whatsapp-rust
  wa-rs-core            -> wacore
  wa-rs-binary          -> wacore-binary
  wa-rs-proto           -> waproto
  wa-rs-tokio-transport -> whatsapp-rust-tokio-transport
  wa-rs-ureq-http       -> whatsapp-rust-ureq-http-client

Source imports renamed in the two channel adapters
(`whatsapp_web.rs`, `whatsapp_storage.rs`); textual doc-comment
mentions left as-is for a cosmetic follow-up.

The `tokio-runtime` feature is enabled on `whatsapp-rust` to
satisfy the 4-slot BotBuilder typestate added in
oxidezap/whatsapp-rust#586.

Build is intentionally broken at this commit -- the next commit
patches `whatsapp_web.rs` and `whatsapp_storage.rs` for the
breaking deltas surfaced by the 0.2 -> 0.5 jump (~109 compile
errors expected: Arc<Event> wrapping from zeroclaw-labs#613, BotBuilder
typestate from zeroclaw-labs#586, removed methods, fixed-size crypto arrays,
SendResult typed return, StoreError::Database boxed errors).

Refs:
- zeroclaw-labs#6246 (root issue)
- oxidezap/whatsapp-rust#487, zeroclaw-labs#586, zeroclaw-labs#597, zeroclaw-labs#601, zeroclaw-labs#613 (breaking changes)
- #52, #55 (validated downstream on the rebrand fork)
The 0.2 → 0.6 jump across multiple oxidezap/whatsapp-rust release
cycles broke the WhatsApp Web channel adapter. This commit rebuilds
the web-side adapter against the new trait/type surface (storage
adapter follows in the next commit).

- BotBuilder gained a 4th typestate slot (with_runtime). Use
  `whatsapp_rust::TokioRuntime` to satisfy it; the `tokio-runtime`
  feature was already enabled on the optional dep in the prior commit.
- `with_device_props` rewritten to take a `DevicePropsOverride`
  builder (oxidezap/whatsapp-rust#586) instead of three positional
  Option args.
- Event handlers now receive `Arc<Event>` (PR zeroclaw-labs#613); switched the
  match to `match &*event` so variant-data binding still works.
- `Client::get_phone_number_from_lid` was removed in favor of the
  unified `Client::get_lid_pn_entry(&Jid) -> Result<Option<LidPnEntry>>`
  (zeroclaw-labs#487); call site rewritten to extract `entry.phone_number`, error
  branch swallowed back to None to preserve best-effort enrichment
  semantics.
- `Client::upload` gained a third `UploadOptions` arg; pass
  `UploadOptions::default()` for legacy behavior.
- `UploadResponse` cryptographic fields are now `[u8; 32]` (was
  Vec<u8>); use the `*_vec()` accessors to convert before populating
  protobuf message fields.
- `Client::send_message` now returns `SendResult { message_id, to }`
  rather than a bare String (PR zeroclaw-labs#597); log via `.message_id`.
- `Bot::run()` returns `BotHandle` (Future + abort handle wrapper);
  field type updated from `tokio::task::JoinHandle<()>` accordingly.

Refs: zeroclaw-labs#6246, oxidezap/whatsapp-rust#487, zeroclaw-labs#586, zeroclaw-labs#597, zeroclaw-labs#613.
Validated downstream on #52 + #55.
The whatsapp-rust 0.6 adapt commit (e8e34c3) dropped Box::pin
on the text send path while keeping it on the voice and media
paths. The send_message future is large (~34KB) on all three
sites; dropping the wrap at one site risks tripping
clippy::large_futures once the storage adapter compiles and
clippy can run.

Restore Box::pin parity with the voice/media call sites and
the reference fork commit (40aee96).
Rebuilds the RusqliteStore backend against the new wacore 0.6
trait/type surface so the WhatsApp Web feature compiles cleanly.

- `to_store_err!` macro now wraps errors as
  `Box<dyn Error + Send + Sync>` to match the new
  `StoreError::Database(Box<dyn Error + Send + Sync>)` variant
  (was `StoreError::Database(String)`).
- `SignalStore::load_identity` returns `Option<[u8; 32]>` (was
  `Option<Vec<u8>>`); validates length and copies into a fixed
  array.
- `SignalStore::get_session` and `SignalStore::load_prekey`
  return `Option<bytes::Bytes>`; added `bytes` as a
  feature-gated direct dep and convert via `Bytes::from(vec)`.
- Added `SignalStore::get_max_prekey_id` (queries MAX(id), 0 on
  empty) and `AppSyncStore::get_latest_sync_key_id`
  (most-recent key by key_id desc) — both new in wacore 0.6.
- ProtocolStore's per-device sender-key tracking replaces the
  legacy SKDM model. Removed `get_skdm_recipients` /
  `add_skdm_recipients` / `clear_skdm_recipients` /
  `mark_forget_sender_key` / `consume_forget_marks` (no longer
  trait members) and added `get_sender_key_devices`,
  `set_sender_key_status`, `clear_sender_key_devices`,
  `delete_sender_key_device_rows`, `clear_all_sender_key_devices`.
- Added `ProtocolStore::delete_devices` (force re-fetch on next
  query).
- Added the sent-message retry trio: `store_sent_message`,
  plus getter/cleaner — new persistence surface from wacore 0.6.
- DeviceListRecord raw_id round-trip — `Some(u32) <-> i64`
  conversion preserved (catches regressions where a wrong cast
  would falsely invalidate live sessions).
- Schema migrations for the new sender-key-devices and
  sent-messages tables; backward-compatible additive ALTERs.
- Fixed the transitive \`prost::Message\` import that lost its
  re-export under wacore 0.6 by widening the \`prost\` direct
  dep's feature gate to include \`whatsapp-web\` (previously
  gated only to \`channel-lark\`).

Refs: zeroclaw-labs#6246, oxidezap/whatsapp-rust#487, zeroclaw-labs#586, zeroclaw-labs#597, zeroclaw-labs#613.
Validated downstream on #52 + #55.
clippy::needless_borrow tightened in rust-1.93 and now flags two
sites in whatsapp_web.rs where we passed &T into a fn already
expecting &T (so the compiler immediately auto-derefs):

- extract_mentioned_jids takes &Message, we passed &msg where msg
  is already &Message via the Arc<Event>-wrapped event arm
- render_pairing_qr takes &str, we passed &code where code is
  already &String via the same event pattern

Validated downstream on #55.
@github-actions github-actions Bot added dependencies Auto scope: dependency manifest/lock/policy changed. channel Auto scope: src/channels/** changed. channel:whatsapp Auto module: channel/whatsapp changed. labels May 16, 2026
alexandme added 3 commits May 16, 2026 14:34
The per-entry INSERT...ON CONFLICT loop in set_sender_key_status could
leave partial state if a panic or connection drop interrupted the batch:
some (group, device) pairs flipped to has_key=true and others not. That
silently breaks SKDM resend logic, since the device-list cache no longer
matches reality.

Mirror the immediate-transaction pattern already used in take_sent_message
so the whole batch either commits or rolls back together.
`delete_expired_sent_messages` filters on (device_id, created_at) but the
table only has a primary-key index on (chat_jid, message_id, device_id),
so the cleanup pass full-scans the table. With the cleanup cron not yet
wired, `sent_messages` grows unbounded and the scan cost rises with it.

Add a covering composite index so the cleanup remains cheap once it is
scheduled. `IF NOT EXISTS` keeps schema-init idempotent across restarts.
`init_schema` previously ran the CREATE TABLE batch and the conditional
`ALTER TABLE device_registry ADD COLUMN raw_id` as separate statements
on a raw connection. A crash or SIGKILL between them (or a transient
`database is locked` from a launchd start-up race) left the DB with the
new tables but no `raw_id` column. On next boot the PRAGMA probe sees
the column missing and re-runs the ALTER — which works in the happy
path, but if it failed permanently the partial-migration state stuck.

Move the PRAGMA probe ahead of the transaction (it's read-only, and on
a fresh DB it correctly observes no table → skip ALTER), then open a
transaction around the CREATEs, new sent_messages index, and the
conditional ALTER. The whole migration now commits or rolls back as a
unit.
@alexandme

alexandme commented May 16, 2026

Copy link
Copy Markdown
Contributor Author

Fixes shipped in the follow-up

  • 577aaa757 — wrap set_sender_key_status loop in a transaction
  • a633dc5ea — add idx_sent_messages_device_created index on (device_id, created_at)
  • e80b71bb4 — make schema init atomic across migrations

Tests green (1044 passed, 0 failed).

Deferred to follow-up issues

Out of scope for a wa-rs version bump. Items 1–2 will be filed as separate issues; 3–7 are for a future cleanup pass.

  1. Wire delete_expired_sent_messages to the daemon cron with a 24–72h TTL. The method exists, nothing schedules it. Without it sent_messages grows monotonically with plaintext outbound payloads (~50 KB–500 KB/day at typical bot scale). Privacy and disk-growth concern.
  2. Make the SQLite storage layer truly async via tokio::task::spawn_blocking or migrate to tokio-rusqlite. Every trait method is async fn but the body is parking_lot::Mutex::lock() plus blocking rusqlite on the runtime thread. Latency trigger: offline-backlog drain (fix(offline): drive WA Web pull-batch loop for offline backlog oxidezap/whatsapp-rust#629 now drains the full queue on reconnect) on a bot also running Slack/Telegram.
  3. Migrate take_sent_message to DELETE ... RETURNING to eliminate the SELECT+DELETE double-evaluation. Correct today under BEGIN IMMEDIATE, more robust as a single statement.
  4. Override load_prekeys_batch with a WHERE id IN (?,?,...) query instead of the default-impl loop that takes N mutex acquisitions for N prekey IDs.
  5. Drop legacy skdm_recipients and sender_key_status tables in a future release once wacore 0.6 has soaked. No read path, retained for now.
  6. Defensive u32::try_from on the raw_id read path instead of as u32. Lossless under all valid persistence paths, more self-documenting.
  7. Doc-comment textual wa-rs mentions in whatsapp_web.rs, whatsapp_storage.rs, and crates/zeroclaw-config/src/schema.rs. Cosmetic, deferred for review velocity.

Note

Pre-existing on master, not introduced by this PR:

  • crates/zeroclaw-providers/src/openai_codex.rs:781 trips clippy::collapsible_match under rust-1.95 (the lint tightened in feat(codex): support native Responses tool calls #6117). Doesn't block this PR, one-line follow-up.
  • The use prost::Message; import in whatsapp_storage.rs:27 is now properly feature-gated by whatsapp-web (added in the dep-bump commit).

@alexandme

Copy link
Copy Markdown
Contributor Author

CI status — need toolchain guidance

Quality Gate on e80b71bb4: Lint ❌, CI Required Gate ❌. Test, Build, Check, Security, Benchmarks skipped because Lint failed first. Path Labeler is the only success.

Root cause

wacore 0.6.0 (new dep this PR pulls from crates.io) uses an if let guard at wacore-0.6.0/src/history_sync.rs:93. That feature stabilized in Rust 1.94 (rust-lang/rust#51114). Upstream CI pins toolchain: 1.93.0 in .github/workflows/ci.yml, so clippy fails with error[E0658]: 'if let' guards are experimental.

Same constraint the fork hit onboarding wacore 0.5 in Micra-io#52. The fix there was patching wacore/src/history_sync.rs inside a [patch.crates-io]-pinned fork to lift the guard. This PR drops that patch because oxidezap published 0.6 to crates.io, but the published tarball still ships the unstabilized feature.

Validate PR title also reports startup_failure with no jobs scheduled. Looks like a fork-PR permissions issue at the workflow-startup level, separate from this PR.

Three paths to green

  1. Bump CI rustc to 1.94.0 in this PR. Update .github/workflows/ci.yml and the workspace rust-version (1.871.94). 1.94 is the rustc that stabilized if_let_guard, so it's the minimum that compiles wacore 0.6.0. Smallest delta, aligns the project with the dep.
  2. Re-introduce [patch.crates-io] pointing at Micra-io/whatsapp-rust@8b6d64f (same SHA chore(deps): sync wa-rs* fork to oxidezap HEAD (0.5→0.6, 12 commits) Micra-io/zeroclaw#55 ships). Keeps CI on 1.93, loses the "drop the shim" win, matches what's running on Micra-io's production fork today.
  3. Pin to a wacore version without the feature. Not viable on its own: 0.5.0 (last published before the 2026-04-24 protocol bump) lacks the fix this PR exists to deliver. Needs a separate upstream patch to oxidezap.

Path 1 is my preference. Path 2 is the safe fallback if the MSRV bump is unwelcome. Say the word and I'll push the fix.

Pinging @theonlyhennygod @singlerider @JordanTheJet since this touches the toolchain pin and the channel:whatsapp work you've been tracking.

@singlerider

singlerider commented May 16, 2026

Copy link
Copy Markdown
Contributor

@alexandme updating CI versions in this PR is a no-go.

You may have more luck running fmt with the +nightly option.

@Audacity88

Copy link
Copy Markdown
Collaborator

@alexandme I checked the live PR state, the failed lint job, #6246, and @singlerider's toolchain comment. I agree with @singlerider that bumping ZeroClaw's CI/MSRV in this PR would make the scope too broad.

For this PR, please keep ZeroClaw on the current CI/toolchain contract and use a compatible dependency path instead: either a patched/pinned wacore source that compiles under the pinned toolchain, or an upstream wacore release that removes the if let guard before we depend on it. Once CI can run past lint, the main things I want to check are branch-specific WhatsApp smoke evidence for this exact branch (pair/restart, inbound, outbound, channel doctor) and the retention story for the new sent_messages retry table.

@alexandme

Copy link
Copy Markdown
Contributor Author

Read both comments, won't touch CI here.

Plan applying the pinned wacore source option you mentioned:

  1. Open an upstream PR against oxidezap/whatsapp-rust rewriting the if let guard at wacore/src/history_sync.rs:93.
  2. Pin whatsapp-rust / wacore / waproto to that upstream commit via a { git = "...", rev = "..." } source, so the workspace builds under the pinned 1.93 toolchain.
  3. When the upstream MSRV fix releases, swap the git refs back to versioned crates.io deps in a follow-up commit.

Will ping for re-review once CI Lint is green with the evidence and retention details.

The published wacore 0.6.0 crate uses an `if let` guard at
`wacore/src/history_sync.rs:93` that requires Rust 1.94+; ZeroClaw
CI pins 1.93.0, so the workspace fails to compile against the
crates.io release. Temporarily pinning the six WA crates
(whatsapp-rust, wacore, wacore-binary, waproto,
whatsapp-rust-ureq-http-client, whatsapp-rust-tokio-transport) to
upstream commit 9734fb2 — the head of oxidezap/whatsapp-rust#632
which rewrites the guard for 1.93 compatibility.

This is a plain `git` source dep, not a [patch.crates-io] shim:
package names already match upstream, so the workspace deps just
point at the upstream repo. CI Lint expected to flip green here.

A follow-up commit on this PR will swap these git refs back to
versioned crates.io deps once upstream publishes the fix as 0.6.1.

Refs: oxidezap/whatsapp-rust#632, zeroclaw-labs#6246.
@alexandme

Copy link
Copy Markdown
Contributor Author

Ready for re-review.

  • CI Lint: passing — whatsapp-rust / wacore / waproto and the two transports are interim-pinned to fix(wacore): drop if-let guard so the pushname arm builds on Rust 1.93 oxidezap/whatsapp-rust#632's fix commit (9734fb2) via a plain git source. Not a [patch.crates-io] shim. ZeroClaw CI/MSRV pin unchanged.
  • Branch-specific WhatsApp smoke evidence (pair / restart / inbound / outbound) in the Validation section. Live-verified on the production daemon via in-place binary swap.
  • sent_messages retention story added: 300s TTL, keepalive-driven cleanup every ~5min, indexed cleanup query.

Will swap the git refs back to versioned crates.io deps in a follow-up commit on this PR once upstream releases the fix (likely 0.6.1).

@Audacity88 Audacity88 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alexandme I reviewed current head 54e82a0be2ea23fb765467edfa57bff265eaccf8 against #6246, the current PR body, the green CI run, the changed WhatsApp storage/web adapter code, and the pinned whatsapp-rust 0.6 storage contract. I agree with the direction and the MSRV-compatible temporary git pin, but I think one storage migration gap still blocks this from safely restoring WhatsApp Web sessions.

🟢 What looks good — The protocol dependency path and cleanup story are much closer now

The move to the upstream 0.6 crate family looks aligned with the April protocol break, and the temporary 9734fb2 pin is a reasonable way to avoid bundling a Rust/MSRV bump into this PR. I also checked the sent-message retention path: upstream’s keepalive loop does call delete_expired_sent_messages on a periodic cadence when the default TTL is nonzero, so the added sent_messages(device_id, created_at) index is the right shape for that cleanup path.

🔴 Blocking — Persist the full 0.6 device state before relying on existing sessions

The custom RusqliteStore schema/save/load path in this PR still does not add or round-trip the new 0.6 Device fields. In the pinned 0.6 upstream contract, several additional fields are persistent device state, including next_pre_key_id, server_has_prekeys, nct_salt, server_cert_chain, and login_counter. Upstream has explicit sqlite migrations for at least next_pre_key_id and login_counter, and treats the rest as persistent Device fields rather than runtime-only data.

That means an existing ZeroClaw WhatsApp session can connect, save updated 0.6 device state, then lose those fields on the next daemon restart. The loss is not just cosmetic: next_pre_key_id is the monotonic pre-key counter, server_cert_chain enables cached Noise IK reconnects, and login_counter is explicitly sent on login as persisted anti-abuse state. This undercuts the PR’s upgrade/session-reuse claim and leaves restart behavior different from the upstream 0.6 storage backend.

Please add the missing device columns/migrations to the custom schema, persist them in DeviceStore::save, reload them in DeviceStore::load, and add at least one restart-style round-trip test that proves those 0.6 fields survive through RusqliteStore. Once that is in place, I think the remaining review can focus on final smoke evidence and the branch refresh.

alexandme added 2 commits May 19, 2026 11:47
Round-trip the 5 new Device fields introduced in whatsapp-rust 0.6
(next_pre_key_id, server_has_prekeys, nct_salt, server_cert_chain,
login_counter) through RusqliteStore. Without this, daemon restart
silently resets the monotonic pre-key counter and anti-abuse login
counter, and drops the cached Noise IK server cert chain — undermining
session reuse claimed by the upstream-mirror PR.

server_cert_chain serializes as JSON inside a BLOB column for schema
tolerance. Column types and defaults mirror upstream's sqlite-storage
migrations so a future cross-store migration tool is trivial.

Addresses CHANGES_REQUESTED on zeroclaw-labs#6706.
Mirrors the existing needs_raw_id pragma-probe pattern for the 5 new
device columns added in the previous commit. Probe runs before the
transaction; conditional ALTER TABLE ... ADD COLUMN statements run
inside the same transaction as the CREATEs so a crash between them
rolls back cleanly. Existing on-disk WhatsApp session databases
(including ~/.zeroclaw/whatsapp-session.db on the deployed daemon)
upgrade in-place on the next RusqliteStore::new() call.
@alexandme

Copy link
Copy Markdown
Contributor Author

@Audacity88,

Verified at pin 9734fb2: all 5 fields you flagged are marked persistent in wacore::store::Device with #[serde(default)], not #[serde(skip)]. Pushed two commits.

90d18599 adds the 5 columns to CREATE TABLE device and wires DeviceStore::save and load. server_cert_chain round-trips as serde_json::to_vec inside a nullable BLOB. Column types and defaults match your sqlite-storage migrations 1:1.

5a62df7c adds idempotent ALTER TABLE migrations for pre-0.6 databases. It mirrors the needs_raw_id pragma-probe pattern in crates/zeroclaw-channels/src/whatsapp_storage.rs (used for device_registry). All five ALTERs run inside the existing init transaction, so a mid-init crash rolls back cleanly.

Two new tests:

  • device_save_load_round_trips_wacore_06_fields: save, drop the store, reopen, load. Asserts all 5 fields survived with non-default values.
  • pre_06_device_table_gets_new_columns_on_open: hand-creates a legacy 18-column device table, opens the store, runs save/load, reopens to confirm the migration is a no-op the second time.

Also smoke-tested on a deployed daemon with a real pre-0.6 session DB. Behavior matched the spec: next_pre_key_id recovered from the 0 sentinel to MAX(prekeys.id)+1 = 12181 via src/prekeys.rs:115. server_cert_chain populated to 366 bytes after one XX fallback per src/handshake.rs:129. login_counter incremented through two normal logins. Channels stayed healthy throughout, no re-pair required.

server_cert_chain uses serde_json rather than your bincode path. Picked JSON for schema tolerance and on-disk debuggability. Can switch to bincode if you want byte-parity for migration tooling.

@Audacity88 Audacity88 added the needs-author-action Author response needed before review or merge can continue; not a stale warning. label May 20, 2026
@Audacity88

Copy link
Copy Markdown
Collaborator

@alexandme Thanks for the detailed follow-up here. I saw the new device-state persistence commits and the extra pre-0.6 migration/test evidence.

Before I do the next re-review pass, could you refresh this branch against current master? GitHub is currently reporting the PR as conflicted/dirty.

Refresh PR zeroclaw-labs#6706 against current master per reviewer request
(zeroclaw-labs#6706 comment by @Audacity88).

Conflicts resolved in crates/zeroclaw-channels/src/whatsapp_web.rs:

- recipient_to_jid JID parse: kept master's structured zeroclaw_log
  Reject event on parse failure, with the wacore_binary::jid::Jid
  type from this branch.
- send() text path: kept this branch's Box::pin + SendResult.message_id
  for the whatsapp-rust 0.6 send_message return shape, with master's
  zeroclaw_log::record! DEBUG event (replacing the tracing::debug! call).
- listen() prelude: combined this branch's whatsapp-rust / wacore /
  wacore_binary / whatsapp_rust_tokio_transport / whatsapp_rust_ureq_http_client
  imports with master's Arc<alias> capture (used downstream by the
  per-message attribution sites added in zeroclaw-labs#6398).
- PairingQrCode event arm: kept master's structured zeroclaw_log::record!
  for the QR-received notice; kept this branch's `render_pairing_qr(code)`
  call site (whatsapp-rust 0.6's `match &*event` already binds `code` as
  a reference, so no explicit `&` is needed).
- Took master's removal of the dead-code WIP attachment scaffold
  (normalize_incoming_content / send_wa_attachment / WaAttachmentKind /
  WaAttachment / parse_attachment_markers / mime_from_path /
  wa_media_type / find_matching_close). These were all
  `#[allow(dead_code)] // WIP: not yet wired into send path` and were
  deliberately dropped in zeroclaw-labs#6398; preserving them would expand this PR's
  scope. No callers outside the WIP block.

Validation:
- cargo check -p zeroclaw-channels --features whatsapp-web: clean.
- cargo test  -p zeroclaw-channels --features whatsapp-web --locked:
  1128 passed, 0 failed (plus 4 in proof_orchestrator_session_context).
- cargo fmt --all: clean.
@alexandme

Copy link
Copy Markdown
Contributor Author

@Audacity88 Refreshed against master at 2ef4e97cb.

Merge (not rebase) per @singlerider's note on #6009. Single conflict file: crates/zeroclaw-channels/src/whatsapp_web.rs, 6 regions; Cargo.toml / Cargo.lock auto-merged.

Resolution choices:

  • JID parse (recipient_to_jid), send() debug log, PairingQrCode info log → kept master's structured zeroclaw_log::record! macros, with this branch's whatsapp-rust 0.6 types (wacore_binary::jid::Jid, Box::pin(client.send_message(...)).await? returning SendResult.message_id, code ref derived from match &*event).
  • listen() prelude → this branch's wacore / wacore_binary / whatsapp_rust* imports, plus master's let alias = std::sync::Arc::new(self.alias.clone()) capture (used downstream at the per-message attribution sites added in feat!: multi-agent runtime and schema V3 #6398; without it the merged tree fails to compile).
  • Dropped the dead-code WIP attachment scaffold that master removed in feat!: multi-agent runtime and schema V3 #6398: normalize_incoming_content, send_wa_attachment, WaAttachment / WaAttachmentKind, parse_attachment_markers, mime_from_path, wa_media_type, find_matching_close. All were #[allow(dead_code)] // WIP: not yet wired into send path; verified no callers outside the WIP block (rg in crates/). Preserving them on the merged tree would expand this PR's scope into surface area feat!: multi-agent runtime and schema V3 #6398 deliberately retired.

Validation on the merge commit:

$ cargo check -p zeroclaw-channels --features whatsapp-web
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 00s

$ cargo test -p zeroclaw-channels --features whatsapp-web --locked
test result: ok. 1128 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
test result: ok. 4 passed; 0 failed; 0 ignored  (proof_orchestrator_session_context)

$ cargo fmt --all -- --check
(clean)

Ready for the next re-review pass.

@Audacity88

Copy link
Copy Markdown
Collaborator

@alexandme Thanks for the update here. I checked the current live PR state, and GitHub is still reporting the branch as conflicting/dirty against master, so I cannot clear my prior changes-requested review yet.

Could you refresh against current master one more time? Once the branch is clean, I will re-review the device-state persistence fix, the migration tests, and the latest WhatsApp smoke/retention evidence.

Refresh against current master (d05c8a9) to clear PR zeroclaw-labs#6706 dirty state.

Single conflict: crates/zeroclaw-channels/Cargo.toml — the `whatsapp-web`
feature definition. Upstream still references the `wa-rs*` rebrand names;
this PR carries the canonical `whatsapp-rust` / `wacore` / `waproto` /
`whatsapp-rust-*` transports plus `dep:bytes` (required by wacore 0.6
storage traits for `Bytes` return types in SignalStore::get_session /
load_prekey). Resolved in favor of HEAD.

Cargo.lock auto-merged cleanly with only the canonical crate entries.
@alexandme

Copy link
Copy Markdown
Contributor Author

@Audacity88 Refreshed against current master at d05c8a9b0 → merge commit f6a36b774.
Resolved in favor of HEAD. Cargo.lock auto-merged cleanly.

Validation on the merge commit:

$ cargo check -p zeroclaw-channels --features whatsapp-web
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 34.39s

$ cargo fmt --all -- --check
(clean)

$ cargo test -p zeroclaw-channels --features whatsapp-web --locked
test result: ok. 1133 passed; 0 failed; 0 ignored
test result: ok. 4 passed; 0 failed; 0 ignored  (proof_orchestrator_session_context)

mergeable: MERGEABLE Ready for the next re-review pass.

@Audacity88 Audacity88 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alexandme I re-reviewed current head f6a36b7 against #6246, my earlier changes-requested review, the latest refresh comments, the current WhatsApp storage/web diff, the green CI run, and GitHub's current BEHIND mergeability state. The blocker I raised on the 0.6 device-state migration is addressed.

✅ Resolved — wacore 0.6 device state now survives restart

The current head adds the five missing Device fields to the fresh device schema and the pre-0.6 migration path, writes them in DeviceStore::save, reloads them in DeviceStore::load, and covers both a restart-style save/load round trip and a legacy 18-column table migration/reopen path. That closes my concern that an upgraded WhatsApp Web session could connect once and then lose persisted pre-key, server-cert, salt, or login-counter state on the next daemon restart.

🟢 What looks good — the refresh kept the dependency recovery scoped

The later master-refresh commits kept the PR centered on the WhatsApp Web protocol recovery: the 0.6 crate-family pin remains explicit and MSRV-compatible, the current branch drops the WIP attachment scaffold that master already removed, and the merge conflict resolution preserves the structured logging and alias attribution changes from master while carrying the new whatsapp-rust / wacore type surface. CI is green on the refreshed head, and the latest author validation comment covers the current merge commit.

No blocking findings from me on this reviewed head. Before merge, this still needs the normal branch-refresh/mergeability pass because GitHub currently reports the PR as BEHIND, and I’ll clean up the stale maintainer labels separately.

@Audacity88 Audacity88 added risk: medium and removed needs-author-action Author response needed before review or merge can continue; not a stale warning. labels May 22, 2026
@Audacity88
Audacity88 merged commit fa51898 into zeroclaw-labs:master May 22, 2026
12 checks passed
github-actions Bot pushed a commit that referenced this pull request May 23, 2026
…whatsapp-rust 0.6 + namespace revert (#6246) (#6706)

- 8c25871 chore(deps)!: bump wa-rs* family to upstream whatsapp-rust 0.6
- e8e34c3 fix(channels/whatsapp): adapt whatsapp_web.rs to whatsapp-rust 0.6
- 0b82469 fix(channels/whatsapp): Box::pin send_message text path for parity
- 6dc949d fix(channels/whatsapp): adapt whatsapp_storage.rs to wacore 0.6
- 6e1e164 fix(channels/whatsapp): drop needless borrows surfaced by rust-1.93
- f4406d3 style: apply cargo fmt to whatsapp adapters
- 577aaa7 fix(channels/whatsapp): wrap set_sender_key_status loop in a transaction
- a633dc5 perf(channels/whatsapp): index sent_messages(device_id, created_at)
- e80b71b fix(channels/whatsapp): make schema init atomic across migrations
- 54e82a0 chore(deps): pin whatsapp-rust family to upstream fix commit 9734fb2
- 90d1859 fix(channels/whatsapp): persist wacore 0.6 device fields
- 5a62df7 fix(channels/whatsapp): migrate pre-0.6 device tables to add 0.6 columns
- 2ef4e97 Merge upstream/master into feat/wa-rs-revert-rebrand-to-upstream
- f6a36b7 Merge upstream/master into feat/wa-rs-revert-rebrand-to-upstream
- 3de8f9c Merge branch 'master' into feat/wa-rs-revert-rebrand-to-upstream fa51898
SUqerBrave pushed a commit to SUqerBrave/zeroclaw that referenced this pull request May 24, 2026
…-rust 0.6 + namespace revert (zeroclaw-labs#6246) (zeroclaw-labs#6706)

- 8c25871 chore(deps)!: bump wa-rs* family to upstream whatsapp-rust 0.6
- e8e34c3 fix(channels/whatsapp): adapt whatsapp_web.rs to whatsapp-rust 0.6
- 0b82469 fix(channels/whatsapp): Box::pin send_message text path for parity
- 6dc949d fix(channels/whatsapp): adapt whatsapp_storage.rs to wacore 0.6
- 6e1e164 fix(channels/whatsapp): drop needless borrows surfaced by rust-1.93
- f4406d3 style: apply cargo fmt to whatsapp adapters
- 577aaa7 fix(channels/whatsapp): wrap set_sender_key_status loop in a transaction
- a633dc5 perf(channels/whatsapp): index sent_messages(device_id, created_at)
- e80b71b fix(channels/whatsapp): make schema init atomic across migrations
- 54e82a0 chore(deps): pin whatsapp-rust family to upstream fix commit 9734fb2
- 90d1859 fix(channels/whatsapp): persist wacore 0.6 device fields
- 5a62df7 fix(channels/whatsapp): migrate pre-0.6 device tables to add 0.6 columns
- 2ef4e97 Merge upstream/master into feat/wa-rs-revert-rebrand-to-upstream
- f6a36b7 Merge upstream/master into feat/wa-rs-revert-rebrand-to-upstream
- 3de8f9c Merge branch 'master' into feat/wa-rs-revert-rebrand-to-upstream
SUqerBrave pushed a commit to SUqerBrave/zeroclaw that referenced this pull request May 27, 2026
…-rust 0.6 + namespace revert (zeroclaw-labs#6246) (zeroclaw-labs#6706)

- 8c25871 chore(deps)!: bump wa-rs* family to upstream whatsapp-rust 0.6
- e8e34c3 fix(channels/whatsapp): adapt whatsapp_web.rs to whatsapp-rust 0.6
- 0b82469 fix(channels/whatsapp): Box::pin send_message text path for parity
- 6dc949d fix(channels/whatsapp): adapt whatsapp_storage.rs to wacore 0.6
- 6e1e164 fix(channels/whatsapp): drop needless borrows surfaced by rust-1.93
- f4406d3 style: apply cargo fmt to whatsapp adapters
- 577aaa7 fix(channels/whatsapp): wrap set_sender_key_status loop in a transaction
- a633dc5 perf(channels/whatsapp): index sent_messages(device_id, created_at)
- e80b71b fix(channels/whatsapp): make schema init atomic across migrations
- 54e82a0 chore(deps): pin whatsapp-rust family to upstream fix commit 9734fb2
- 90d1859 fix(channels/whatsapp): persist wacore 0.6 device fields
- 5a62df7 fix(channels/whatsapp): migrate pre-0.6 device tables to add 0.6 columns
- 2ef4e97 Merge upstream/master into feat/wa-rs-revert-rebrand-to-upstream
- f6a36b7 Merge upstream/master into feat/wa-rs-revert-rebrand-to-upstream
- 3de8f9c Merge branch 'master' into feat/wa-rs-revert-rebrand-to-upstream
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

channel:whatsapp Auto module: channel/whatsapp changed. channel Auto scope: src/channels/** changed. dependencies Auto scope: dependency manifest/lock/policy changed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: WhatsApp Web channel: pair succeeds but messages don't flow after April 2026 server-side protocol bump

3 participants