whatsapp-rust-docs/api/store.mdx at eadd8efcf5dc07d48e57ffd38fcc1431ddd3745c · oxidezap/whatsapp-rust-docs

title	Storage Traits
description	Storage backend traits and SQLite implementation for persistent state

Overview

whatsapp-rust uses a trait-based storage system to persist device state, cryptographic keys, and protocol metadata. The storage layer is split into four domain-specific traits:

SignalStore - Signal protocol cryptographic operations (identity keys, sessions, pre-keys, sender keys)
AppSyncStore - WhatsApp app state synchronization (sync keys, versions, mutation MACs)
ProtocolStore - WhatsApp protocol alignment (SKDM tracking, LID-PN mapping, device registry)
DeviceStore - Device persistence operations

All four traits are combined into the Backend trait for convenience.

The Backend Trait

Any type implementing all four domain traits automatically implements Backend:

pub trait Backend: SignalStore + AppSyncStore + ProtocolStore + DeviceStore + Send + Sync {}

impl<T> Backend for T 
where 
    T: SignalStore + AppSyncStore + ProtocolStore + DeviceStore + Send + Sync
{}

SignalStore Trait

Handles Signal protocol cryptographic storage for end-to-end encryption.

Identity Operations

/// Store an identity key for a remote address
async fn put_identity(&self, address: &str, key: [u8; 32]) -> Result<()>;

/// Load an identity key for a remote address
async fn load_identity(&self, address: &str) -> Result<Option<Vec<u8>>>;

/// Delete an identity key
async fn delete_identity(&self, address: &str) -> Result<()>;

Session Operations

/// Get an encrypted session for an address
async fn get_session(&self, address: &str) -> Result<Option<Vec<u8>>>;

/// Store an encrypted session
async fn put_session(&self, address: &str, session: &[u8]) -> Result<()>;

/// Delete a session
async fn delete_session(&self, address: &str) -> Result<()>;

/// Check if a session exists (default implementation uses get_session)
async fn has_session(&self, address: &str) -> Result<bool>;

PreKey Operations

/// Store a pre-key
async fn store_prekey(&self, id: u32, record: &[u8], uploaded: bool) -> Result<()>;

/// Store multiple pre-keys in a single batch operation (default loops over store_prekey)
async fn store_prekeys_batch(&self, keys: &[(u32, Vec<u8>)], uploaded: bool) -> Result<()>;

/// Load a pre-key by ID
async fn load_prekey(&self, id: u32) -> Result<Option<Vec<u8>>>;

/// Load multiple pre-keys by ID in a single batch operation (default loops over load_prekey)
async fn load_prekeys_batch(&self, ids: &[u32]) -> Result<Vec<(u32, Vec<u8>)>>;

/// Remove a pre-key
async fn remove_prekey(&self, id: u32) -> Result<()>;

/// Get the highest pre-key ID currently stored
async fn get_max_prekey_id(&self) -> Result<u32>;

Signed PreKey Operations

/// Store a signed pre-key
async fn store_signed_prekey(&self, id: u32, record: &[u8]) -> Result<()>;

/// Load a signed pre-key by ID
async fn load_signed_prekey(&self, id: u32) -> Result<Option<Vec<u8>>>;

/// Load all signed pre-keys (returns id, record pairs)
async fn load_all_signed_prekeys(&self) -> Result<Vec<(u32, Vec<u8>)>>;

/// Remove a signed pre-key
async fn remove_signed_prekey(&self, id: u32) -> Result<()>;

Sender Key Operations

For group messaging encryption:

/// Store a sender key for group messaging
async fn put_sender_key(&self, address: &str, record: &[u8]) -> Result<()>;

/// Get a sender key
async fn get_sender_key(&self, address: &str) -> Result<Option<Vec<u8>>>;

/// Delete a sender key
async fn delete_sender_key(&self, address: &str) -> Result<()>;

AppSyncStore Trait

Handles WhatsApp app state synchronization storage.

Sync Key Operations

/// Get an app state sync key by ID
async fn get_sync_key(&self, key_id: &[u8]) -> Result<Option<AppStateSyncKey>>;

/// Set an app state sync key
async fn set_sync_key(&self, key_id: &[u8], key: AppStateSyncKey) -> Result<()>;

/// Get the latest sync key ID
async fn get_latest_sync_key_id(&self) -> Result<Option<Vec<u8>>>;

Version Tracking

/// Get the app state version for a collection
async fn get_version(&self, name: &str) -> Result<HashState>;

/// Set the app state version for a collection
async fn set_version(&self, name: &str, state: HashState) -> Result<()>;

Mutation MAC Operations

/// Store mutation MACs for a version
async fn put_mutation_macs(
    &self,
    name: &str,
    version: u64,
    mutations: &[AppStateMutationMAC],
) -> Result<()>;

/// Get a mutation MAC by index
async fn get_mutation_mac(&self, name: &str, index_mac: &[u8]) -> Result<Option<Vec<u8>>>;

/// Delete mutation MACs by their index MACs
async fn delete_mutation_macs(&self, name: &str, index_macs: &[Vec<u8>]) -> Result<()>;

ProtocolStore Trait

Handles WhatsApp protocol alignment and tracking.

Per-device sender key tracking

Tracks sender key distribution status per device in groups, matching WhatsApp Web's participant.senderKey Map<deviceJid, boolean> model. Each device has a boolean indicating whether it holds a valid sender key (true) or needs a fresh SKDM (false).

/// Get sender key distribution status for all known devices in a group.
/// Returns (device_jid_string, has_key) pairs.
async fn get_sender_key_devices(&self, group_jid: &str) -> Result<Vec<(String, bool)>>;

/// Set sender key status for devices. Use has_key=true after successful
/// SKDM distribution (WA Web: markHasSenderKey), or has_key=false to mark
/// devices as needing fresh SKDM (WA Web: markForgetSenderKey).
async fn set_sender_key_status(&self, group_jid: &str, entries: &[(&str, bool)]) -> Result<()>;

/// Clear all sender key device tracking for a group (on sender key rotation).
async fn clear_sender_key_devices(&self, group_jid: &str) -> Result<()>;

/// Delete specific `sender_key_devices` rows by device JID across all groups.
/// Mirrors WhatsApp Web's per-group `senderKey.delete(deviceJid)` cleanup.
/// Called when a device is removed from a user's registry so a future re-add
/// of the same device id cannot hit a stale `has_key=true` entry and skip SKDM.
async fn delete_sender_key_device_rows(&self, device_jids: &[&str]) -> Result<()>;

LID-PN Mapping

Manages mappings between LID (Locally Indexed Device) and phone numbers:

/// Get a mapping by LID
async fn get_lid_mapping(&self, lid: &str) -> Result<Option<LidPnMappingEntry>>;

/// Get a mapping by phone number (returns the most recent LID)
async fn get_pn_mapping(&self, phone: &str) -> Result<Option<LidPnMappingEntry>>;

/// Store or update a LID-PN mapping
async fn put_lid_mapping(&self, entry: &LidPnMappingEntry) -> Result<()>;

/// Get all LID-PN mappings (for cache warm-up)
async fn get_all_lid_mappings(&self) -> Result<Vec<LidPnMappingEntry>>;

Base Key Collision Detection

/// Save the base key for a session address during retry collision detection
async fn save_base_key(&self, address: &str, message_id: &str, base_key: &[u8]) -> Result<()>;

/// Check if the current session has the same base key as the saved one
async fn has_same_base_key(
    &self,
    address: &str,
    message_id: &str,
    current_base_key: &[u8],
) -> Result<bool>;

/// Delete a base key entry
async fn delete_base_key(&self, address: &str, message_id: &str) -> Result<()>;

Device Registry

/// Update the device list for a user (called after usync responses)
async fn update_device_list(&self, record: DeviceListRecord) -> Result<()>;

/// Get all known devices for a user
async fn get_devices(&self, user: &str) -> Result<Option<DeviceListRecord>>;

TcToken Storage

Trusted contact privacy tokens:

/// Get a trusted contact token for a JID (stored under LID)
async fn get_tc_token(&self, jid: &str) -> Result<Option<TcTokenEntry>>;

/// Store or update a trusted contact token for a JID
async fn put_tc_token(&self, jid: &str, entry: &TcTokenEntry) -> Result<()>;

/// Delete a trusted contact token for a JID
async fn delete_tc_token(&self, jid: &str) -> Result<()>;

/// Get all JIDs that have stored tc tokens
async fn get_all_tc_token_jids(&self) -> Result<Vec<String>>;

/// Delete tc tokens with token_timestamp older than cutoff (returns count deleted)
async fn delete_expired_tc_tokens(&self, cutoff_timestamp: i64) -> Result<u32>;

Sent message store

Persists sent message payloads for retry handling. Matches WhatsApp Web's getMessageTable pattern where retry receipts look up the original message from storage.

/// Store a sent message's serialized payload for retry handling.
/// Called after each send_message(); the payload is the protobuf-encoded Message.
async fn store_sent_message(
    &self,
    chat_jid: &str,
    message_id: &str,
    payload: &[u8],
) -> Result<()>;

/// Retrieve and delete a sent message (atomic take). Returns serialized payload.
/// Called when a retry receipt arrives; consuming prevents double-retry.
async fn take_sent_message(
    &self,
    chat_jid: &str,
    message_id: &str,
) -> Result<Option<Vec<u8>>>;

/// Delete sent messages older than cutoff (unix timestamp seconds).
/// Returns count deleted.
async fn delete_expired_sent_messages(
    &self,
    cutoff_timestamp: i64,
) -> Result<u32>;

The `take_sent_message` method is an atomic read-and-delete operation. Once a message payload is taken for retry, it is removed from storage to prevent double-retry. For status broadcasts where multiple devices may retry, the client re-adds the message after taking it.

DeviceStore Trait

Handles device data persistence:

/// Save device data
async fn save(&self, device: &Device) -> Result<()>;

/// Load device data
async fn load(&self) -> Result<Option<Device>>;

/// Check if a device exists
async fn exists(&self) -> Result<bool>;

/// Create a new device row and return its generated device_id
async fn create(&self) -> Result<i32>;

/// Create a snapshot of the database state
/// Optional: label with name, save extra_content (e.g. failing message)
async fn snapshot_db(&self, name: &str, extra_content: Option<&[u8]>) -> Result<()>;

SqliteStore implementation

The default storage implementation using SQLite with Diesel ORM. SQLite is bundled by default — you don't need it installed on your system.

Bundled SQLite

The whatsapp-rust-sqlite-storage crate enables the bundled-sqlite feature by default, which compiles SQLite from source and statically links it. To use a system-installed SQLite instead:

[dependencies]
whatsapp-rust-sqlite-storage = { version = "0.5", default-features = false }

Creating a store

use whatsapp_rust::store::SqliteStore;

// Basic usage - creates/opens database at path
let store = SqliteStore::new("whatsapp.db").await?;

// With device_id for multi-device support
let store = SqliteStore::new_for_device("whatsapp.db", 1).await?;

// Using sqlite:// URL format
let store = SqliteStore::new("sqlite://path/to/db.sqlite").await?;

Features

Connection pooling - Uses Diesel r2d2 with pool size of 2
WAL mode - Write-Ahead Logging for better concurrency
Automatic migrations - Runs embedded migrations on startup
Semaphore-based locking - Prevents concurrent writes
Retry logic - Automatic retry with exponential backoff for locked database
Multi-device support - Single database can store multiple device sessions

Database Configuration

SqliteStore automatically configures connections with:

PRAGMA journal_mode = WAL;
PRAGMA busy_timeout = 30000;
PRAGMA synchronous = NORMAL;
PRAGMA cache_size = 512;
PRAGMA temp_store = memory;
PRAGMA foreign_keys = ON;

Usage Example

use whatsapp_rust_sqlite_storage::SqliteStore;
use whatsapp_rust::Bot;
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create store
    let backend = Arc::new(SqliteStore::new("whatsapp.db").await?);
    
    // Use with Bot builder
    let mut bot = Bot::builder()
        .with_backend(backend.clone())
        .with_transport_factory(transport_factory)
        .with_http_client(http_client)
        .with_runtime(TokioRuntime)
        .on_event(|event, client| async move { /* handle events */ })
        .build()
        .await?;
    
    // Store implements all traits
    // SignalStore
    backend.put_identity("address@s.whatsapp.net", [0u8; 32]).await?;
    let identity = backend.load_identity("address@s.whatsapp.net").await?;
    
    // AppSyncStore
    let version = backend.get_version("regular").await?;
    
    // ProtocolStore
    let devices = backend.get_sender_key_devices("group@g.us").await?;
    
    // DeviceStore
    if backend.exists().await? {
        let device = backend.load().await?;
    }
    
    Ok(())
}

CacheStore Trait

The CacheStore trait enables pluggable cache backends for the client's data caches. By default, caches use in-process moka; implementing this trait lets you use Redis, Memcached, or any other external cache.

Location: wacore/src/store/cache.rs

#[async_trait]
pub trait CacheStore: Send + Sync + 'static {
    /// Retrieve a cached value by namespace and key.
    async fn get(&self, namespace: &str, key: &str) -> anyhow::Result<Option<Vec<u8>>>;

    /// Store a value with an optional TTL.
    /// When `ttl` is `None`, the entry persists until explicitly deleted.
    async fn set(
        &self,
        namespace: &str,
        key: &str,
        value: &[u8],
        ttl: Option<Duration>,
    ) -> anyhow::Result<()>;

    /// Delete a single key from the given namespace.
    async fn delete(&self, namespace: &str, key: &str) -> anyhow::Result<()>;

    /// Delete all keys in a namespace.
    async fn clear(&self, namespace: &str) -> anyhow::Result<()>;

    /// Approximate entry count (diagnostics only). Default returns 0.
    async fn entry_count(&self, _namespace: &str) -> anyhow::Result<u64> {
        Ok(0)
    }
}

Namespaces

Each logical cache uses a unique namespace string. Implementations should partition keys by namespace (e.g., prefix as {namespace}:{key} in Redis).

Namespace	Cache	Description
`"group"`	`group_cache`	Group metadata
`"device_registry"`	`device_registry_cache`	Device registry entries
`"lid_pn_by_lid"`	`lid_pn_cache`	LID-to-phone bidirectional mappings

Error handling

Cache operations are best-effort. The client treats read failures as cache misses and logs warnings on write failures. Implementations should still return errors for observability.

CacheStores configuration

pub struct CacheStores {
    pub group_cache: Option<Arc<dyn CacheStore>>,
    pub device_registry_cache: Option<Arc<dyn CacheStore>>,
    pub lid_pn_cache: Option<Arc<dyn CacheStore>>,
}

Set individual caches or use CacheStores::all(store) to route all pluggable caches to the same backend:

use whatsapp_rust::{CacheConfig, CacheStores};

let redis = Arc::new(MyRedisCacheStore::new("redis://localhost:6379"));
let config = CacheConfig {
    cache_stores: CacheStores::all(redis),
    ..Default::default()
};

See Custom backends — cache store for a full implementation example.

TypedCache

TypedCache<K, V> is a generic wrapper that dispatches to either moka or a custom CacheStore backend.

Location: src/cache_store.rs

Method	Signature	Description
`from_moka`	`fn from_moka(cache: Cache<K, V>) -> Self`	Wrap an existing moka cache (zero overhead)
`from_store`	`fn from_store(store: Arc<dyn CacheStore>, namespace: &'static str, ttl: Option<Duration>) -> Self`	Create a cache backed by a custom store
`get`	`async fn get<Q>(&self, key: &Q) -> Option<V>`	Look up a value. Misses and deserialization failures return `None`
`insert`	`async fn insert(&self, key: K, value: V)`	Insert or update a value
`invalidate`	`async fn invalidate<Q>(&self, key: &Q)`	Remove a single key
`invalidate_all`	`fn invalidate_all(&self)`	Remove all entries (sync). Requires `tokio-runtime` for custom backends
`clear`	`async fn clear(&self)`	Remove all entries (async). Awaits completion for custom backends
`run_pending_tasks`	`async fn run_pending_tasks(&self)`	Run internal housekeeping (moka only)
`entry_count`	`fn entry_count(&self) -> u64`	Approximate entry count (sync). Returns `0` for custom backends
`entry_count_async`	`async fn entry_count_async(&self) -> u64`	Approximate entry count, delegating to custom backend if available

`invalidate_all()` on custom `CacheStore` backends requires the `tokio-runtime` feature. Without it, the clear is silently skipped. Use the async `clear()` method as an alternative.

Implementing Custom Storage

To implement a custom storage backend:

Implement all four domain traits
The Backend trait is automatically implemented
All methods must be async and thread-safe (Send + Sync)

Example: Redis Store

use async_trait::async_trait;
use redis::aio::ConnectionManager;
use wacore::store::traits::*;
use wacore::store::error::Result;

pub struct RedisStore {
    client: ConnectionManager,
    device_id: i32,
}

impl RedisStore {
    pub async fn new(redis_url: &str) -> Result<Self> {
        let client = redis::Client::open(redis_url)
            .map_err(|e| StoreError::Connection(Box::new(e)))?;
        let conn = client.get_connection_manager().await
            .map_err(|e| StoreError::Connection(Box::new(e)))?;
        
        Ok(Self {
            client: conn,
            device_id: 1,
        })
    }
}

#[async_trait]
impl SignalStore for RedisStore {
    async fn put_identity(&self, address: &str, key: [u8; 32]) -> Result<()> {
        let mut conn = self.client.clone();
        let key_name = format!("identity:{}:{}", self.device_id, address);
        redis::cmd("SET")
            .arg(key_name)
            .arg(&key[..])
            .query_async(&mut conn)
            .await
            .map_err(|e| StoreError::Database(Box::new(e)))?;
        Ok(())
    }
    
    async fn load_identity(&self, address: &str) -> Result<Option<Vec<u8>>> {
        let mut conn = self.client.clone();
        let key_name = format!("identity:{}:{}", self.device_id, address);
        let result: Option<Vec<u8>> = redis::cmd("GET")
            .arg(key_name)
            .query_async(&mut conn)
            .await
            .map_err(|e| StoreError::Database(Box::new(e)))?;
        Ok(result)
    }
    
    // Implement remaining SignalStore methods...
}

#[async_trait]
impl AppSyncStore for RedisStore {
    // Implement all AppSyncStore methods...
}

#[async_trait]
impl ProtocolStore for RedisStore {
    // Implement all ProtocolStore methods...
}

#[async_trait]
impl DeviceStore for RedisStore {
    // Implement all DeviceStore methods...
}

// Backend is automatically implemented!

Best Practices

Thread Safety - Use Arc for shared state, Mutex for mutable state
Error Handling - Convert backend errors to StoreError variants
Transactions - Use database transactions for atomic operations
Retries - Implement retry logic for transient failures
Connection Pooling - Reuse connections when possible
Blocking Operations - Wrap blocking I/O in tokio::task::spawn_blocking

Data Structures

AppStateSyncKey

pub struct AppStateSyncKey {
    pub key_data: Vec<u8>,
    pub fingerprint: Vec<u8>,
    pub timestamp: i64,
}

LidPnMappingEntry

pub struct LidPnMappingEntry {
    pub lid: String,              // LID user part
    pub phone_number: String,     // Phone number user part
    pub created_at: i64,          // Unix timestamp
    pub updated_at: i64,          // Unix timestamp
    pub learning_source: String,  // e.g. "usync", "peer_pn_message"
}

TcTokenEntry

pub struct TcTokenEntry {
    pub token: Vec<u8>,                    // Raw token bytes
    pub token_timestamp: i64,              // When token was received
    pub sender_timestamp: Option<i64>,     // When we sent our token
}

DeviceListRecord

pub struct DeviceListRecord {
    pub user: String,               // User part of JID
    pub devices: Vec<DeviceInfo>,   // Known devices
    pub timestamp: i64,             // Last update timestamp
    pub phash: Option<String>,      // Participant hash from usync
    pub raw_id: Option<u32>,        // ADV raw_id for identity change detection
}

pub struct DeviceInfo {
    pub device_id: u32,           // 0 = primary, 1+ = companions
    pub key_index: Option<u32>,   // Key index if known
}

The raw_id field stores the ADV (Account Device Verification) key index list raw_id from device notifications. When this value changes for a user, it indicates an identity change (e.g., the user reinstalled WhatsApp). The client uses this to detect identity changes and clear Signal sessions for that user's non-primary devices. Per-device sender key tracking is not wiped globally on identity change — that would empty the tracker too aggressively and feed the no-distribution path on the next group send. SKDM redistribution is instead driven per-group/per-device by retry receipts (matching WhatsApp Web's WAWebUpdateLocalSignalSession/markForgetSenderKey behavior).

Error Handling

All storage operations return Result<T> from wacore::store::error. Each variant preserves the underlying typed error as its source() so callers can downcast to the original backend error when needed:

pub enum StoreError {
    Io(#[from] std::io::Error),
    Serialization(#[source] Box<dyn std::error::Error + Send + Sync>),
    Validation(String),
    Connection(#[source] Box<dyn std::error::Error + Send + Sync>),
    Database(#[source] Box<dyn std::error::Error + Send + Sync>),
    RetriesExhausted { op: String },
    Migration(#[source] Box<dyn std::error::Error + Send + Sync>),
    InvalidConfig(String),
    DeviceNotFound(i32),
}

pub type Result<T> = std::result::Result<T, StoreError>;

`StoreError` exposes a helper `is_database_busy_or_locked()` that walks the source chain looking for SQLite `BUSY`/`LOCKED` markers. Retry layers use it to decide whether a database error is transient without depending on a specific backend crate.

FilesExpand file tree

store.mdx

Latest commit

History