build(codegen): own the whatspec codegen and regenerate at 2.3000.1044659339

.github/workflows/main.yml

@@ -34,6 +34,34 @@ jobs:
           components: rustfmt
       - run: cargo fmt --all -- --check
 
+  generated:
+    name: Generated Artifacts
+    runs-on: ubuntu-latest
+    # The vendored whatspec files are regenerated, never edited, and the tool
+    # that writes them pins the exact IR commit — so a hand-edit or a
+    # half-refreshed tree is a diff this job can name. Lives beside `format`
+    # rather than in a build job: it needs rustfmt and git, not protoc (--check
+    # never rebuilds the descriptor) and not a compiled workspace.
+    # Bounded because the step fetches an external repository: a hung fetch
+    # would otherwise hold a required check open for the full six hours.
+    timeout-minutes: 15
+    steps:
+      # This job never pushes, and the codegen shells out to `git fetch` against
+      # a different repository; the checkout token has no business being left in
+      # .git/config for that.
+      - uses: actions/checkout@v6
+        with:
+          persist-credentials: false
+      - uses: dtolnay/rust-toolchain@master
+        with:
+          toolchain: nightly-2026-06-16
+          components: rustfmt
+      # RUSTC_WRAPPER is set workspace-wide, so cargo needs sccache on PATH even
+      # in a job this small.
+      - name: Setup sccache
+        uses: mozilla-actions/sccache-action@v0.0.10
+      - run: cargo run -p whatspec-codegen -- --check
+
   clippy:
     name: Clippy Linter
     runs-on: ubuntu-latest

AGENTS.md

@@ -9,6 +9,7 @@ Ground truth for protocol behavior is WhatsApp Web itself: query the structured
 - **wacore** — platform-agnostic core: binary protocol, crypto, IQ types, state traits. Also builds for wasm32 and ESP32, so no Tokio here.
 - **waproto** — prost-generated protobufs from `whatsapp.proto`. No feature logic.
 - **whatsapp-rust** — Tokio runtime, SQLite persistence (Diesel), high-level API.
+- **whatspec-codegen** (`tools/`) — build tooling, never published and outside `default-members`. Regenerates every whatspec-derived file in one pass from a pinned IR commit. Nothing links it.
 
 ## Build & verify
 
@@ -32,7 +33,8 @@ Things that look correct and are not:
 - **Locks.** `session_locks` serializes Signal encrypt/decrypt per protocol address; `chat_lanes` (`ChatLane::enqueue_lock` in `src/client.rs`) serializes *incoming* processing per chat. Outgoing sends are deliberately not per-chat locked — WA Web doesn't lock them either.
 - **Wire-tagged enums.** Every protocol enum derives `WireEnum`, and its `#[wire = ...]` attribute is the single source of truth for the wire value. Do not also derive `serde::Serialize`/`Deserialize` or add `#[serde(rename_all)]` — the derive owns both. In tagged mode it generates a sibling `<Name>Tag`; parsers must dispatch on `<Name>Tag::try_from(node.tag.as_ref())` rather than string literals, so renaming a tag stays a one-attribute change. Modes and attributes: `agent_docs/protocol_architecture.md`.
 - **Event payloads are a frozen API.** Sealed with `#[non_exhaustive]` + `#[derive(bon::Builder)]` and constructed via `Type::builder()…build()`; a maybe-absent field is `Option<T>`, never an empty-string or zero sentinel. The full stability policy is the `Event` doc comment in `wacore/src/types/events.rs`.
-- **`whatsapp.proto` is not the whole persisted schema.** It comes from whatspec and is regenerated wholesale, so fields we persist but upstream does not declare live in `LOCAL_FIELDS` in `waproto/build.rs`, spliced into the descriptor at build time. Never hand-edit the `.proto` or `.desc` to add one — the next sync would drop it.
+- **Generated files are generated, not edited.** `wacore/src/iq/abprops.rs`, `wacore/src/iq/mex_operations.rs`, `wacore/appstate/src/schemas.rs`, `wacore/binary/src/tokens.json`, `waproto/src/whatsapp.proto` and `wacore/src/version/generated.rs` all come out of `cargo run -p whatspec-codegen`, together, from one pinned whatspec commit. An action or flag the protocol carries but the bundle no longer builds goes in a hand-written sibling (`wacore/appstate/src/schemas_unlisted.rs`, `props::stale`), never in the generated file.
+- **`whatsapp.proto` is not the whole persisted schema.** It comes from whatspec and is regenerated wholesale, so fields we persist but upstream does not declare live in `LOCAL_FIELDS` in `waproto/build.rs`, spliced into the descriptor at build time, and whole retained messages in `LOCAL_BLOCKS` in the codegen's proto emitter. Never hand-edit the `.proto` or `.desc` to add one — the next sync would drop it.
 - **Blocking work** — `ureq`, heavy CPU — belongs in `tokio::task::spawn_blocking`; it shares a runtime with the read loop.
 - **let-chains**, never nested `if let`. Clippy's `collapsible_if` is denied in CI.
 - **No real PII in tests**, including vectors derived from production captures. Regenerate them from fictitious JIDs and numbers.

Cargo.toml

@@ -28,6 +28,7 @@ members = [
     "storages/sqlite-storage",
     "tests/bench-integration",
     "tests/e2e",
+    "tools/whatspec-codegen",
     "transports/tokio-transport",
     "wacore",
     "wacore/appstate",

agent_docs/wa_web_reference.md

@@ -11,7 +11,27 @@ Before adding or changing protocol logic, check it against what the official cli
 
 [`oxidezap/whatspec`](https://github.com/oxidezap/whatspec) parses the WhatsApp Web JS bundle with an `oxc` AST and emits a language-neutral IR: IQ stanzas, protobuf schemas, GraphQL persisted operations, app-state actions, feature flags, wire enums, notification dispatch, binary-protocol token dictionaries. The IR is a derived model of the contract — a static reading of minified code, not the contract itself — and the committed Rust modules are one consumer of that model. Treat it as high-quality evidence, not as a specification; the limits are spelled out below.
 
-This repo already vendors parts of it — `wacore/src/iq/mex_operations.rs` is copied verbatim from `generated/mex/operations.rs`, and the protobuf and app-state work came from the same place. Refreshing a vendored file is a `cp`.
+This repo vendors parts of it. whatspec commits only the IR now (`generated/**/*.rs` is gitignored there), so the vendored files are produced here, by `tools/whatspec-codegen`:
+
+| Vendored file | Domain |
+| --- | --- |
+| `wacore/src/iq/abprops.rs` | `abprops` |
+| `wacore/src/iq/mex_operations.rs` | `mex` |
+| `wacore/appstate/src/schemas.rs` | `appstate` |
+| `wacore/binary/src/tokens.json` | `tokens` |
+| `waproto/src/whatsapp.proto` | `proto` |
+| `wacore/src/version/generated.rs` | `manifest.json`'s `waVersion` |
+
+```sh
+cargo run -p whatspec-codegen                    # regenerate from the pinned commit
+cargo run -p whatspec-codegen -- --check         # fail if the tree drifted from it
+cargo run -p whatspec-codegen -- --from ../whatspec/generated   # use a local checkout
+cargo run -p whatspec-codegen -- --update-lock --rev main       # move to a newer bundle
+```
+
+The pinned commit and the per-file digests live in `tools/whatspec-codegen/whatspec.lock.json`, and the WhatsApp build every artifact came from is generated as `wacore::version::WA_WEB_VERSION` — which is also the default a fresh device announces. Regeneration is all-or-nothing: refreshing one domain alone is how the tree ended up with abprops and mex describing two different WhatsApp releases, and `whatspec-codegen` refuses to write a set that does not agree.
+
+Writing the `.proto` also rebuilds `waproto/src/whatsapp.desc`, so `protoc` (the version in `.github/workflows/main.yml`'s `PROTOC_VERSION`) has to be on PATH; `--skip-proto-desc` defers it. Local additions to the schema are not hand-edits: a whole message goes in the emitter's `LOCAL_BLOCKS` (`tools/whatspec-codegen/src/emit/proto.rs`), a single field in `LOCAL_FIELDS` (`waproto/build.rs`). Either way it survives the next sync, and a sync that starts declaring the same name fails the build instead of shadowing it.
 
 ```sh
 git clone https://github.com/oxidezap/whatspec

src/version.rs

@@ -5,7 +5,7 @@ use anyhow::{Context as _, Result, anyhow};
 use log::debug;
 use std::sync::Arc;
 
-pub use wacore::version::parse_sw_js;
+pub use wacore::version::{WA_WEB_VERSION, WA_WEB_VERSION_STR, parse_sw_js};
 
 const SW_URL: &str = "https://web.whatsapp.com/sw.js";
 

tools/whatspec-codegen/Cargo.toml

@@ -0,0 +1,23 @@
+[package]
+name = "whatspec-codegen"
+# 0.0.0 like the other non-published members: nothing links this, so it has no
+# version to keep in lockstep with the released crates.
+version = "0.0.0"
+edition = "2024"
+rust-version.workspace = true
+# Build tooling, not a library: it exists to rewrite files in this checkout and
+# has no meaning outside it.
+publish = false
+description = "Regenerates the whatspec-derived artifacts committed in this repository."
+
+[dependencies]
+# `std` on each: the workspace pins these with default-features off for the
+# no_std/wasm builds, and none of that applies to a host-only binary that reads
+# files and shells out.
+anyhow = { workspace = true, features = ["std"] }
+serde = { workspace = true }
+serde_json = { workspace = true, features = ["std"] }
+sha2 = { workspace = true }
+
+[lints]
+workspace = true