Skip to content

perf(noise): pre-key the transport AES-GCM once per connection - #850

Merged
jlucaso1 merged 3 commits into
mainfrom
perf/noise-prekeyed-gcm
Jun 11, 2026
Merged

perf(noise): pre-key the transport AES-GCM once per connection#850
jlucaso1 merged 3 commits into
mainfrom
perf/noise-prekeyed-gcm

Conversation

@jlucaso1

Copy link
Copy Markdown
Collaborator

Problem

The CodSpeed flamegraph for bench_frame_encrypt_in_place[1500] (the typical stanza-frame size) shows setup_gcm at 16.9% of the seal: every Noise frame, in both directions, re-runs the AES-256 key schedule, re-derives the GHASH subkey H = E_K(0), and pays an allocation inside the setup — all key-dependent work, and the transport key is fixed for the whole connection. Same shape as the ltHash fix in #847: a per-call recomputation of a constant.

Change

New crypto::aes_gcm::Aes256GcmKey holds the expanded AES cipher and the keyed GHASH. Per frame, only the nonce-dependent work remains (CTR init and the one pad block); the keyed state is cloned, which is a plain round-key copy with no aeskeygenassist and no polyval re-keying. Aes256GcmEncryption/Aes256GcmDecryption gain new_with_key constructors that feed the existing encrypt/decrypt/tag bodies, and NoiseCipher stores the pre-keyed state instead of raw key bytes.

Scope notes:

  • Byte-identical output, pinned by a differential test against the per-call setup across sizes (0..4096), nonces and aad shapes, plus tamper rejection.
  • The handshake path keeps the per-call setup (its keys change at each step).
  • NoiseCipher grew (round keys live inline), so the IK outcome enum boxes its Continue variant per clippy.
  • NoiseCipher now drives the RustCrypto GCM types directly instead of the pluggable SignalCryptoProvider free functions. No in-repo or known consumer installs a custom provider, and the signal paths still route through it; flagging in case the hook was meant to cover transport crypto too.

Measured

Local wall-time A/B on 1500-byte frames (AES-NI + CLMUL host): decrypt median 11.51 -> 9.19 us (-20%), encrypt 9.76 -> 9.08 us (-7%); 64 KB frames unchanged (data path dominates). The Simulation runner takes the software polyval path where the setup weighs more, so the CodSpeed report should show a larger relative win on the 1500-byte benches.

Tests

Full workspace suites pass (2180 tests, e2e excluded as usual); clippy strict clean.

Every Noise frame, both directions, re-ran the AES-256 key schedule and re-derived the GHASH subkey through setup_gcm even though the transport key is fixed for the connection lifetime; the flamegraph put that setup at 17% of a typical 1500-byte frame seal, plus a per-frame allocation inside it.

Aes256GcmKey holds the expanded cipher and the keyed GHASH; per frame only the nonce-dependent counter init and pad block remain, with the keyed state cloned (a plain copy, no key expansion). NoiseCipher stores it and drives the existing encrypt/decrypt bodies through new_with_key constructors, so the output is byte-identical, pinned by a differential test against the per-call setup across sizes, nonces and aad shapes. The handshake path (keys change per step) keeps the per-call setup; the IK outcome enum boxes its grown variant.

Local wall-time A/B on 1500-byte frames: decrypt median 11.51 to 9.19 us (-20%), encrypt 9.76 to 9.08 us; 64 KB frames unchanged (data path dominates).
@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 0006d4cf-a7e2-4cf2-a98b-13ce9da9732c

📥 Commits

Reviewing files that changed from the base of the PR and between b3b57a8 and 3cbc332.

📒 Files selected for processing (1)
  • wacore/libsignal/src/crypto/provider.rs

📝 Walkthrough

Summary by CodeRabbit

  • New Features

    • Added a pre-keyed AES-256-GCM transport mode for repeated encryption/decryption to reduce per-message overhead.
  • Refactor

    • Transport and handshake paths updated to use a connection-lifetime transport abstraction.
    • Encryption/decryption now use a persistent transport instance, improving performance and lowering latency.
  • Tests

    • Added tests ensuring parity with prior per-call behavior and validating authentication tag verification.

Walkthrough

Adds a pre-keyed AES-256-GCM key type and TransportAead trait, exposes them via crypto re-exports, integrates connection-lifetime AEAD into NoiseCipher to avoid per-call key expansion, and boxes the IkServerHelloOutcome::Continue payload.

Changes

Cryptographic Pre-keying and Noise Protocol Optimization

Layer / File(s) Summary
Pre-keyed AES-GCM infrastructure and testing
wacore/libsignal/src/crypto/aes_gcm.rs
Refactors GcmGhash initialization; adds pub struct Aes256GcmKey that precomputes AES key schedule and keyed GHASH; adds Aes256GcmEncryption::new_with_key and Aes256GcmDecryption::new_with_key; adds unit test pre_keyed_matches_per_call_setup.
TransportAead trait and default implementation
wacore/libsignal/src/crypto/provider.rs
Adds public TransportAead trait and a PerCallTransportAead that delegates per-call operations to the configured SignalCryptoProvider.
RustCryptoProvider Aes256GcmKey-backed transport
wacore/libsignal/src/crypto/provider.rs
RustCryptoProvider::transport_aead returns an Aes256GcmKey-backed transport; Aes256GcmKey implements TransportAead performing in-place AES-GCM via new_with_key and handling tags/auth failures.
Public module re-exports and helper
wacore/libsignal/src/crypto/mod.rs
Exports Aes256GcmKey and TransportAead from the crypto module; transport_aead helper now delegates to provider implementation with reformatted doc/signature.
NoiseCipher integration with TransportAead
wacore/noise/src/state.rs
NoiseCipher stores Box<dyn TransportAead> built in new; encrypt_with_counter, encrypt_in_place_with_counter, and decrypt_in_place_with_counter delegate to the boxed AEAD, preparing buffers and mapping errors as before.
IkServerHelloOutcome envelope optimization
wacore/noise/src/handshake.rs
Changes IkServerHelloOutcome::Continue payload to Box<IkHandshakeOutcome> and updates read_server_hello to return the boxed outcome on success.

Sequence Diagrams

sequenceDiagram
  participant Client
  participant Aes256GcmKey
  participant GcmGhash
  participant AES as AES256_Cipher

  Client->>Aes256GcmKey: new(raw_key)
  Aes256GcmKey->>AES: expand key schedule
  Aes256GcmKey->>GcmGhash: precompute keyed GHASH (H = AES(key, 0))

  Client->>Aes256GcmKey: setup(nonce, aad)
  Aes256GcmKey->>GcmGhash: from_keyed(keyed_h, aad)
  GcmGhash->>GcmGhash: compute GHASH pad and init state
  Aes256GcmKey-->>Client: (CTR, GcmGhash)

  Client->>Client: encrypt/decrypt using CTR + GcmGhash
Loading
sequenceDiagram
  participant NoiseCipher
  participant Provider
  participant Aes256GcmKey
  participant AEAD as TransportAead
  participant Buffer

  NoiseCipher->>Provider: provider().transport_aead(key)
  Provider-->>NoiseCipher: boxed Aes256GcmKey as TransportAead

  rect rgba(100, 150, 200, 0.5)
    Note over NoiseCipher,Buffer: Encryption with counter
    NoiseCipher->>AEAD: encrypt_in_place(iv, buffer, aad)
    AEAD->>Buffer: in-place ciphertext + appended tag
  end

  rect rgba(150, 150, 100, 0.5)
    Note over NoiseCipher,Buffer: Decryption with counter
    NoiseCipher->>NoiseCipher: validate buffer ≥ TAG_LEN
    NoiseCipher->>AEAD: decrypt_in_place(iv, buffer, aad)
    AEAD->>NoiseCipher: success or auth failure
    NoiseCipher->>Buffer: truncate to plaintext on success
  end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

You need this to work end-to-end — verify the provider wiring and the boxed handshake change compile cleanly.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title 'perf(noise): pre-key the transport AES-GCM once per connection' directly describes the main optimization: moving AES-GCM key setup from per-frame to per-connection.
Description check ✅ Passed The description thoroughly explains the problem (16.9% of seal time spent on redundant setup), the solution (pre-keyed state stored in Aes256GcmKey), measured performance improvements, and test coverage.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch perf/noise-prekeyed-gcm

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 4 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread wacore/noise/src/state.rs Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a567ccf452

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread wacore/noise/src/state.rs Outdated
@codspeed-hq

codspeed-hq Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will improve performance by 10.55%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 139 untouched benchmarks

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation bench_frame_encrypt_in_place[1500] 34 µs 30.7 µs +10.55%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing perf/noise-prekeyed-gcm (3cbc332) with main (0bcc3a0)

Open in CodSpeed

…vider hook

NoiseCipher was driving the RustCrypto GCM types directly, so a custom SignalCryptoProvider would silently stop observing transport crypto. The trait now offers transport_aead(key): the default returns a per-call adapter that keeps routing every frame through the configured provider, and RustCryptoProvider overrides it with the pre-keyed Aes256GcmKey fast path. One virtual call per frame, behavior identical on both paths.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@wacore/libsignal/src/crypto/provider.rs`:
- Around line 106-129: PerCallTransportAead currently captures only the key and
re-dispatches to the global provider() in encrypt_in_place/decrypt_in_place,
which loses the SignalCryptoProvider identity passed into transport_aead();
change PerCallTransportAead to hold a reference/handle to the originating
SignalCryptoProvider (or a boxed trait object/cloneable adapter) and call that
provider's aes_256_gcm_encrypt_in_place/aes_256_gcm_decrypt_in_place methods
instead of provider(); alternatively, restrict construction so transport_aead()
only returns implementations tied to the active-provider helper. Also apply the
same fix to the other adapter at the second occurrence (lines referenced around
221-229) so no transport AEAD re-dispatches to the global provider().
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 917d3c37-9e09-456f-96e2-63f8334272bb

📥 Commits

Reviewing files that changed from the base of the PR and between a567ccf and b3b57a8.

📒 Files selected for processing (3)
  • wacore/libsignal/src/crypto/mod.rs
  • wacore/libsignal/src/crypto/provider.rs
  • wacore/noise/src/state.rs

Comment thread wacore/libsignal/src/crypto/provider.rs Outdated
…provider

The default adapter re-dispatched through the global provider(), losing the identity of the instance whose transport_aead was called. It now holds a reference to that instance; the 'static receiver both enables the capture and restricts construction to the installed provider, which is the only reachable caller.
@jlucaso1
jlucaso1 merged commit 47b7f96 into main Jun 11, 2026
13 checks passed
@jlucaso1
jlucaso1 deleted the perf/noise-prekeyed-gcm branch June 11, 2026 14:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant