Skip to content

docs: delivery fan-out pipeline and Soroban client concept docs #518

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
codebestia merged 1 commit into from
Jul 31, 2026
+194 −0
Merged

docs: delivery fan-out pipeline and Soroban client concept docs #518

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
109 changes: 109 additions & 0 deletions apps/backend/docs/concepts-delivery-fanout.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# Per-device delivery, fan-out & receipts pipeline

This document describes how a sent message reaches every recipient device today, and how delivery/read receipts flow back to the sender. It is written against the code as it currently behaves, not the intended design — several pieces exist in the codebase but are not wired into the live send path, and those gaps are called out explicitly.

It covers:

1. what each of `services/fanout.ts`, `services/deliveryPipeline.ts`, `services/deliveryAggregation.ts` and `services/deviceDelivery.ts` is responsible for
2. a step-by-step trace of one message from `send_message` through to every recipient device's delivery receipt reaching the sender
3. how read receipts differ from delivery receipts
4. what is and isn't wired into the live send path

## Actors

- **Sender client**: the device that emits `send_message`
- **Recipient device(s)**: every active (non-revoked) device belonging to every member of the conversation
- **Backend gateway**: the Socket.IO server in `apps/backend/src/index.ts`, potentially one of several instances behind the Redis adapter

## Service responsibilities

### `services/deliveryPipeline.ts` — the live fan-out engine

Given an already-persisted message, loads the conversation's current active devices and delivers a per-device `message_envelope` event to each one that has a matching ciphertext envelope, plus a ciphertext-free `new_message` event to the conversation room for UI/unread-count purposes. This is the only service actually called from `send_message`/`edit_message`.

### `services/deliveryAggregation.ts` — receipt aggregation

Tracks per-device delivery acknowledgements (`messageEnvelopes.deliveredAt`), determines when a recipient user's entire active device set has received a message, and notifies the sender via `message_fully_delivered` when that happens. It also emits a per-device `device_delivery_receipt` to the whole conversation on every ack, independent of full-delivery status.

### `services/fanout.ts` — not wired into the live send path

Implements a stronger fan-out guarantee: validating that the sender-supplied envelope map covers every other conversation member's active devices (not just the sender's own), and persisting the message + envelopes atomically. It is fully implemented and exported (`fanoutMessage`) but has no callers anywhere in the backend and no test exercises it — see [Gaps](#gaps--code-not-wired-into-the-live-path) below.

### `services/deviceDelivery.ts` — half-wired

Defines a Redis pub/sub channel per device (`deliver:device:${deviceId}`) intended as an alternate, cross-gateway-instance delivery path. The *subscribe* side (`GatewayDeviceSubscriber`) is booted for every connected socket and would forward anything published on that channel as a `device_envelope` event — and the web client does listen for `device_envelope` (`apps/web/src/hooks/useInboundPipeline.ts:257`). But the *publish* side (`publishToDevice`) has no production callers, so nothing ever actually flows through this channel today.

## Step-by-step trace: `send_message` → every device → receipt back to sender

All of this happens inside `apps/backend/src/socket/messaging.ts`, wired up per-connection from `apps/backend/src/index.ts`.

**1. Connection setup** — on `io.on('connection', ...)`, each socket joins a `` `device:${deviceId}` `` room (this is the exact room `deliveryPipeline.ts`'s `deviceRoom()` targets), the user's own `` `room:user:${userId}` `` room, and every conversation room it belongs to. `registerMessagingHandlers` then wires up the event handlers below.

**2. Client emits `send_message`** — the handler in `socket/messaging.ts`:
- clears typing timers for the conversation
- validates `messageId`, `contentType`, `ciphertext`/`envelopes`/`fileId` via `validateMessagePayload`
- verifies sender conversation membership
- short-circuits (re-emits `message_ack`) if the message id already exists (idempotency)
- checks that the sender supplied envelopes for all of their **own** other active devices (`fetchSiblingDeviceIds`) — note this only covers the sender's sibling devices, not every recipient's devices (see gaps below)
- persists the `messages` row and the per-recipient-device `messageEnvelopes` rows in one transaction
- emits `message_ack` back to the sender
- calls `deliverMessage(io, message, conversationId)` — the hand-off into fan-out

**3. Fan-out — `deliveryPipeline.ts`'s `deliverMessage`**:
- re-queries `conversationMembers` for the conversation (source of truth, not room state)
- loads active (non-revoked) devices for those members
- loads the envelopes just persisted, filtered to those active devices
- for each active device with a matching envelope: emits `message_envelope` (with ciphertext) to `` io.to(`device:${deviceId}`) ``. Because every connected socket already joined that room at connect time, and the Socket.IO Redis adapter fans room emits across gateway instances, this reaches the recipient wherever it's connected
- emits a ciphertext-free `new_message` to the conversation room(s) for UI/unread-count updates

**4. Offline fallback** — after `deliverMessage` returns, `send_message` invalidates conversation-list caches and fires `dispatchOfflinePush` for any recipient device with no live socket connection, queuing a Web Push notification. This path is independent of the Socket.IO fan-out and never touches `messageEnvelopes.deliveredAt`.

**5. Recipient acks receipt — `message_delivered`** — when a recipient device receives `message_envelope`, its client emits `message_delivered` with `{conversationId, messageId, envelopeId, sequenceNumber}`. The handler validates required fields and membership, then calls `handleDeviceDeliveryReceipt(io, redis, messageId, recipientDeviceId, recipientUserId, conversationId)`.

**6. Aggregation — `deliveryAggregation.ts`'s `handleDeviceDeliveryReceipt`**:
- looks up the message's `senderId`
- short-circuits if this device's envelope is already marked delivered (idempotency)
- sets `messageEnvelopes.deliveredAt = now()` for `(messageId, recipientDeviceId)`
- checks `isMessageFullyDeliveredToUser`: whether every active device belonging to that recipient user now has `deliveredAt` set
- **if fully delivered**: emits `message_fully_delivered` to `` `room:user:${senderId}` `` — this is the moment a delivery receipt actually reaches the sender for full multi-device delivery — and republishes it over Redis (`publishEphemeral`) so it survives replay if the sender wasn't connected at that instant
- **always**: also emits a per-device `device_delivery_receipt` to the whole conversation room (`.volatile`), visible to all members, not just the sender

**7. Read receipts are a separate, simpler mechanism** (see below) and never touch `deliveryAggregation.ts` or `messageEnvelopes`.

```text
Sender Backend (send_message) deliveryPipeline Recipient device(s) deliveryAggregation
|--send_message------->| | | |
|<--message_ack---------| | | |
| |--deliverMessage--------->| | |
| | |--message_envelope------->| |
| | |--new_message (no ct)---->| (conversation room) |
| | | |--message_delivered------------------->|
| | | | |--sets deliveredAt
|<---message_fully_delivered (once all devices ack)-------------------------------------------------------|
| (also) <--device_delivery_receipt to conversation room, per device ack-----------------------|
```

## Read receipts vs. delivery receipts

Read receipts are conversation-level, not per-device: the `message_read` handler updates a single `conversationMembers.lastReadMessageId` cursor per `(conversationId, userId)` and broadcasts `read_receipt` to the whole conversation room. There is no per-envelope/per-device read tracking and no "fully read by all devices" concept analogous to `message_fully_delivered` — delivery tracks per-device state via `messageEnvelopes.deliveredAt`; reads do not have an equivalent per-device column that's actually used.

## Gaps — code not wired into the live path

- **`fanoutMessage` (`services/fanout.ts`) has zero callers.** The live `send_message` handler only validates the sender's own sibling devices via `fetchSiblingDeviceIds`, not the full "every recipient's active devices are covered" guarantee that `fanoutMessage` implements.
- **`publishToDevice` (`services/deviceDelivery.ts`) has zero callers.** Its subscribe side is booted on every connection, but nothing publishes to `deliver:device:${deviceId}` in production — the room-based delivery in `deliveryPipeline.ts` is what actually runs.
- **`getMessageDeliveryStatus` (`services/deliveryAggregation.ts`) has zero callers** outside its own test — no route or handler exposes a delivery-status read model.
- **`POST /messages` (REST route) bypasses the fan-out pipeline entirely.** It persists the message/envelopes and does a bare `new_message` room emit, but never calls `deliverMessage` or `dispatchOfflinePush` — clients sending via REST get no per-device envelope push and no offline Web Push fallback.
- **`device_delivery_receipt` is emitted twice per `message_delivered` call** — once from `deliveryAggregation.ts` and once again from `socket/messaging.ts`'s own handler — to the same conversation room.
- **`messageEnvelopes.readAt` and `users.sendReadReceipts` exist in the schema but are never read or written** anywhere in the backend.

## Implementation references

- `apps/backend/src/socket/messaging.ts` — `send_message`, `message_delivered`, `message_read`, `edit_message` handlers
- `apps/backend/src/services/deliveryPipeline.ts` — per-device fan-out (`deliverMessage`, `deviceRoom`)
- `apps/backend/src/services/deliveryAggregation.ts` — receipt aggregation (`handleDeviceDeliveryReceipt`, `isMessageFullyDeliveredToUser`, `getMessageDeliveryStatus`)
- `apps/backend/src/services/fanout.ts` — unused stronger fan-out validation (`fanoutMessage`)
- `apps/backend/src/services/deviceDelivery.ts` — Redis pub/sub per-device channel (`publishToDevice`, `GatewayDeviceSubscriber`)
- `apps/backend/src/services/pushNotification.ts` — offline Web Push fallback (`dispatchOfflinePush`)
- `apps/backend/src/index.ts` — connection setup, device/user/conversation room joins, Redis adapter attachment
- `apps/backend/src/routes/messages.ts` — REST send path that bypasses `deliveryPipeline.ts`
- See [`e2ee-onboarding.md`](./e2ee-onboarding.md) for how envelopes/ciphertext are constructed client-side before `send_message` is emitted.
85 changes: 85 additions & 0 deletions apps/web/docs/api-soroban-client.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# Soroban contract client usage

This document describes how the frontend builds, signs, and submits Soroban (Stellar smart contract) invocations via the Freighter wallet, which contract functions are currently invoked and from where, and how network/RPC configuration works today.

It covers:

1. `apps/web/src/lib/soroban.ts` and `apps/web/src/lib/freighter.ts`
2. the full invocation flow: build transaction → request Freighter signature → submit → await result
3. every contract function currently invoked from the frontend and its trigger
4. network/RPC configuration

## Actors

- **User** interacting with the app in a browser with the Freighter extension installed
- **Freighter wallet** — browser extension holding the user's Stellar keypair, used to sign transactions/messages without exposing the private key to the app
- **Soroban RPC** — the Stellar RPC endpoint the frontend talks to directly to simulate and submit transactions

## Modules

### `apps/web/src/lib/freighter.ts`

Thin wrapper around `@stellar/freighter-api`:

- `requestWalletAccess(): Promise<string>` — calls Freighter's `requestAccess()`, normalizes the response (`address` or `publicKey`), and returns the connected wallet's public key, or throws if Freighter didn't return one.
- `signWalletMessage(message: string, address?: string): Promise<string>` — calls Freighter's `signMessage()`, normalizes the response (`signedMessage` or `signature`), and returns the signature string, or throws.

### `apps/web/src/lib/soroban.ts`

- `transferToken(recipient: string, amount: number|string, memo = ''): Promise<string>` (default export) — builds a `transfer(from, to, amount, memo)` invocation against the configured token-transfer contract, requests a Freighter signature, submits the signed transaction to Soroban RPC, polls until it resolves, and returns the transaction hash.

## Contract functions invoked from the frontend

| Function | Called via | Triggered from | User action |
|---|---|---|---|
| `transfer` (token-transfer contract) | `transferToken` in `lib/soroban.ts` | `components/chat/MessageInput.tsx` (`handleConfirmTransfer`) | Clicking the token icon in the chat message input to open the "Send token" popover, entering an amount, then clicking **Confirm** |
| Freighter `requestAccess` (wallet connect, not a contract call) | `requestWalletAccess` in `lib/freighter.ts` | `app/app/layout.tsx` (`handleWalletAction`) | Clicking **Connect Wallet** in the app sidebar |
| Freighter `signMessage` (message signing, not a contract call) | `signWalletMessage` in `lib/freighter.ts` | `components/treasury/ProposalCard.tsx` (`castVote`) | Clicking **Approve** or **Reject** on a treasury proposal card — signs `` `${type}:${proposalId}` `` and POSTs it to the backend for verification |

`transferToken` is currently the only function that submits an actual Soroban **contract** invocation; `requestWalletAccess`/`signWalletMessage` are Freighter wallet operations (connect / sign-message) used for wallet connection and off-chain approval signatures respectively, not contract calls.

Two other consumers exist in the codebase but are not reachable from the live app and are not part of the flow above:

- `components/wallet/WalletConnectButton.tsx` also calls `requestWalletAccess`/wallet `disconnect`, but this component is never imported or rendered anywhere — the sidebar button in `app/app/layout.tsx` duplicates the same logic inline instead.
- `contexts/AuthContext.tsx` implements a full "connect wallet → sign a challenge message → verify with backend" login flow using `signWalletMessage`, but this provider is never mounted — the root layout wires a different, unrelated `AuthProvider` (`components/auth/AuthProvider.tsx`) that has no Freighter interaction.

## Invocation flow: `transferToken`

All steps are in `apps/web/src/lib/soroban.ts`:

1. **Lazy-load dependencies** — `@stellar/freighter-api` and `stellar-sdk` are dynamically imported inside the function so the SDK stays out of the initial bundle.
2. **Resolve config** — reads `NEXT_PUBLIC_SOROBAN_RPC_URL`, `NEXT_PUBLIC_NETWORK_PASSPHRASE`, `NEXT_PUBLIC_TOKEN_TRANSFER_CONTRACT` (see [Network configuration](#network-configuration)).
3. **Check wallet connectivity** — `freighter.isConnected()`; throws if Freighter isn't installed/connected.
4. **Read the active address** — `freighter.getAddress()`.
5. **Build the contract call** — `new Contract(CONTRACT_ID)`, converts `from`/`to` to `ScVal` addresses, amount to `i128` via `BigInt`, memo to a symbol `ScVal`, then `contract.call('transfer', from, to, amount, memo)`.
6. **Fetch the source account** — `new SorobanRpc.Server(RPC_URL, { allowHttp: false })`, then `server.getAccount(publicKey)`.
7. **Build the transaction** — `TransactionBuilder(account, { fee: BASE_FEE, networkPassphrase }).addOperation(op).setTimeout(30).build()`.
8. **Simulate** — `server.simulateTransaction(tx)`; throws on a simulation error.
9. **Assemble from simulation** — `SorobanRpc.assembleTransaction(tx, simResult).build()`, then serialize to XDR.
10. **Request Freighter signature** — `freighter.signTransaction(txXdr, { networkPassphrase })`. This is the wallet popup where the user approves/signs.
11. **Validate the signature response** — throws if Freighter returned an error or no signed XDR.
12. **Submit** — rehydrate a `Transaction` from the signed XDR and call `server.sendTransaction(signedTx)`; throws if submission itself errors.
13. **Poll for confirmation** — up to 30 attempts, 2s apart, calling `server.getTransaction(hash)`. Resolves on `SUCCESS`, throws on `FAILED`, swallows transient lookup errors mid-poll, and throws a timeout error after ~60s with no resolution.
14. **Return the transaction hash** to the caller. `MessageInput.tsx` then sends a chat message of the form `{ type: 'transfer', amount, token: 'TOKEN', txHash }`, later rendered by `TransferCard.tsx`, which links out to the Stellar Explorer for that hash.

## Network configuration

Configuration is read directly in `soroban.ts` from `NEXT_PUBLIC_*` environment variables, each with a hardcoded fallback:

| Env var | Default if unset | Used for |
|---|---|---|
| `NEXT_PUBLIC_SOROBAN_RPC_URL` | `https://soroban-testnet.stellar.org` | Soroban RPC server URL |
| `NEXT_PUBLIC_NETWORK_PASSPHRASE` | `Networks.TESTNET` (from `stellar-sdk`) | Network passphrase for building/signing the transaction |
| `NEXT_PUBLIC_TOKEN_TRANSFER_CONTRACT` | literal placeholder string `REPLACE_WITH_TOKEN_TRANSFER_CONTRACT_ID` | Contract ID passed to `new Contract(...)` |
| `NEXT_PUBLIC_NETWORK` (read separately in `components/chat/TransferCard.tsx`) | `test` | Only used to build the Stellar Explorer link, not for building/submitting the transaction |

None of these are currently listed in the root `.env.example` — that file only defines backend-facing equivalents without the `NEXT_PUBLIC_` prefix (`RPC_URL`, `TOKEN_TRANSFER_CONTRACT_ID`, `GROUP_TREASURY_CONTRACT_ID`, `PROPOSALS_CONTRACT_ID`), which are validated separately in `apps/backend/src/config.ts` and are not exposed to the browser. `next.config.ts` doesn't do anything special for the Soroban vars — Next.js exposes any `NEXT_PUBLIC_*` var automatically — but because they aren't documented anywhere in `apps/web`, it's easy to leave `NEXT_PUBLIC_TOKEN_TRANSFER_CONTRACT` unset and silently deploy with the placeholder contract ID. Backend and frontend contract IDs are validated independently and can point at different contracts if misconfigured.

## Implementation references

- `apps/web/src/lib/soroban.ts` — `transferToken`, build/sign/submit/poll flow
- `apps/web/src/lib/freighter.ts` — `requestWalletAccess`, `signWalletMessage`
- `apps/web/src/components/chat/MessageInput.tsx` — token-transfer UI and trigger
- `apps/web/src/app/app/layout.tsx` — wallet connect/disconnect UI and trigger
- `apps/web/src/components/treasury/ProposalCard.tsx` — proposal vote signing and trigger
- `apps/web/src/contexts/WalletContext.tsx` — wallet connection state