Skip to content
Merged
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 55 additions & 1 deletion api/chat-store.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,45 @@ Newest-first, **keyset-paginated**: pass the [`MessageCursor`](#messagecursor) o

`chat` accepts either of a 1:1 peer's identities (phone number or LID) — see [PN/LID identity aliasing](#pnlid-identity-aliasing).

```rust
async fn messages_by_arrival(
&self,
after: Option<ArrivalCursor>,
limit: i64,
) -> Result<Vec<StoredMessage>>;

async fn messages_by_arrival_in_range(
&self,
after: Option<ArrivalCursor>,
since: Option<DateTime<Utc>>,
until: Option<DateTime<Utc>>,
limit: i64,
) -> Result<Vec<StoredMessage>>;
```

The session-wide feed: every chat interleaved, newest arrival first. This is the read a reconciliation consumer wants — "everything that landed since I last looked, across all chats" — which `chats` plus `messages` can otherwise only answer by paging every thread. `messages_by_arrival` is `messages_by_arrival_in_range` with no bounds; the ranged form additionally restricts the feed to a half-open wall-clock window, `since <= timestamp < until`, so adjacent windows tile without double-counting or dropping a row. Either bound may be `None`. A `limit` of zero, or negative (which SQLite reads as unbounded), returns nothing rather than the whole table.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Address the reader directly in the feed overview

The new overview describes what “a reconciliation consumer” wants instead of addressing the reader. Rewrite this as direct guidance such as “Use this read when you want…” so the section follows the repository’s required second-person voice.

AGENTS.md reference: AGENTS.md:L24-L24

Useful? React with 👍 / 👎.

Comment thread
jlucaso1 marked this conversation as resolved.
Outdated

**Keyset-paginated by [`ArrivalCursor`](#arrivalcursor)**, not `MessageCursor` — the feed sorts by [`seq`](#storedmessage) alone rather than `(timestamp_ms, seq)`, so a cursor from `messages()` can't page it. The intended usage is a loop that re-enters at the head every pass (`after: None`) and walks down until it recognizes rows it already has, comparing by content — `(chat_jid, id)` — never by stopping at a remembered `seq`:
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Outdated
Comment thread
jlucaso1 marked this conversation as resolved.

```rust
let mut after = None;
loop {
let page = chat_store.messages_by_arrival(after, 100).await?;
let Some(oldest) = page.last() else { break };
after = Some(oldest.into());
if page.iter().all(|m| already_stored(&m.chat_jid, &m.id)) { break }
// ... take the ones that are new ...
}
```

Stopping at a saved watermark instead skips messages, silently. SQLite hands out the rowid backing `seq` as `max(rowid) + 1`: deleting whichever row currently holds the table's highest rowid frees that number for the next arrival to reuse. `delete-for-me` and `clear-chat` — both routine, since they go through this store — delete rows scoped to one chat, not the whole per-device table, so a clear only resets the counter all the way to 1 when the chat it empties happened to hold every remaining row; with other chats still populated, it simply frees whatever rowids that chat held, which is already enough for a later insert to land at or below a remembered watermark. Either way, a message that does so reads as already seen and never surfaces again under a watermark comparison.

Ordered by arrival rather than `timestamp_ms` for the same reason `MessageCursor` isn't reused here: history-sync backfill inserts old conversations at *new* arrival positions, so a poller keyed on `timestamp` files that backfill behind its watermark and never looks at it again, while an arrival-keyed one sees it at the head on its next pull.

A revoke, edit, star, or status change rewrites a row in place and leaves its `seq` untouched — `seq` is assigned once, by the insert — so a message this feed has already walked past never resurfaces no matter what happens to it afterward. A tombstone or an undecryptable placeholder is a row like any other and does appear, since it's inserted like one. A consumer that needs to react to mutations, not just arrivals, subscribes to [`StoreChange::Messages`](#storechange) instead.

The wall-clock window is a filter over the arrival scan, not a seek, so cost tracks rows walked rather than rows returned — a narrow window over an old part of a large store still reads everything newer than it before yielding anything.

```rust
async fn message(&self, chat: &Jid, msg_id: &str) -> Result<Option<StoredMessage>>;
async fn reactions(&self, chat: &Jid, msg_id: &str) -> Result<Vec<ReactionEntry>>;
Expand Down Expand Up @@ -331,7 +370,7 @@ pub struct StoredMessage {
pub starred: bool,
pub edited_at: Option<DateTime<Utc>>,
pub revoked: bool,
pub seq: i64, // arrival order (SQLite rowid) — see MessageCursor
pub seq: i64, // arrival order (SQLite rowid) — see MessageCursor, ArrivalCursor
}
```

Expand All @@ -352,6 +391,20 @@ The keyset-pagination cursor used by [`messages()`](#querying). `seq` is the row
**Breaking change:** `MessageCursor.msg_id: String` is now `seq: i64`. The previous tiebreak sorted same-second messages (the server's timestamp is whole seconds, so a live back-and-forth often lands several on one value) by comparing message ids, which put a peer's message above a same-second reply most of the time — outgoing ids carry a fixed generated prefix that isn't comparable to a peer's. Same-second messages now resolve in arrival order instead. Always build a cursor with `MessageCursor::from(&stored_message)` rather than constructing the fields directly, and this change needs no code update on your side.
</Note>

### ArrivalCursor

```rust
pub struct ArrivalCursor {
pub seq: i64,
}

impl From<&StoredMessage> for ArrivalCursor { /* ... */ }
```

The keyset-pagination cursor used by [`messages_by_arrival()` and `messages_by_arrival_in_range()`](#querying). Separate from `MessageCursor` because the two feeds sort by different keys — a per-chat page orders by `(timestamp_ms, seq)`, the session-wide feed by `seq` alone — so a cursor from one cannot page the other; a type that silently ignored the timestamp half would be worse than a compile error. Build one with `ArrivalCursor::from(&stored_message)`.

Same non-durability caveat as `MessageCursor.seq`, and one more besides: `seq` is good for a live paging session and must not be persisted across restarts, *and* must never be compared against a remembered value as a watermark — a new message can legitimately land at or below a `seq` you've already seen. See [Querying](#querying) for why.

### MessageKind

```rust
Expand Down Expand Up @@ -471,6 +524,7 @@ pub type Result<T> = std::result::Result<T, ChatStoreError>;
- **Reaction removal is a tombstone, not a delete.** Removing a reaction (an empty-emoji event, or [`record_reaction`](#recording-local-amendments) with `emoji: ""`) keeps the row so a stale, older reaction arriving later — e.g. from a history chunk — can't resurrect it. [`reactions()`](#querying) hides tombstoned rows.
- **PN/LID splits heal, they don't recur.** Once live traffic (or [`reconcile_chat`](#pnlid-identity-aliasing)) merges a peer's phone-number- and LID-keyed rows into one thread, later traffic under either identity keeps routing to that same thread — it can't re-split.
- **Companion-device traffic never forks a thread.** A device-suffixed identity is normalized to the bare peer before it reaches routing, so a linked device can't materialize a chat, contact, or receipt row of its own — see [Companion-device identities](#companion-device-identities).
- **The arrival feed tracks insertion, not mutation.** [`messages_by_arrival`/`messages_by_arrival_in_range`](#querying) order by `seq`, assigned once at insert and left untouched by any later edit, revoke, star, or status change — so a row the feed has already walked past never resurfaces there no matter what happens to it afterward. Subscribe to [`StoreChange::Messages`](#storechange) for that; the feed answers "what has arrived", not "what has changed".

## See also

Expand Down