doc: design durable subscribe

A `SUBSCRIBE` cannot be resumed today. A client that loses its connection
re-runs the subscribe and processes a fresh snapshot before it sees a new
update. `AS OF` already expresses the resume, but nothing keeps the timestamp
readable, and the default compaction window is one second, so a client that
remembers a position finds it compacted away.

This design proposes a durable subscription: a named catalog object holding a
read hold on a storage collection, which the consumer advances by acknowledging
what it has committed. The hold defines a window of readable time, the
acknowledgement moves its lower edge, and a wall-clock time to live bounds how
long the window stays open without progress.

Two decisions carry most of the weight. The mechanism is a read hold rather
than a read policy, because `ReadPolicy::Multiple` has no construction sites,
policy installation only ratchets capabilities upward, and an ordinary `ALTER
... RETAIN HISTORY` would discard the contribution silently. And the
subscription's hold is the attached dataflow's input floor rather than a second
hold, because the compute controller never relaxes a sink's input hold, so a
continuously connected reader would otherwise pin history from its attach point
forever.

The user documentation is included. It documents a feature that does not exist
yet, and was written as part of the design to expose awkwardness in the SQL
surface. It earned its place: writing it is what surfaced that progress
messages are mandatory rather than optional, that the time to live must be
required rather than defaulted, that retention is the minimum over readers, and
that at-least-once delivery is a step back from what the existing manual
pattern achieves for consumers with a transactional sink.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: antiguru

doc/developer/design/20260825_durable_subscribe.md

@@ -0,0 +1,107 @@
+---
+title: "ACKNOWLEDGE"
+description: "`ACKNOWLEDGE` advances the position of a durable subscription."
+menu:
+  main:
+    parent: 'commands'
+---
+
+`ACKNOWLEDGE` tells Materialize how far you have processed a [durable
+subscription](/sql/create-durable-subscription/), which advances the position it
+resumes from and releases the history before it.
+
+## Syntax
+
+```mzsql
+ACKNOWLEDGE DURABLE SUBSCRIPTION <name> AT <timestamp>
+;
+```
+
+| Field | Use |
+| --- | --- |
+| `<name>` | The durable subscription to advance. |
+| `<timestamp>` | An [`mz_timestamp`](/sql/types/mz_timestamp/). Asserts that you have durably processed every update at times **strictly before** this value. |
+
+## Details
+
+### What to acknowledge
+
+Acknowledge the `mz_timestamp` of a progress message, which is why
+[`SUBSCRIBE`](/sql/subscribe/#progress) must be run `WITH (PROGRESS)` when you
+intend to acknowledge. A progress message with timestamp `t` means no further
+updates will arrive at times strictly before `t`, which is exactly the claim
+`ACKNOWLEDGE ... AT t` makes back to Materialize.
+
+Do not acknowledge the timestamp of an ordinary row. Not every timestamp
+produces a progress message, and a row at time `t` does not mean that time `t`
+is complete, so acknowledging it can skip updates you have not seen.
+
+You do not have to subtract anything from the timestamp you acknowledge. When you
+resume without an explicit `AS OF`, Materialize positions the subscription so
+that you receive updates at and after the acknowledged time, whether or not you
+request a snapshot. The subtraction is only needed if you choose to pass [`AS
+OF`](/sql/create-durable-subscription/#where-reading-starts) yourself, which is
+an exclusive bound.
+
+### Order of operations
+
+Commit your data first, then acknowledge. If you acknowledge before your own
+processing is durable, and your application then fails, the acknowledged updates
+are gone and cannot be re-delivered.
+
+### Semantics
+
+`ACKNOWLEDGE` is:
+
+*   **Monotone.** Acknowledging a timestamp at or below the current position has
+    no effect. It is not an error, so retrying is safe.
+
+*   **Idempotent.** Sending the same acknowledgement twice is indistinguishable
+    from sending it once.
+
+*   **Not transactional.** The acknowledgement takes effect immediately and is
+    not undone by `ROLLBACK`. This is deliberate: your data really was
+    committed, so rolling back must not un-acknowledge it.
+
+Acknowledging a timestamp above the current time of the target object is an
+error.
+
+### Where you can run it
+
+`ACKNOWLEDGE` may be run on the same connection as the subscription, interleaved
+between `FETCH` statements, or on a separate connection. Because a subscription
+is named, no coordination between connections is needed, and because
+acknowledgements are monotone, they cannot arrive out of order in any way that
+matters.
+
+Running `ACKNOWLEDGE` while no one is reading the subscription is allowed. This
+matters for the separate-connection case, where the reading connection may drop
+while an acknowledgement is in flight.
+
+### Effect on storage
+
+The acknowledged position determines how much history Materialize retains for
+the subscription. Acknowledging more often releases storage sooner; acknowledging
+less often reduces round trips but retains more. Materialize records the position
+durably on a short interval rather than on every statement, so history is
+released slightly after you acknowledge.
+
+## Examples
+
+```mzsql
+ACKNOWLEDGE DURABLE SUBSCRIPTION winning_bids_feed AT 1723459200000;
+```
+
+Acknowledging from a progress message received on the same connection:
+
+```mzsql
+FETCH ALL c WITH (timeout = '1s');
+--  mz_timestamp  | mz_progressed | mz_diff | ...
+--  1723459200000 | t             |         |
+ACKNOWLEDGE DURABLE SUBSCRIPTION winning_bids_feed AT 1723459200000;
+```
+
+## Related pages
+
+*   [`CREATE DURABLE SUBSCRIPTION`](/sql/create-durable-subscription/)
+*   [`SUBSCRIBE`](/sql/subscribe/)

doc/user/content/sql/acknowledge.md

@@ -0,0 +1,65 @@
+---
+title: "ALTER DURABLE SUBSCRIPTION"
+description: "`ALTER DURABLE SUBSCRIPTION` changes the time to live of a durable subscription, or resets one that has expired."
+menu:
+  main:
+    parent: 'commands'
+---
+
+`ALTER DURABLE SUBSCRIPTION` changes the time to live of a [durable
+subscription](/sql/create-durable-subscription/), or resets one that has
+expired.
+
+## Syntax
+
+```mzsql
+ALTER DURABLE SUBSCRIPTION <name> SET (TTL = <interval>);
+ALTER DURABLE SUBSCRIPTION <name> RESET;
+```
+
+| Field | Use |
+| --- | --- |
+| `SET (TTL = <interval>)` | Change how long you have to acknowledge. Takes effect immediately, including for a subscription that is currently behind. |
+| `RESET` | Re-arm an expired subscription at the current time. |
+
+## Details
+
+### Setting the time to live
+
+Increasing the `TTL` gives a reader more time to recover, and increases the
+history that may be retained on its behalf. Decreasing it can expire a
+subscription immediately, if the reader is already further behind than the new
+value allows.
+
+The `TTL` must remain above the system-wide minimum, which exists because the
+acknowledged position is recorded durably on an interval. A time to live close to
+that interval would expire readers that are acknowledging correctly.
+
+### Resetting an expired subscription
+
+`RESET` moves an expired subscription back to the current time and makes it
+usable again. It does not recover the history that was released when the
+subscription expired, so the next read must request a snapshot.
+
+Use `RESET` rather than dropping and recreating: it preserves the subscription's
+name, owner, and privileges. Requiring it, instead of silently resuming from
+whatever history happens to remain, is what keeps a gap in the data from passing
+unnoticed.
+
+`RESET` on a subscription that has not expired is an error. Fencing a live reader
+by resetting its position is not something to do by accident.
+
+## Examples
+
+```mzsql
+ALTER DURABLE SUBSCRIPTION winning_bids_feed SET (TTL = '5m');
+```
+
+```mzsql
+ALTER DURABLE SUBSCRIPTION winning_bids_feed RESET;
+```
+
+## Related pages
+
+*   [`CREATE DURABLE SUBSCRIPTION`](/sql/create-durable-subscription/)
+*   [`DROP DURABLE SUBSCRIPTION`](/sql/drop-durable-subscription/)