Skip to content

Commit 790ee3f

Browse files
committed
doc: point at the lifecycle event log rather than a written_at column
The compute change took a different shape than the one recorded here. Rather than redefining `hydrated_at` and adding a `written_at` column, #38403 adds an append-only lifecycle log and leaves `mz_compute_hydration_times_per_worker` untouched. So there is no semantic shift for rows this collector has already written, and no ordering constraint between that change and enabling collection in production. What survives is the guidance that recording gates on hydration rather than on a write stage, and a note that the event log carries the as-of, which this table does not, and without which a duration does not say how much work was done. Ref: SQL-644
1 parent 61ff662 commit 790ee3f

1 file changed

Lines changed: 14 additions & 15 deletions

File tree

‎doc/developer/design/20260817_durable_object_hydration_history.md‎

Lines changed: 14 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -136,21 +136,20 @@ hydrating is skipped and picked up by a later sweep. The cutoff and the anti-joi
136136
apply to the aggregate's output, since as `WHERE` clauses either one would drop the
137137
unfinished rows and make that check trivially true.
138138

139-
Compute intends to split this stamp in two (CPU-226): `hydrated_at` redefined as
140-
the dataflow stage, uniform across workers and across read-only mode, and a new
141-
`written_at` for the durable stage. `hydrated_at` stays the stamp that gates
142-
recording here. A materialized view being replaced runs read-only and does not write
143-
until cutover, so gating on a write stamp would never record the hydration a
144-
deployment most wants to measure, and would wait forever on a replacement that is
145-
rolled back. `written_at` arrives as an additive nullable column, which schema
146-
evolution keeps the existing rows through.
147-
148-
That split changes what a recorded `hydrated_at` means for a materialized view,
149-
from the durable stage to the dataflow stage, and rows already written are not
150-
relabelled. The staged rollout is what makes that cheap: if the split lands before
151-
production is enabled, only CI and staging ever hold the older meaning. Once the
152-
stamp is uniform across workers, the aggregate can also collapse back to a single
153-
worker, which removes the skew below.
139+
Compute is adding an append-only lifecycle log for the same stages, currently
140+
proposed in #38403: one row per export, worker and event, with a reason and the
141+
dataflow's as-of. It leaves `mz_compute_hydration_times_per_worker` alone, so
142+
everything above keeps its meaning and nothing recorded here is relabelled.
143+
144+
That log is the signal this design says is missing, and moving onto it later buys
145+
three things at once: stages that mean the same thing on every worker, a reason
146+
that distinguishes a replacement waiting for cutover from an index that will never
147+
write, and the as-of, without which an interval says nothing about how much work
148+
was done, since a replacement with a far behind as-of does far more of it in the
149+
same wall-clock time. Recording should keep gating on hydration when that happens.
150+
A replacement runs read-only and does not write until cutover, so gating on a write
151+
stage would never record the hydration a deployment most wants to measure, and
152+
would wait forever on a rollback.
154153

155154
Two consequences. An interval can span workers in different processes, each
156155
anchoring its logging clock at its own `SystemTime`, so skew inflates a duration.

0 commit comments

Comments
 (0)