Skip to content

Commit 1b91fd4

Browse files
maheshwaripclaude
andauthored
docs: Complete the scoped freshness CCDF query (#38378)
### Motivation On [How to monitor freshness](https://materialize.com/docs/transform-data/monitor-freshness/#summarize-freshness-with-a-ccdf), the last code block in the "Summarize freshness with a CCDF" section is incomplete. It shows only the middle of the statement (the object-name placeholder below is spelled `MV_NAME` here, and appears in the page as that name wrapped in angle brackets): ```mzsql FROM mz_internal.mz_wallclock_global_lag_recent_history wl JOIN mz_catalog.mz_objects o ON wl.object_id = o.id WHERE o.name = 'MV_NAME' AND wl.lag IS NOT NULL AND wl.lag > INTERVAL '0' ``` It starts mid-statement at `FROM`, drops the `SELECT extract(epoch FROM wl.lag) AS lag_seconds` line that the `FROM` belongs to, and has no enclosing `WITH lags AS (`, no `thresholds` CTE, and no outer aggregation. Running it verbatim fails: ``` ERROR: Unexpected keyword FROM at the beginning of a statement LINE 1: FROM mz_internal.mz_wallclock_global_lag_recent_history wl ^ ``` A reader has to reassemble the statement by hand from the aggregate query further up the page. Every other code block on this page is a complete, runnable statement, so this one is the odd one out. ### Description Show the whole statement in the block, so it can be copied and run directly, and reword the lead-in sentence so it describes the query that follows rather than reading as a patch instruction ("add a join ... to the `lags` CTE" becomes "join `mz_catalog.mz_objects` in the `lags` CTE and filter on the object name"). The SQL is unchanged from what the fragment implied: the `lags` CTE gains the join to `mz_catalog.mz_objects` and the object-name filter, and the rest of the query matches the aggregate version above it. `mz_wallclock_global_lag_recent_history.object_id` is documented as corresponding to `mz_objects.id`, so the join key is correct. The placeholder in the page is left exactly as it was. ### Verification Manual, against a local `environmentd` built from this branch with a materialized view producing wallclock lag history. - The block as it appears on the published page fails with `ERROR: Unexpected keyword FROM at the beginning of a statement` (exit 3). - The block extracted programmatically from the edited markdown runs clean and returns the three-row shape the surrounding prose describes: ``` lag_threshold_seconds | fraction_of_time_at_or_above -----------------------+------------------------------ 1 | 1 10 | 0 100 | 0 (3 rows) ``` - Re-ran the page's other two `mzsql` blocks (the per-object lag history query and the aggregate CCDF) to confirm they were already complete and still run clean. Docs-only change, so no automated tests were added or modified. ### Release notes No release note. This is a documentation-only fix with no user-visible product change. Co-authored-by: Claude <noreply@anthropic.com>
1 parent f9be4ed commit 1b91fd4

1 file changed

Lines changed: 18 additions & 2 deletions

File tree

doc/user/content/transform-data/monitor-freshness.md

Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -128,15 +128,31 @@ busier instance under real load might show non-zero fractions at those higher
128128
thresholds.
129129

130130
By default this query aggregates across every object. To scope the CCDF to a
131-
single object, add a join to `mz_catalog.mz_objects` and a name filter to the
132-
`lags` CTE (replace `<your_mv_name>` with the name of your object):
131+
single object, join `mz_catalog.mz_objects` in the `lags` CTE and filter on the
132+
object name (replace `<your_mv_name>` with the name of your object):
133133

134134
```mzsql
135+
WITH lags AS (
136+
-- Convert each lag to seconds, dropping unhydrated (NULL) observations and
137+
-- any non-positive lag.
138+
SELECT extract(epoch FROM wl.lag) AS lag_seconds
135139
FROM mz_internal.mz_wallclock_global_lag_recent_history wl
136140
JOIN mz_catalog.mz_objects o ON wl.object_id = o.id
137141
WHERE o.name = '<your_mv_name>'
138142
AND wl.lag IS NOT NULL
139143
AND wl.lag > INTERVAL '0'
144+
),
145+
thresholds AS (
146+
-- Fixed decade thresholds, so the CCDF always reports 1s, 10s, and 100s.
147+
SELECT unnest(ARRAY[1, 10, 100]) AS lag_threshold_seconds
148+
)
149+
SELECT
150+
t.lag_threshold_seconds,
151+
count(*) FILTER (WHERE l.lag_seconds >= t.lag_threshold_seconds)::float8
152+
/ count(*) AS fraction_of_time_at_or_above
153+
FROM thresholds t, lags l
154+
GROUP BY t.lag_threshold_seconds
155+
ORDER BY t.lag_threshold_seconds;
140156
```
141157

142158
To compare against an SLO, pick your target freshness (say 10 seconds) and read

0 commit comments

Comments
 (0)