Skip to content

Commit 780c9c1

Browse files
antiguruclaude
antiguru
and
claude
authored
compute: expose replica resource usage observations (#38250)
### Motivation Replica resource usage is only visible as periodic orchestrator samples, roughly one per minute, landing in `mz_cluster_replica_metrics_history`. A spike between two samples is invisible, so the usage of a hydration episode that starts and finishes inside one sampling gap cannot be recovered at all. This is the replica-local, higher-cadence half of the Improved Hydration Visibility work. ### Description `mz_metrics::usage` reads cgroup v2 interface files, `getrusage`, `/proc/self/status` and `statvfs` on the existing periodic metrics task. A new `ComputeLog::ResourceUsage` logging dataflow reports them as `mz_introspection.mz_cluster_replica_resource_usage(process_id, source, metric, value)`, following the `ComputeLog::PrometheusMetrics` shape: usage is per-process, so one worker per process reports and the rest drop their capability. Three decisions the diff cannot show. **The relation surfaces uninterpreted observations.** It does not decide which number is "the" memory usage of a replica, and it never combines two sources into a third figure. Sources measure overlapping but distinct quantities: `cgroup memory.current` is the accounting that limit enforcement and the OOM killer act on, while `VmRSS` is the process's resident set. They disagree by a measured ~96 MiB on staging, in the direction of `VmRSS` being the larger, because a page is charged to whichever cgroup first faulted it in and this binary's resident text was faulted in by the runtime that unpacked the image. That disagreement is the diagnostic signal, and it is not recoverable once two sources have been folded into one number. Combining sources belongs in views, where it can change without a migration and where a reader can see what was combined. For what fusing costs, note that `mz_cluster_replica_metrics.disk_bytes` currently reports disk *plus swap* because a consumer wanted it displayed that way, and the fix is blocked on porting that consumer. **Long rows, not one column per metric.** A wide row retracts and re-inserts on any field's change, so its churn is set by its noisiest field: a filesystem-usage number that moves every sample would drag a memory peak that has not moved in an hour through a retraction every time. Long rows churn only what changed. They also make a missing observation representable as an absent row rather than an in-band sentinel, and let a new metric be added as kernels gain interface files without a catalog migration. **Peaks are reported, not derived.** A peak cannot be recovered downstream from this relation: each sample retracts the previous one, so a `reduce` computing a maximum sees one value per key and returns it. Retaining the samples instead is not an option, because a compute log collection is an arrangement with no retention, window or TTL mechanism available to it, so a retained history would grow unbounded for the process's lifetime. Most peaks cost nothing to report because the kernel maintains them: `cgroup memory.peak` and `memory.swap.peak` are exact high-water marks of the cgroup accounting, kept per cgroup for the container's lifetime and therefore unaffected by how rarely we read them, by the sampler being disabled and restarted, or by `environmentd` restarting; `ru_maxrss` is already a high-water mark. Only a source with no kernel-side peak gets one folded in the sampler, currently the scratch filesystem and swap below the kernel version providing `memory.swap.peak`, and those are reported under a distinct `_peak` metric name so a reader can tell an exact peak from a sampled one. The logging operator downgrades its capability on its own timer rather than on the sampler's. Were the sampler to drive progress, disabling it (which `mz_metrics_usage_refresh_interval = 0` is meant to allow) would freeze the collection's frontier and wedge every query over the relation along with the `mz_catalog_server` indexes on it. Cadence is a new dyncfg, `mz_metrics_usage_refresh_interval` (5s), kept separate from `memory_limiter_interval`, which governs OOM-kill behavior and must not be retuned for introspection's sake. #### Cost No source read here is proportional to heap size. `VmRSS` and `VmSwap` come from per-mm counters and the cgroup figures are `page_counter` reads. Measured on a 32-core Linux host, p50 over 2000 iterations: | read | empty heap | 32 GiB resident | |---|---|---| | `getrusage(RUSAGE_SELF)` | 0.22 us | 0.22 us | | `statvfs` | 0.41 us | 0.41 us | | `/proc/self/status` | 4.67 us | 4.74 us | | `/proc/self/smaps_rollup` | 9.6 us | 137 ms | | `/proc/self/numa_maps` | 20.8 us | 142 ms | The last two rows are not read by this path and must not be added to it. They walk page tables, so their cost scales with resident pages and rises superlinearly. Held at 16 GiB resident, `numa_maps` costs 28.9 ms over one fully-touched VMA, 11.0 ms over 8192 VMAs, and 1.5 ms when only every 64th page is written, confirming the cost tracks page-table entries rather than VMA count. At the 5s default the whole sampler is roughly 0.0002% of one core. #### Incidental refactors `ProcStatus` and the `statvfs` helper move from `mz_compute::memory_limiter` to `mz_metrics::usage`, so the limiter, the clusterd usage collector and the sampler share one reader. The cgroup helpers move from `mz_ore::memory` to `mz_ore::cgroup` and lose their `target_os = "linux"` gate, since a second crate now uses them and the file is pure `std`. The module gains a `CgroupV2` handle that resolves the cgroup directory once and reads its interface files, rather than this becoming a fourth copy of the `/proc/self/cgroup` parse. It exposes the resolved path rather than logging it, so the module keeps its property of depending on nothing outside `std`: it is compiled unconditionally now, including into the wasm32 build where `mz_ore`'s optional dependencies are absent. The three existing cgroup *limit* readers (`usage_metrics.rs`, `ore::cgroup`, `service::boot`) are deliberately left alone; consolidating them is worth doing separately. The Prometheus metrics operator reports a sampled source in the same shape, so the interval-boundary capability downgrade and the snapshot diff become `downgrade_to_interval_boundary` and `emit_snapshot_diff` in `mz_compute::logging`, and both operators call them. Docs: the relation is described in `mz_introspection.md`, and the sources, the metrics they report, how to interpret them and an example query live on a new page, `manage/monitor/replica-resource-usage.md`. Adding a builtin log changes the inlined-VALUES fingerprints of `mz_catalog.mz_indexes` and `mz_catalog.mz_sources`, so both get a `MigrationStep::replacement` at the current dev version. ### Verification - Unit tests in `mz_metrics::usage`: a sample is non-empty, a derived peak rises with a higher observation and survives a lower one, an unreadable source is absent rather than zero, and a derived peak is only published for a source that was actually read. - `test-resource-usage` in `test/cluster/mzcompose.py`: asserts both processes of a 2-process replica report, that they report the same metric set, that `rusage` and `proc_status` are present, and that no peak goes backwards across a workload. cgroup metrics are deliberately not asserted, since `memory.peak` and `memory.swap.peak` depend on the kernel version. - Catalog goldens regenerated. Note that `autogenerated/mz_introspection.slt` needed `ci/test/lint-docs-catalog.sh --rewrite`, not just `--rewrite-results`: the relation name is baked into the query text, and a plain rewrite silently produced an empty result. - Verified on staging (eu-west-1). `memory_peak` and `swap_peak` are both present on the node kernel, and `cgroup memory_current` matches cAdvisor's `container_memory_working_set_bytes` for the same container to within 1.5% across every pod, so the cgroup directory resolution is correct. - A replica hydrating a 60M-row index against a 3.79 GiB RAM limit reported `memory_current` 1.20 GiB / `swap_current` 695 MiB while `memory_peak` was 3.79 GiB and `swap_peak` 3.98 GiB: the instantaneous readings understate the episode by 3.2x and 5.9x. Sampled again minutes later, `memory_current` had risen 2.46 GiB and `swap_current` fallen 616 MiB as pages swapped back in, with both peaks byte-identical. That is the monotonicity property under a bidirectional swing. - `VmRSS` runs a roughly constant 96 MiB above the replica's own cgroup charge (7% spread across replicas spanning 36 MiB to 400 MiB of `memory_current`), because this binary's resident text is charged to the runtime that unpacked the image. Taking `ru_maxrss` as the replica's footprint, as the previous design did, overstates the enforced figure by 2.75x on an idle replica and 1.20x on a loaded one. - `memory.events`' `max` stayed 0 even with the cgroup pinned at its ceiling, because reclaim succeeded by swapping rather than failing. It is therefore not a reliable limit-hit signal where swap is configured; `memory_peak` reaching `memory_max` is. Both are reported without interpretation. ### Release notes This release adds `mz_introspection.mz_cluster_replica_resource_usage`, reporting the resource usage of each cluster replica process as measured by each available source, including cgroup memory and swap usage and their kernel-maintained peaks. Co-authored-by: Claude <noreply@anthropic.com>
1 parent 7a7a306 commit 780c9c1

45 files changed

Lines changed: 1291 additions & 237 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎Cargo.lock‎

Lines changed: 2 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎doc/user/content/manage/monitor/replica-resource-usage.md‎

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
---
2+
title: "Replica resource usage"
3+
description: "The per-process memory, swap and disk observations a cluster replica reports about itself, and how to interpret them."
4+
menu:
5+
main:
6+
parent: "monitor"
7+
identifier: "monitor-replica-resource-usage"
8+
weight: 18
9+
---
10+
11+
Every process of a cluster replica reports its own resource usage through
12+
[`mz_introspection.mz_cluster_replica_resource_usage`](/reference/system-catalog/mz_introspection/#mz_cluster_replica_resource_usage).
13+
Each row is one measurement, taken from one source, reported as that source
14+
gave it. Sources measure overlapping but distinct quantities, and the
15+
differences between them are informative, so no row is a combination of two
16+
others. Deciding which number is "the" memory usage of a replica, or how close
17+
it is to its limit, is left to queries over the relation.
18+
19+
A metric whose name ends in `peak` is a high-water mark since the process
20+
started, and the rest are instantaneous. Peaks the operating system maintains
21+
itself are exact, and are unaffected by how often the replica reads them. Peaks
22+
folded from samples are marked as such below and can miss a spike shorter than
23+
the sampling interval, which makes them lower bounds. An observation the
24+
replica could not read is absent rather than zero, so which metrics appear
25+
depends on the platform and the kernel version.
26+
27+
## Sources
28+
29+
| Source | Reads | Measures |
30+
|---------------|-----------------------------------------|-------------------------------------------------------------------------------------------|
31+
| `cgroup` | the process's cgroup v2 interface files | the whole container, and the accounting that limit enforcement and the OOM killer act on |
32+
| `proc_status` | `/proc/self/status` | this process only, with resident memory broken down by backing |
33+
| `rusage` | `getrusage(RUSAGE_SELF)` | this process only |
34+
| `statvfs` | the replica's scratch filesystem | the filesystem as a whole, absent where disk is provided as swap |
35+
36+
## Metrics
37+
38+
| Source | Metric | Meaning |
39+
|---------------|-------------------|--------------------------------------------------------------------------------------------------------------------------------|
40+
| `cgroup` | `memory_current` | Memory charged to the cgroup: anonymous, page cache, kernel and socket memory. |
41+
| `cgroup` | `memory_peak` | High-water mark of `memory_current`, maintained by the kernel. |
42+
| `cgroup` | `memory_max` | The cgroup's memory limit. |
43+
| `cgroup` | `swap_current` | Swap charged to the cgroup, including pages already read back whose swap slot is still allocated. |
44+
| `cgroup` | `swap_peak` | High-water mark of `swap_current`, maintained by the kernel. |
45+
| `cgroup` | `swap_max` | The cgroup's swap limit. |
46+
| `cgroup` | `anon` | The part of `memory_current` backed by no file. |
47+
| `cgroup` | `file` | Page cache charged to the cgroup. |
48+
| `cgroup` | `shmem` | Shared memory and tmpfs pages. |
49+
| `cgroup` | `swapcached` | Pages resident in memory whose swap slot is still allocated. Counted in both `anon` and `swap_current`. |
50+
| `cgroup` | `kernel` | Kernel memory charged to the cgroup. |
51+
| `cgroup` | `slab` | Kernel slab allocations, part of `kernel`. |
52+
| `cgroup` | `sock` | Socket buffer memory. |
53+
| `cgroup` | `events_max` | Times an allocation hit `memory_max`. See the caveat below before using this as a limit-hit signal. |
54+
| `cgroup` | `events_oom_kill` | Processes in the cgroup killed by the OOM killer. |
55+
| `proc_status` | `vm_rss` | Resident set size of this process, the sum of `rss_anon`, `rss_file` and `rss_shmem`. |
56+
| `proc_status` | `rss_anon` | Resident memory backed by no file. The replica's own memory. |
57+
| `proc_status` | `rss_file` | Resident file-backed memory, largely this binary's text. Shared between replicas and charged to whichever cgroup first faulted it in. |
58+
| `proc_status` | `rss_shmem` | Resident shared memory. |
59+
| `proc_status` | `vm_swap` | This process's pages currently in swap. Excludes swap-cached pages, so it reads below `cgroup` `swap_current`. |
60+
| `proc_status` | `vm_swap_peak` | Maximum `vm_swap` over samples, so a lower bound on the true peak. |
61+
| `proc_status` | `heap` | `vm_rss` plus `vm_swap`, the quantity a replica is limited on. |
62+
| `proc_status` | `heap_peak` | Maximum `heap` over samples, so a lower bound on the true peak. |
63+
| `rusage` | `max_rss` | Peak resident set size. Maintained by the kernel, but refreshed only at internal checkpoints, so it can read below a concurrent `vm_rss`. |
64+
| `statvfs` | `fs_used` | Used bytes of the filesystem, which on a shared filesystem counts writes this replica never made. |
65+
| `statvfs` | `fs_used_peak` | Maximum `fs_used` over samples, so a lower bound on the true peak. |
66+
67+
## Interpreting
68+
69+
Values from different sources are not interchangeable, and adding them together
70+
generally produces a number that means nothing. In particular:
71+
72+
* **How close is this replica to its memory limit?** Compare `cgroup`
73+
`memory_current` against `memory_max`. **Did it ever reach it?** Compare
74+
`memory_peak` against `memory_max`. A `memory_peak` at the limit means the
75+
replica ran out of RAM and spilled to swap, even if the current reading is
76+
comfortable.
77+
* **How close is it to its heap limit?** Compare `proc_status` `heap` against
78+
[`mz_internal.mz_cluster_replica_metrics`](/reference/system-catalog/mz_internal/#mz_cluster_replica_metrics)'s
79+
`heap_limit`. `heap_peak` bounds the high-water mark from below, and no source
80+
bounds it from above: the `cgroup` peaks describe a smaller quantity, and
81+
`max_rss` lags.
82+
* **Do not use `events_max` as a limit-hit signal.** Where swap is configured it
83+
stays at zero even for a replica pinned at its ceiling, because reclaim
84+
succeeds by swapping instead of failing. `events_oom_kill` does report kills.
85+
* **How much memory does this replica itself account for?** Use `proc_status`
86+
`rss_anon`. Do not use `vm_rss`: it includes `rss_file`, which is charged to
87+
another cgroup and so runs a roughly constant amount above the replica's own
88+
charge.
89+
* **Do not add `memory_current` and `swap_current`.** A page read back from swap
90+
is counted in both, and `swapcached` reports how much is in that state.
91+
* **Where disk is provided as swap**, disk usage appears as `swap_current` and
92+
there are no `statvfs` rows at all.
93+
* **These readings live and die with the replica process.** A restart resets
94+
every peak. For history that survives restarts, see
95+
[`mz_internal.mz_cluster_replica_metrics_history`](/reference/system-catalog/mz_internal/#mz_cluster_replica_metrics_history).
96+
97+
## Example
98+
99+
Introspection relations are replica-local: a query reads the replica that
100+
serves it, so pin both the cluster and the replica. This reports how close each
101+
process came to a memory-limiter kill, comparing the quantity the limiter
102+
enforces against the limit it enforces:
103+
104+
```mzsql
105+
SET cluster = <cluster_name>;
106+
SET cluster_replica = <replica_name>;
107+
108+
SELECT
109+
u.process_id,
110+
round((max(u.value) FILTER (WHERE u.metric = 'heap'))::numeric / 1073741824, 2) AS heap_gib,
111+
round((max(u.value) FILTER (WHERE u.metric = 'heap_peak'))::numeric / 1073741824, 2) AS heap_peak_gib,
112+
round(m.heap_limit::numeric / 1073741824, 2) AS limit_gib,
113+
round(100 * (max(u.value) FILTER (WHERE u.metric = 'heap_peak'))::numeric / m.heap_limit, 1) AS peak_pct
114+
FROM mz_introspection.mz_cluster_replica_resource_usage u
115+
JOIN mz_cluster_replicas r
116+
ON r.name = current_setting('cluster_replica')
117+
AND r.cluster_id = (SELECT id FROM mz_clusters WHERE name = current_setting('cluster'))
118+
JOIN mz_internal.mz_cluster_replica_metrics m
119+
ON m.replica_id = r.id AND m.process_id = u.process_id
120+
WHERE u.source = 'proc_status'
121+
GROUP BY u.process_id, m.heap_limit
122+
ORDER BY u.process_id;
123+
```
124+
125+
`peak_pct` is a lower bound, because `heap_peak` is: a spike shorter than the
126+
sampling interval can slip through it. No source provides a matching upper
127+
bound. The query stays within `proc_status` deliberately. The `cgroup` metrics
128+
measure a different quantity, and `memory_current` plus `swap_current`
129+
double-counts every swap-cached page.

‎doc/user/content/reference/system-catalog/mz_internal.md‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -204,6 +204,13 @@ The `mz_cluster_replica_metrics` view gives the last known CPU and RAM utilizati
204204
for all processes of all extant cluster replicas.
205205

206206
At this time, we do not make any guarantees about the exactness or freshness of these numbers.
207+
They are sampled roughly once a minute, so a spike shorter than the sampling interval is not
208+
visible here at all. For a view of a single replica sampled every few seconds, including high-water
209+
marks that survive a spike the sampling missed, see [Replica resource
210+
usage](/manage/monitor/replica-resource-usage/).
211+
212+
Where a replica's disk is provided as swap rather than as a filesystem, `disk_bytes` reports swap
213+
usage.
207214

208215
<!-- RELATION_SPEC mz_internal.mz_cluster_replica_metrics -->
209216
| Field | Type | Meaning
@@ -223,6 +230,11 @@ The `mz_cluster_replica_metrics_history` table records resource utilization metr
223230
for all processes of all extant cluster replicas.
224231

225232
At this time, we do not make any guarantees about the exactness or freshness of these numbers.
233+
They are sampled roughly once a minute, so a spike shorter than the sampling interval leaves no
234+
trace. Unlike
235+
[`mz_introspection.mz_cluster_replica_resource_usage`](/reference/system-catalog/mz_introspection/#mz_cluster_replica_resource_usage),
236+
which is sampled every few seconds but is replica-local and resets when a replica restarts, this
237+
history is retained across restarts.
226238

227239
<!-- RELATION_SPEC mz_internal.mz_cluster_replica_metrics_history -->
228240
| Field | Type | Meaning

‎doc/user/content/reference/system-catalog/mz_introspection.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -170,6 +170,27 @@ Summaries are flattened into separate quantile, sum, and count rows.
170170
| `value` | [`double precision`] | The numeric value of the metric. |
171171
| `help` | [`text`] | The help string describing the metric. |
172172

173+
## `mz_cluster_replica_resource_usage`
174+
175+
The `mz_cluster_replica_resource_usage` source reports the resource usage of each process of a
176+
cluster replica, as one row per measurement source and metric. Each row is what that source
177+
reported, without interpretation: sources measure overlapping but distinct quantities, so combining
178+
them into a single figure for memory usage, or for how close a replica is to its limit, is left to
179+
queries over this relation.
180+
181+
The replica samples its sources every few seconds, and reports a high-water mark alongside the
182+
instantaneous value where one is available. For the sources and metrics that appear here, how to
183+
interpret them, and an example query, see [Replica resource
184+
usage](/manage/monitor/replica-resource-usage/).
185+
186+
<!-- RELATION_SPEC mz_introspection.mz_cluster_replica_resource_usage NO_COMMENTS -->
187+
| Field | Type | Meaning |
188+
|--------------|-----------|----------------------------------------------------------------------|
189+
| `process_id` | [`uint8`] | The ID of the process within the replica. |
190+
| `source` | [`text`] | The measurement source, for example `cgroup` or `rusage`. |
191+
| `metric` | [`text`] | What the source measured, for example `memory_current`. |
192+
| `value` | [`uint8`] | The reported value, in bytes for a size and as a count otherwise. |
193+
173194
## `mz_dataflows`
174195

175196
The `mz_dataflows` view describes the [dataflows][dataflow] in the system.

‎doc/user/data/metrics.yml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1402,6 +1402,13 @@ metrics:
14021402
help: user CPU time used
14031403
source: src/metrics/src/rusage.rs
14041404
visibility: internal
1405+
- name: mz_metrics_resource_usage
1406+
help: Resource usage observations, by source and metric.
1407+
labels:
1408+
- metric
1409+
- source
1410+
source: src/metrics/src/usage.rs
1411+
visibility: internal
14051412
- name: mz_metrics_update_duration_bucket
14061413
help: The time it took to update lgalloc stats
14071414
labels:

‎misc/python/materialize/mzcompose/__init__.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -800,6 +800,7 @@ def get_default_system_parameters(
800800
"mz_metrics_lgalloc_map_refresh_interval",
801801
"mz_metrics_lgalloc_refresh_interval",
802802
"mz_metrics_rusage_refresh_interval",
803+
"mz_metrics_usage_refresh_interval",
803804
"compute_peek_response_stash_batch_max_runs",
804805
"compute_peek_response_stash_read_batch_size_bytes",
805806
"compute_peek_response_stash_read_memory_budget_bytes",

‎misc/python/materialize/parallel_workload/action.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3388,6 +3388,7 @@ def __init__(
33883388
"mz_metrics_lgalloc_map_refresh_interval",
33893389
"mz_metrics_lgalloc_refresh_interval",
33903390
"mz_metrics_rusage_refresh_interval",
3391+
"mz_metrics_usage_refresh_interval",
33913392
"compute_peek_stash_num_batches",
33923393
"compute_peek_stash_batch_size",
33933394
"compute_peek_response_stash_batch_max_runs",

‎src/adapter/src/catalog/open/builtin_schema_migration.rs‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -424,6 +424,23 @@ static MIGRATIONS: LazyLock<Vec<MigrationStep>> = LazyLock::new(|| {
424424
MZ_CATALOG_SCHEMA,
425425
"mz_views",
426426
),
427+
// Required because we added the `mz_cluster_replica_resource_usage` builtin log.
428+
// make_mz_indexes and make_mz_sources inline the builtin-log set as
429+
// VALUES, so adding one changes both MVs' SQL fingerprints. See the NOTE
430+
// above: this version must stay at the workspace's current dev version
431+
// until the change ships.
432+
MigrationStep::replacement(
433+
"26.40.0-dev.0",
434+
CatalogItemType::MaterializedView,
435+
MZ_CATALOG_SCHEMA,
436+
"mz_indexes",
437+
),
438+
MigrationStep::replacement(
439+
"26.40.0-dev.0",
440+
CatalogItemType::MaterializedView,
441+
MZ_CATALOG_SCHEMA,
442+
"mz_sources",
443+
),
427444
]
428445
});
429446

‎src/catalog/src/builtin.rs‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1108,6 +1108,7 @@ pub static BUILTINS_STATIC: LazyLock<Vec<Builtin<NameReference>>> = LazyLock::ne
11081108
Builtin::Log(&MZ_COMPUTE_EXPORTS_PER_WORKER),
11091109
Builtin::Log(&MZ_COMPUTE_DATAFLOW_GLOBAL_IDS_PER_WORKER),
11101110
Builtin::Log(&MZ_CLUSTER_PROMETHEUS_METRICS),
1111+
Builtin::Log(&MZ_CLUSTER_REPLICA_RESOURCE_USAGE),
11111112
Builtin::Log(&MZ_MESSAGE_COUNTS_RECEIVED_RAW),
11121113
Builtin::Log(&MZ_MESSAGE_COUNTS_SENT_RAW),
11131114
Builtin::Log(&MZ_MESSAGE_BATCH_COUNTS_RECEIVED_RAW),

‎src/catalog/src/builtin/mz_introspection.rs‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -248,6 +248,21 @@ pub static MZ_CLUSTER_PROMETHEUS_METRICS: LazyLock<BuiltinLog> = LazyLock::new(|
248248
}),
249249
});
250250

251+
pub static MZ_CLUSTER_REPLICA_RESOURCE_USAGE: LazyLock<BuiltinLog> = LazyLock::new(|| BuiltinLog {
252+
name: "mz_cluster_replica_resource_usage",
253+
schema: MZ_INTROSPECTION_SCHEMA,
254+
oid: oid::LOG_MZ_CLUSTER_REPLICA_RESOURCE_USAGE_OID,
255+
variant: LogVariant::Compute(ComputeLog::ResourceUsage),
256+
access: vec![PUBLIC_SELECT],
257+
ontology: Some(Ontology {
258+
entity_name: "cluster_replica_resource_usage",
259+
description: "Resource usage of each process of the cluster replica, as reported by each \
260+
measurement source.",
261+
links: &const { [] },
262+
column_semantic_types: &[],
263+
}),
264+
});
265+
251266
pub static MZ_COMPUTE_FRONTIERS_PER_WORKER: LazyLock<BuiltinLog> = LazyLock::new(|| BuiltinLog {
252267
name: "mz_compute_frontiers_per_worker",
253268
schema: MZ_INTROSPECTION_SCHEMA,

0 commit comments

Comments
 (0)