Skip to content

Commit 03e5ca1

Browse files
committed
adapter: collect durable replica hydration history
Extend the hydration history sweep to record the latest completed replica episode after collecting per-object rows. Derive episode boundaries from connected object hydration intervals and wait for every configured process before snapshotting resource peaks. Use the maximum cgroup memory peak and filesystem or swap peak across processes, retain rows with the existing 30-day policy, and cover incremental hydration, restart durability, deduplication, and retention.
1 parent 4ac3ffc commit 03e5ca1

6 files changed

Lines changed: 349 additions & 40 deletions

File tree

doc/user/data/metrics.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -963,7 +963,7 @@ metrics:
963963
source: src/adapter/src/metrics.rs
964964
visibility: internal
965965
- name: mz_hydration_history_retention_batch_full_total
966-
help: Total hydration-history sweeps whose retention batch was full. Repeated increments mean retention may not be keeping up with its schedule.
966+
help: Total hydration-history retention batches that were full. Repeated increments mean retention may not be keeping up with its schedule.
967967
source: src/adapter/src/metrics.rs
968968
visibility: internal
969969
- name: mz_hydration_history_rows_affected_total

src/adapter-types/src/dyncfgs.rs

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -421,19 +421,19 @@ pub const ARRANGEMENT_SIZE_HISTORY_RETENTION_PERIOD: Config<Duration> = Config::
421421
ParameterScope::Environment,
422422
);
423423

424-
/// How often to sweep replicas for completed object hydration episodes.
424+
/// How often to sweep replicas for completed object and replica hydration episodes.
425425
pub const HYDRATION_HISTORY_COLLECTION_INTERVAL: Config<Duration> = Config::new(
426426
"hydration_history_collection_interval",
427427
Duration::ZERO,
428-
"How often to record completed object hydration episodes. A zero duration disables collection.",
428+
"How often to record completed object and replica hydration episodes. A zero duration disables collection.",
429429
ParameterScope::Environment,
430430
);
431431

432-
/// How long to retain completed object hydration episodes.
432+
/// How long to retain completed object and replica hydration episodes.
433433
pub const HYDRATION_HISTORY_RETENTION_PERIOD: Config<Duration> = Config::new(
434434
"hydration_history_retention_period",
435435
Duration::from_hours(30 * 24),
436-
"How long to retain rows in mz_internal.mz_object_hydration_history.",
436+
"How long to retain rows in mz_internal.mz_object_hydration_history and mz_internal.mz_replica_hydration_history.",
437437
ParameterScope::Environment,
438438
);
439439

0 commit comments

Comments
 (0)