Skip to content

compute: add regression guards for the MV sink write path - #38679

Draft
antiguru wants to merge 4 commits into
mainfrom
claude/materialize-regression-test-24sbk7
Draft

compute: add regression guards for the MV sink write path#38679
antiguru wants to merge 4 commits into
mainfrom
claude/materialize-regression-test-24sbk7

Conversation

@antiguru

@antiguru antiguru commented Sep 5, 2026

Copy link
Copy Markdown
Member

The MV sink has two dyncfg-gated alternative implementations, enable_compute_correction_v2 for the correction buffer and enable_compute_sync_mv_sink for the operator structure. Nothing in CI guarded their performance: the criterion bench comparing the correction buffers never runs in CI, and no benchmark exercised the sync sink under a hydration backlog, where it starved CPU and, once that was fixed, exhausted memory.

This adds two guards.

A deterministic unit test, catch_up_work_is_linear, drives CorrectionV2 through the bench's hydration workloads (append, upsert, temporal filter) and counts structural work through the existing introspection logging hook instead of wall-clock time. Catching up through 4x more timestamps must cost at most 8x the work: linear scaling measures 4.0 to 4.2, quadratic would be 16. The workload generators move into a shared correction_workload module so the bench and the test cannot drift apart. The correction_v2 tests move out of line, as the module exceeds the inline size limit.

An MvSink feature benchmark family stresses the write path end to end on wall-clock and replica memory: MvSinkCatchUp replays many distinct timestamps after the view's cluster comes back online, MvSinkRetractions drives full-table rewrites against a hydrated view, and MvSinkTemporalFilter runs small ingest rounds against a large far-future update mass. The nightly run compares against the merge base with identical flags, so it catches code regressions and flag flips. The two flags can be A/B'd on one build with --this-params and --other-params, as documented on the scenario family.

The feature benchmark scenarios have not run against a real Materialize yet, so their scale constants may need tuning after the first nightly.

🤖 Generated with Claude Code


Generated by Claude Code

@antiguru antiguru added the ci-nightly PR CI control: also trigger Nightly label Sep 5, 2026 — with Claude
The MV sink has two dyncfg-gated alternative implementations,
`enable_compute_correction_v2` for the correction buffer and
`enable_compute_sync_mv_sink` for the operator structure. Nothing in CI
guarded their performance: the criterion bench comparing the correction
buffers never runs in CI, and no benchmark exercised the sync sink under a
hydration backlog, where it starved CPU and, once that was fixed, exhausted
memory.

Add a deterministic unit test that drives `CorrectionV2` through the
bench's hydration workloads (append, upsert, temporal filter) and counts
structural work through the existing introspection logging hook instead of
wall-clock time. Catching up through 4x more timestamps must cost at most
8x the work: linear scaling measures 4.0 to 4.2, quadratic would be 16.
The workload generators move into a shared module so the bench and the
test cannot drift apart.

Add an `MvSink` feature benchmark family that stresses the write path end
to end on wall-clock and replica memory: catch-up through many distinct
timestamps after the view's cluster comes back online, retraction-heavy
full-table rewrites, and small ingest rounds against a large far-future
update mass behind a temporal filter. The nightly run compares against the
merge base; the two flags can be A/B'd on one build with `--this-params`
and `--other-params`.

Move the `correction_v2` tests out of line, as they exceed the size limit
for inline test modules.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GAf7V5GxLQRwhN8siyQtHn
@antiguru
antiguru force-pushed the claude/materialize-regression-test-24sbk7 branch from 37434e5 to 54606a3 Compare September 5, 2026 14:34

antiguru commented Sep 5, 2026

Copy link
Copy Markdown
Member Author

Nightly build #18249 on 54606a3 has two red steps that I don't believe are this PR's:

  • unused-deps: runs cargo check --workspace --lib --bins with -W unused_crate_dependencies. This PR adds no dependencies and changes no lib or bin code: the new correction_workload module is cfg(any(test, feature = "bench")), the correction_v2 test module only moved out of line, and the rest is a bench and Python. The lib and bin targets the step compiles are identical to the merge base.
  • launchdarkly: exercises a service the diff doesn't touch and needs LAUNCHDARKLY_API_TOKEN; its sibling launchdarkly-flag-consistency, which needs the same token, passed on this build.

I can't read the Buildkite logs or re-run jobs from this session, so I can't name the flagged dependency or confirm the same steps are red on main. If someone can paste the unused-deps findings or retry the two jobs, I'll take it from there. The feature-benchmark job with the new MvSink scenarios is still running.


Generated by Claude Code

Override the feature-benchmark workflow so THIS runs with a flag on and
OTHER with it off on the same build, restricted to the MvSink family.
The parallel jobs split by parity between enable_compute_sync_mv_sink
and enable_compute_correction_v2. Revert before merging.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GAf7V5GxLQRwhN8siyQtHn
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GAf7V5GxLQRwhN8siyQtHn
Reverts the two TEMP commits that forced the feature-benchmark workflow
into an MvSink flag A/B for nightly #18259. The tree is back to the PR's
real content.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GAf7V5GxLQRwhN8siyQtHn
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-nightly PR CI control: also trigger Nightly

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants