sql: accept WITH (WAIT ...) on ALTER CLUSTER unconditionally

[aljoscha]

Motivation

WITH (WAIT ...) on ALTER CLUSTER is gated behind
enable_zero_downtime_cluster_reconfiguration, default off. Stacked on #38103,
every graceful reconfiguration already runs through one durable controller path,
so the flag only controls whether users can express a deadline and a timeout
action for it.

Description

Removes the feature flag and its planner gate, so WITH (WAIT ...) is accepted
on every deployment. The two rejections that share that code path stay: a WAIT
without a replica-shape change, and a WAIT on an unmanaged cluster.

The mixed-version default is bounded at v26.41 so upgrade scenarios running
against an older binary still set the flag, and the private-preview badges come
off the ALTER CLUSTER reference page.

enable_cluster_schedule_refresh, the near-identical sibling gate, is
deliberately untouched. It gates a separate SQL surface under its own rollout.

Verification

Existing graceful-reconfiguration coverage in testdrive, sqllogictest, and
platform checks now exercises the surface without setting a flag.

User-visible behavior

Graceful cluster reconfiguration with WITH (WAIT UNTIL READY ...) and
WITH (WAIT FOR ...) becomes generally available, no longer private preview and
no longer requiring a feature flag.

[ggevay]

Fine to remove the "private preview" status from my side! cc @maheshwarip

[def-]

QA LLM Review

1. MEDIUM -- Mixed-version pin for enable_zero_downtime_cluster_reconfiguration is bounded two minor versions too low

misc/python/materialize/mzcompose/__init__.py:175

The planner gate is removed in v26.41.0-dev (the version at this head), but the pin only applies to version < v26.39.0-dev. v26.39 and v26.40 binaries still carry the gate with default: false, so upgrade scenarios that run a WITH (WAIT ...) phase against them now get Enable zero-downtime reconfiguration for alter cluster is not available instead of the coverage this PR says it relies on.

Details

The pin has to cover every binary that still gates the surface, i.e. everything below the version where the gate disappears. That is exactly the convention the sibling block ten lines above uses: enable_cluster_controller was removed at v26.38.0-dev and is pinned for version < v26.38.0-dev.

Concretely, with MzVersion.parse_cargo() == v26.41.0-dev.0, get_published_minor_mz_versions(..., limit=4, include_release_candidates=True) yields [v26.40.0-rc.2, v26.39.0, v26.38.2, v26.37.0], and v26.39.0 < v26.39.0-dev is False. AlterClusterGracefulReconfiguration (misc/python/materialize/checks/all_checks/cluster.py:137) only skips below v26.35.0-dev, and this PR deleted the ALTER SYSTEM SET it used to do in initialize(), so:

• UpgradeEntireMz — base v26.40.0-rc.2, manipulate phase 1 runs ALTER CLUSTER ... WITH (WAIT UNTIL READY ...) on that binary, unpinned.
• UpgradeEntireMzTwoVersions — phase 1 on v26.40.0-rc.2, unpinned.
• UpgradeEntireMzFourVersions — phase 2 on v26.39.0, unpinned.

StartMz passes system_parameter_version = tag (misc/python/materialize/checks/mzcompose_actions.py:65), so get_minimal_system_parameters is evaluated per old image and the pin is genuinely absent for those tags. Sending the pin to a binary that no longer knows the name is harmless: catalog/open.rs:258 downgrades an unknown --system-parameter-default to a warn!.

-    # The `WITH (WAIT ...)` graceful-reconfiguration surface. Always accepted
-    # from v26.39 on. Older binaries still gate it behind this feature flag, so
+    # The `WITH (WAIT ...)` graceful-reconfiguration surface. Always accepted
+    # from v26.41 on. Older binaries still gate it behind this feature flag, so
     # pin it on for them: the tests that use the surface no longer enable it
     # themselves, and in a mixed-version run some of their phases execute
     # against the old binary.
-    if version < MzVersion.parse_mz("v26.39.0-dev"):
+    if version < MzVersion.parse_mz("v26.41.0-dev"):
         config["enable_zero_downtime_cluster_reconfiguration"] = "true"

Worth re-checking the bound at merge time: if the stack sits until the next version bump, it needs to move again.

[aljoscha]

Fixed. Current upstream main is v26.41.0-dev.0, so mixed-version defaults now pin enable_zero_downtime_cluster_reconfiguration=true for versions below v26.41.0-dev. The PR description and commit message use the same bound.