sql: accept WITH (WAIT ...) on ALTER CLUSTER unconditionally

Graceful cluster reconfiguration has been behind the
`enable_zero_downtime_cluster_reconfiguration` feature flag, default off, so
the `WITH (WAIT ...)` surface is rejected at plan time unless a deployment
turns it on. The controller record is now the only managed-cluster reshape
path, which leaves the flag controlling only whether users can express its
deadline and timeout behavior.

Remove the flag and planner gate. The two rejections that share the code path
stay: a `WAIT` without a replica-shape change, and a `WAIT` on an unmanaged
cluster.

Every test that used the surface enabled the flag itself, so those statements
go. In a mixed-version run some phases execute against a released binary that
still enforces the gate, so `get_minimal_system_parameters` pins the flag on
below v26.39 instead.

The docs drop the private-preview badges because the surface is generally
available once nothing gates it.

Author: aljoscha

doc/user/content/sql/alter-cluster.md

@@ -177,10 +177,6 @@ by default), Materialize rolls back the resize and the cluster keeps its current
 size. To customize the timeout behavior, use the `WAIT UNTIL READY` or `WAIT FOR` options.
 The resize still proceeds in the background.
 
-{{< private-preview >}}
-Customizing the resize timeout with `WAIT UNTIL READY` or `WAIT FOR`
-{{< /private-preview >}}
-
 - `WAIT UNTIL READY (TIMEOUT = ..., ON TIMEOUT = ...)` sets the timeout for the
   resize. On timeout, `ON TIMEOUT` selects whether to `COMMIT` (retire the old
   replicas and proceed with the not-yet-hydrated new ones, which can cause
@@ -225,7 +221,6 @@ current size. Materialize drops the target replicas and keeps the current
 configuration.
 
 #### Downtime considerations for v26.34 or before
-{{< private-preview />}}
 
 You can use the `WAIT UNTIL READY` option to perform a zero-downtime resizing,
 which incurs **no downtime**. Instead of restarting the cluster, this approach

doc/user/data/examples/alter_cluster.yml

@@ -96,8 +96,8 @@
         The following `<with_option>`s are supported:
         | Option  | Description |
         |--------|-------------|
-        | `WAIT UNTIL READY(...)`    | ***Private preview.** This option has known performance or stability issues and is under active development.* {{< include-from-yaml data="examples/alter_cluster" name="wait-until-ready-cmd-option" >}} |
-        | `WAIT FOR` |  ***Private preview.** This option has known performance or stability issues and is under active development.* A fixed duration to wait for the new replicas to be ready. This option can lead to downtime. As such, we recommend using the `WAIT UNTIL READY` option instead.|
+        | `WAIT UNTIL READY(...)`    | {{< include-from-yaml data="examples/alter_cluster" name="wait-until-ready-cmd-option" >}} |
+        | `WAIT FOR` | A fixed duration to wait for the new replicas to be ready. This option can lead to downtime. As such, we recommend using the `WAIT UNTIL READY` option instead.|
 
 - name: "syntax-reset-to-default"
   code: |

misc/python/materialize/checks/all_checks/cluster.py

@@ -152,9 +152,6 @@ def _can_run(self, e: Executor) -> bool:
 
     def initialize(self) -> Testdrive:
         return Testdrive(dedent("""
-            $ postgres-execute connection=postgres://mz_system@${testdrive.materialize-internal-sql-addr}
-            ALTER SYSTEM SET enable_zero_downtime_cluster_reconfiguration = true
-
             $ postgres-execute connection=postgres://postgres:postgres@postgres
             CREATE USER graceful_reconfig WITH SUPERUSER PASSWORD 'postgres';
             ALTER USER graceful_reconfig WITH replication;

misc/python/materialize/mzcompose/__init__.py

@@ -167,6 +167,14 @@ def get_minimal_system_parameters(
             "true" if version >= MzVersion.parse_mz("v26.29.0-dev") else "false"
         )
 
+    # The `WITH (WAIT ...)` graceful-reconfiguration surface. Always accepted
+    # from v26.39 on. Older binaries still gate it behind this feature flag, so
+    # pin it on for them: the tests that use the surface no longer enable it
+    # themselves, and in a mixed-version run some of their phases execute
+    # against the old binary.
+    if version < MzVersion.parse_mz("v26.39.0-dev"):
+        config["enable_zero_downtime_cluster_reconfiguration"] = "true"
+
     return config
 
 

misc/python/materialize/parallel_workload/parallel_workload.py

@@ -168,10 +168,6 @@ def run(
         system_exe.execute("ALTER SYSTEM SET max_sql_server_connections = 1000000")
         system_exe.execute("ALTER SYSTEM SET max_kafka_connections = 1000000")
         system_exe.execute("ALTER SYSTEM SET idle_in_transaction_session_timeout = 0")
-        # Gates the WITH (WAIT ...) clause used by ReconfigureClusterAction.
-        system_exe.execute(
-            "ALTER SYSTEM SET enable_zero_downtime_cluster_reconfiguration = true"
-        )
         # Most queries should not fail because of privileges
         for object_type in [
             "TABLES",

src/sql/src/plan/statement/ddl.rs

@@ -6727,15 +6727,6 @@ pub fn plan_alter_cluster(
                         );
                     }
 
-                    match alter_strategy {
-                        AlterClusterPlanStrategy::None => {}
-                        _ => {
-                            scx.require_feature_flag(
-                                &crate::session::vars::ENABLE_ZERO_DOWNTIME_CLUSTER_RECONFIGURATION,
-                            )?;
-                        }
-                    }
-
                     if replica_defs.is_some() {
                         sql_bail!("REPLICAS not supported for managed clusters");
                     }

src/sql/src/session/vars/definitions.rs

@@ -2253,12 +2253,6 @@ feature_flags!(
         default: false,
         enable_for_item_parsing: false,
     },
-    {
-        name: enable_zero_downtime_cluster_reconfiguration,
-        desc: "Enable zero-downtime reconfiguration for alter cluster",
-        default: false,
-        enable_for_item_parsing: false,
-    },
     {
         name: enable_network_policies,
         desc: "ENABLE NETWORK POLICIES",

test/cloudtest/test_managed_cluster.py

@@ -146,7 +146,6 @@ def test_zero_downtime_reconfiguration(mz: MaterializeApplication) -> None:
     # within the short poll loops below.
     mz.environmentd.sql(
         """
-        ALTER SYSTEM SET enable_zero_downtime_cluster_reconfiguration = true;
         ALTER SYSTEM SET cluster_controller_tick_interval = '5ms';
         """,
         port="internal",

test/cluster/mzcompose.py

@@ -6318,7 +6318,6 @@ def workflow_test_zero_downtime_reconfigure(
             key${kafka-ingest.iteration}:value${kafka-ingest.iteration}
 
             $ postgres-execute connection=postgres://mz_system:materialize@${testdrive.materialize-internal-sql-addr}
-            ALTER SYSTEM SET enable_zero_downtime_cluster_reconfiguration = true;
             CREATE CLUSTER cluster1 ( SIZE = 'scale=1,workers=1');
             GRANT ALL ON CLUSTER cluster1 TO materialize;
 
@@ -6435,13 +6434,6 @@ def workflow_test_zero_downtime_reconfigure(
             > SELECT count(*) FROM kafka_tbl
             1000
             """))
-        c.sql(
-            """
-            ALTER SYSTEM RESET enable_zero_downtime_cluster_reconfiguration;
-            """,
-            port=6877,
-            user="mz_system",
-        )
 
 
 def workflow_test_pending_replica_audit_events(
@@ -6457,11 +6449,10 @@ def workflow_test_pending_replica_audit_events(
     """
     c.up("materialized")
 
-    # Enable the WAIT surface and drive the controller tick down so the (empty)
-    # cluster's reconfiguration converges quickly.
+    # Drive the controller tick down so the (empty) cluster's reconfiguration
+    # converges quickly.
     c.sql(
         """
-        ALTER SYSTEM SET enable_zero_downtime_cluster_reconfiguration = true;
         ALTER SYSTEM SET cluster_controller_tick_interval = '5ms';
         CREATE CLUSTER test_audit (SIZE = 'scale=1,workers=1');
         GRANT ALL ON CLUSTER test_audit TO materialize;
@@ -6556,7 +6547,6 @@ def workflow_test_pending_replica_audit_events(
     c.sql(
         """
         DROP CLUSTER test_audit CASCADE;
-        ALTER SYSTEM RESET enable_zero_downtime_cluster_reconfiguration;
         """,
         port=6877,
         user="mz_system",

test/launchdarkly-flag-consistency/mzcompose.py

@@ -480,6 +480,7 @@
     enable_repr_typecheck
     enable_unified_cluster_arrangment
     enable_yugabyte_connection
+    enable_zero_downtime_cluster_reconfiguration
     kafka_default_metadata_fetch_interval
     mysql_offset_known_interval
     persist_enable_arrow_lgalloc_noncc_sizes
@@ -516,7 +517,6 @@
     "enable_lgalloc",
     "enable_timely_zero_copy_lgalloc",
     "enable_upsert_paged_spill",
-    "enable_zero_downtime_cluster_reconfiguration",
     "kafka_client_id_enrichment_rules",
     "kafka_progress_record_fetch_timeout",
     "kafka_socket_timeout",