@@ -20,25 +20,26 @@ new instance running the new version, and the replication slot does **not**
2020carry over automatically. How you handle the slot determines whether reads stay
2121fresh through the upgrade.
2222
23- This guide covers two approaches:
24-
25- | Approach | Reads stay fresh? | Who moves the data? | Use when |
26- | ----------| -------------------| ---------------------| ----------|
27- | [ Parallel source (self-managed)] ( #approach-a-parallel-source-self-managed ) | ** Yes, continuously** | You (native logical replication) | Zero read downtime is a hard requirement. |
28- | [ Amazon RDS Blue/Green] ( #approach-b-amazon-rds-bluegreen-managed ) | No — bounded freshness gap | AWS | A brief staleness window is acceptable and you want AWS to manage the cutover. |
23+ The approach in this guide keeps Materialize continuously fresh. You keep the
24+ existing source running against the old primary the entire time, build the
25+ new-version primary in parallel, and hydrate a second source against it before
26+ cutting consumers over.
2927
3028{{< note >}}
31- Both approaches involve a brief ** write** freeze on the application at the
32- moment of cutover, which is inherent to any major-version upgrade. The
33- difference is whether Materialize keeps serving fresh ** reads** throughout.
29+ This procedure involves a brief ** write** freeze on the application at the
30+ moment of cutover, which is inherent to any major-version upgrade. Materialize
31+ keeps serving fresh ** reads** throughout.
3432{{< /note >}}
3533
36- ## Approach A: Parallel source (self-managed)
34+ {{< warning >}}
35+ Managed upgrade services that swap the primary out from under Materialize — such
36+ as [ Amazon RDS Blue/Green
37+ deployments] ( https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/blue-green-deployments.html )
38+ — cannot be used while a Materialize source is attached. See [ Why managed
39+ Blue/Green deployments don't work] ( #why-managed-bluegreen-deployments-dont-work ) .
40+ {{< /warning >}}
3741
38- This is the only approach that keeps Materialize continuously fresh. You keep
39- the existing source running against the old primary the entire time, build the
40- new-version primary in parallel, and hydrate a second source against it before
41- cutting consumers over.
42+ ## Upgrade with a parallel source
4243
4344``` text
4445app writes ──▶ PG (old major) ──native logical replication──▶ PG (new major)
@@ -145,110 +146,86 @@ Once the new source has caught up, perform a coordinated cutover:
1451461. **Drain both sources** in Materialize so they reflect the frozen state, then
146147 verify the fingerprint matches across the old primary, new primary, existing
147148 source, and new source.
148- 1. **Repoint consumers** from the existing source' s objects to the ` upgrade`
149- schema' s objects.
150- 1. **Resume application writes**, now pointed at the new primary.
151-
152- Throughout the cutover, reads against Materialize stay live and fresh — the
153- existing source serves until the flip, and the new source is already caught up
154- at the flip.
149+ 1. **Swap the schemas.** Rather than repointing each consumer, swap the
150+ production schema with the `upgrade` schema. The swap is atomic, so
151+ consumers keep referencing the same schema-qualified names and move together
152+ at a single instant:
155153
156- ### 5. Decommission
154+ ```mzsql
155+ ALTER SCHEMA public SWAP WITH upgrade;
156+ ```
157157
158- Drop the subscription on the new primary, drop the old source in Materialize,
159- and decommission the old primary.
158+ To roll back, run the same statement again.
160159
161- ## Approach B: Amazon RDS Blue/Green (managed)
160+ 1. **Resume application writes**, now pointed at the new primary.
162161
163- [Amazon RDS Blue/Green deployments](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/blue-green-deployments.html)
164- let AWS build the upgraded "green" instance, keep it in sync, and swap
165- endpoints at switchover. This is operationally simpler, but it **cannot keep
166- Materialize continuously fresh** because of the constraint below.
162+ Throughout the cutover, reads against Materialize stay live and fresh — the
163+ existing source serves until the swap, and the new source is already caught up
164+ at the swap.
167165
168166{{< warning >}}
169- **A Blue/Green deployment cannot be created while a Materialize source is
170- attached to the blue instance.** Materialize' s logical replication slot counts
171- as * external replication* , and RDS refuses to create the deployment
172- (` CREATING_READ_REPLICA_OF_SOURCE` fails with * " external replication on the blue
173- primary instance" * ). You must [drop the
174- source](/ sql/ drop- source/ ) first, which starts an unavoidable ** freshness
175- gap** . No RDS flag bypasses this.
167+ If you have [sinks](/sql/create-sink/), create them in a **dedicated schema and
168+ cluster** that is excluded from the swap. Sinks must not be recreated as part of
169+ a blue/green cutover; instead, cut them over to the new definition of their
170+ upstream dependencies after the swap. This mirrors the guidance in [blue/green
171+ deployments with dbt](/manage/dbt/blue-green-deployments/).
176172{{< /warning >}}
177173
178- # ## 1. Detach Materialize and create the deployment
179-
180- 1 . ` DROP SOURCE` in Materialize to release the replication slot. Confirm
181- ` pg_replication_slots` no longer lists Materialize' s slot. **The freshness
182- gap starts here.**
183- 1. Create the Blue/Green deployment, targeting the new major version with a
184- parameter group that has `rds.logical_replication = 1`. Wait for the green
185- instance to become **Available** (a major-version upgrade can take tens of
186- minutes; it scales with database size).
187-
188- The publication and `REPLICA IDENTITY` settings are copied from blue into green.
189-
190- ### 2. Attach Materialize to green
191-
192- The green instance is a logical replica that is itself a primary
193- (`pg_is_in_recovery()` returns `false`), so it can host Materialize' s slot.
194- Create a source against ** green' s temporary endpoint** and let it snapshot and
195- catch up:
196-
197- ```mzsql
198- CREATE SOURCE my_source
199- FROM POSTGRES CONNECTION pg_green (PUBLICATION ' mz_source' )
200- FOR ALL TABLES;
201- ```
202-
203- **The freshness gap ends when this source has caught up.** As in Approach A,
204- snapshot time scales with data volume.
205-
206- ### 3. Switch over and repoint the source
207-
208- Trigger the Blue/Green switchover. AWS promotes green **in place** — it takes
209- over the production endpoint, while the old blue is demoted and renamed. The
210- switchover completes in tens of seconds, during which the application has a
211- brief write pause.
212-
213- Because green is promoted in place (same instance, same LSN timeline), its
214- replication slot is preserved. Repoint the source' s connection at the
215- production endpoint so Materialize resumes from the ** same slot with no
216- re- snapshot** :
217-
218- ` ` ` mzsql
219- ALTER CONNECTION pg_green SET (HOST = 'PRODUCTION_ENDPOINT') WITH (VALIDATE = true);
220- ` ` `
221-
222- After the switchover, green' s temporary endpoint is removed; a source restart
223- would otherwise fail to resolve it, so run the `ALTER CONNECTION` promptly.
224-
225174{{< note >}}
226- This seamless resume works **only** because a Blue/Green switchover promotes
227- green in place, preserving the slot and LSN timeline. Repointing a source at an
228- unrelated instance (one that doesn' t carry the same slot) forces a fresh
229- snapshot instead.
175+ Any active [`SUBSCRIBE`](/sql/subscribe/) commands attached to the swapped
176+ cluster(s) will break at the swap. On retry, the client automatically connects
177+ to the newly deployed cluster.
230178{{< /note >}}
231179
232- # ## 4. Clean up
180+ ### 5. Decommission
181+
182+ Drop the subscription on the new primary, drop the old source in Materialize,
183+ and decommission the old primary.
233184
234- Delete the Blue/ Green deployment and the demoted old instance.
185+ ## Why managed Blue/Green deployments don' t work
186+
187+ [Amazon RDS Blue/ Green deployments](https:// docs .aws .amazon .com / AmazonRDS/ latest/ UserGuide/ blue- green- deployments .html )
188+ let AWS build the upgraded " green" instance, keep it in sync, and swap endpoints
189+ at switchover. This is an appealing shortcut, but it is ** not compatible with an
190+ attached Materialize source** .
191+
192+ Creating the deployment fails at ` CREATING_READ_REPLICA_OF_SOURCE` with * " external
193+ replication on the blue primary instance" * . One of the documented prerequisites
194+ is that the DB instance is not the source or target of external replication, and
195+ Materialize' s logical replication slot is external replication. No RDS setting
196+ bypasses this.
197+
198+ Releasing the slot — by [dropping the source](/sql/drop-source/) — does let the
199+ deployment be created, but that is not a viable production step:
200+
201+ - `DROP SOURCE` defaults to `RESTRICT` and fails when the source has dependents.
202+ Forcing it with `CASCADE` drops the entire downstream dataflow — tables, views,
203+ materialized views, indexes, and sinks — which means rebuilding and rehydrating
204+ your environment, not a brief pause.
205+ - Even after the switchover, Materialize' s original slot on blue is unusable, so
206+ the new source must snapshot from scratch against green.
207+
208+ The result is a staleness window that begins when you detach from blue and lasts
209+ through provisioning and a full re- snapshot. Use the [parallel
210+ source](# upgrade-with-a-parallel-source) procedure above instead: it keeps the
211+ existing source serving fresh for the entire upgrade, and the only coordinated
212+ pause is the write freeze at cutover.
235213
236214# # Considerations
237215
238- - ** DDL is not replicated.** For Approach A, pre - create the target schema on the
239- new instance before subscribing.
216+ - ** DDL is not replicated.** Pre - create the target schema on the new instance
217+ before subscribing.
240218- ** Sequences are not advanced on the target** by native logical replication.
241- Synchronize them at cutover (Approach A) or RDS handles it at switchover
242- (Approach B).
219+ Synchronize them at cutover.
243220- ** ` REPLICA IDENTITY FULL` ** must be set on replicated tables so Materialize
244221 captures all column values on updates and deletes.
245- - ** ` ALTER CONNECTION ... SET (HOST = ...)` ** resumes from the existing slot
246- only when the target instance preserves that slot (an in - place promotion).
247- Otherwise Materialize re- snapshots.
222+ - ** A publication can feed multiple subscribers** , so Materialize' s slot and the
223+ upgrade subscription' s slot coexist on the old primary without conflict.
248224
249225# # Related pages
250226
251227- [Ingest data from Amazon RDS](/ ingest- data/ postgres/ amazon- rds/ )
252228- [Guide: Ingest from a dedicated PostgreSQL replica](/ ingest- data/ postgres/ logical- replica/ )
253229- [Guide: Handle upstream schema changes with zero downtime](/ ingest- data/ postgres/ source- versioning/ )
254- - [` ALTER CONNECTION` ](/ sql/ alter- connection/ )
230+ - [Blue/ green deployments with dbt](/ manage/ dbt/ blue- green- deployments/ )
231+ - [` ALTER SCHEMA` ](/ sql/ alter- schema/ )
0 commit comments