You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
| [`tidb_cdc_write_source`](/system-variables.md#tidb_cdc_write_source-new-in-v650) | Newly added | When this variable is set to a value other than 0, data written in this session is considered to be written by TiCDC. This variable can only be modified by TiCDC. Do not manually modify this variable in any case. |
329
329
| [`tidb_enable_plan_replayer_capture`](/system-variables.md#tidb_enable_plan_replayer_capture) | Newly added | The feature controlled by this variable is not fully functional in TiDB v6.5.0. Do not change the default value. |
330
330
| [`tidb_index_merge_intersection_concurrency`](/system-variables.md#tidb_index_merge_intersection_concurrency-new-in-v650) | Newly added | Sets the maximum concurrency for the intersection operations that index merge performs. It is effective only when TiDB accesses partitioned tables in the dynamic pruning mode. |
331
-
| [`tidb_source_id`](/system-variables.md#tidb_source_id-new-in-v650) | Newly added | This variable is used to configure the different cluster IDs in a [bi-directional replication](/ticdc/ticdc-bidirectional-replication.md) cluster.|
331
+
| [`tidb_source_id`](/system-variables.md#tidb_source_id-new-in-v650) | Newly added | This variable is used to configure the different cluster IDs in a [bidirectional replication](/ticdc/ticdc-bidirectional-replication.md) cluster.|
332
332
| [`tidb_sysproc_scan_concurrency`](/system-variables.md#tidb_sysproc_scan_concurrency-new-in-v650) | Newly added | This variable is used to set the concurrency of scan operations performed when TiDB executes internal SQL statements (such as an automatic update of statistics). The default value is `1`. |
333
333
| [`tidb_ttl_delete_batch_size`](/system-variables.md#tidb_ttl_delete_batch_size-new-in-v650) | Newly added | This variable is used to set the maximum number of rows that can be deleted in a single `DELETE` transaction in a TTL job. |
334
334
| [`tidb_ttl_delete_rate_limit`](/system-variables.md#tidb_ttl_delete_rate_limit-new-in-v650) | Newly added | This variable is used to limit the maximum number of `DELETE` statements allowed per second in a single node in a TTL job. When this variable is set to `0`, no limit is applied. |
Copy file name to clipboardExpand all lines: system-variables.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3814,13 +3814,13 @@ For details, see [Identify Slow Queries](/identify-slow-queries.md).
3814
3814
3815
3815
<CustomContent platform="tidb">
3816
3816
3817
-
- This variable is used to configure the different cluster IDs in a [bi-directional replication](/ticdc/ticdc-bidirectional-replication.md) cluster.
3817
+
- This variable is used to configure the different cluster IDs in a [bidirectional replication](/ticdc/ticdc-bidirectional-replication.md) cluster.
3818
3818
3819
3819
</CustomContent>
3820
3820
3821
3821
<CustomContent platform="tidb-cloud">
3822
3822
3823
-
- This variable is used to configure the different cluster IDs in a [bi-directional replication](https://docs.pingcap.com/tidb/stable/ticdc-bidirectional-replication) cluster.
3823
+
- This variable is used to configure the different cluster IDs in a [bidirectional replication](https://docs.pingcap.com/tidb/stable/ticdc-bidirectional-replication) cluster.
@@ -5,13 +5,17 @@ summary: Learn how to use bidirectional replication of TiCDC.
5
5
6
6
# Bidirectional Replication
7
7
8
+
<<<<<<< HEAD
8
9
Starting from v6.5.0, TiCDC supports bi-directional replication among two TiDB clusters. Based on this feature, you can create a multi-active TiDB solution using TiCDC.
10
+
=======
11
+
TiCDC supports bidirectional replication (BDR) among two TiDB clusters. Based on this feature, you can create a multi-active TiDB solution using TiCDC.
12
+
>>>>>>> 1e9dac24d7 (ticdc: fix wrong names in bidirectional replication (#21928))
9
13
10
-
This section describes how to use bi-directional replication taking two TiDB clusters as an example.
14
+
This section describes how to use bidirectional replication taking two TiDB clusters as an example.
11
15
12
-
## Deploy bi-directional replication
16
+
## Deploy bidirectional replication
13
17
14
-
TiCDC only replicates incremental data changes that occur after a specified timestamp to the downstream cluster. Before starting the bi-directional replication, you need to take the following steps:
18
+
TiCDC only replicates incremental data changes that occur after a specified timestamp to the downstream cluster. Before starting the bidirectional replication, you need to take the following steps:
15
19
16
20
1. (Optional) According to your needs, import the data of the two TiDB clusters into each other using the data export tool [Dumpling](/dumpling-overview.md) and data import tool [TiDB Lightning](/tidb-lightning/tidb-lightning-overview.md).
17
21
@@ -21,25 +25,30 @@ TiCDC only replicates incremental data changes that occur after a specified time
21
25
22
26
3. Specify the starting time point of data replication for the upstream and downstream clusters.
23
27
24
-
1. Check the time point of the upstream and downstream clusters. In the case of two TiDB clusters, make sure that data in the two clusters are consistent at certain time points. For example, the data of TiDB A at `ts=1` and the data of TiDB B at `ts=2` are consistent.
28
+
1. Check the time point of the upstream and downstream clusters. In the case of two TiDB clusters, make sure that data in the two clusters are consistent at certain time points. For example, the data of TiDB 1 at `ts=1` and the data of TiDB 2 at `ts=2` are consistent.
25
29
26
-
2. When you create the changefeed, set the `--start-ts` of the changefeed for the upstream cluster to the corresponding `tso`. That is, if the upstream cluster is TiDB A, set `--start-ts=1`; if the upstream cluster is TiDB B, set `--start-ts=2`.
30
+
2. When you create the changefeed, set the `--start-ts` of the changefeed for the upstream cluster to the corresponding `tso`. That is, if the upstream cluster is TiDB 1, set `--start-ts=1`; if the downstream cluster is TiDB 2, set `--start-ts=2`.
27
31
28
32
4. In the configuration file specified by the `--config` parameter, add the following configuration:
29
33
30
34
```toml
31
-
# Whether to enable the bi-directional replication mode
35
+
# Whether to enable the bidirectional replication mode
32
36
bdr-mode = true
33
37
```
34
38
35
-
After the configuration takes effect, the clusters can perform bi-directional replication.
39
+
After the configuration takes effect, the clusters can perform bidirectional replication.
36
40
37
41
## Execute DDL
38
42
43
+
<<<<<<< HEAD
39
44
After the bidirectional replication is enabled, TiCDC does not replicate any DDL statements. You need to execute DDL statements in the upstream and downstream clusters respectively.
45
+
=======
46
+
Starting from v7.6.0, to support DDL replication as much as possible in bidirectional replication, TiDB divides the [DDLs that TiCDC originally supports](/ticdc/ticdc-ddl.md) into two types: replicable DDLs and non-replicable DDLs, according to the impact of DDLs on the business.
47
+
>>>>>>> 1e9dac24d7 (ticdc: fix wrong names in bidirectional replication (#21928))
40
48
41
49
Note that some DDL statements might cause table structure changes or data change time sequence problems, which might lead to data inconsistency after the replication. Therefore, after enabling bidirectional replication, only the DDL statements in the following table can be executed without stopping the write operations of the application.
42
50
51
+
<<<<<<< HEAD
43
52
| Event | Does it cause changefeed errors | Note |
44
53
|---|---|---|
45
54
| create database | Yes | After you manually execute the DDL statements in the upstream and downstream clusters, the errors can be automatically recovered. |
@@ -59,23 +68,149 @@ Note that some DDL statements might cause table structure changes or data change
59
68
| alter table remove ttl | No | |
60
69
| add **not unique** index | No | |
61
70
| drop **not unique** index | No | |
71
+
=======
72
+
Replicable DDLs are the DDLs that can be directly executed and replicated to other TiDB clusters in bidirectional replication.
73
+
>>>>>>> 1e9dac24d7 (ticdc: fix wrong names in bidirectional replication (#21928))
62
74
63
75
If you need to execute DDL statements that are not in the preceding table, take the following steps:
64
76
77
+
<<<<<<< HEAD
65
78
1. Pause the write operations in the tables that need to execute DDL in all clusters.
66
79
2. After the write operations of the corresponding tables in all clusters have been replicated to other clusters, manually execute all DDL statements in each TiDB cluster.
67
80
3. After the DDL statements are executed, resume the write operations.
81
+
=======
82
+
- [`ALTER TABLE ... ADD COLUMN`](/sql-statements/sql-statement-add-column.md): the column can be `null`, or has `not null` and `default value` at the same time
- [`ALTER TABLE ... DROP PRIMARY KEY`](/sql-statements/sql-statement-alter-table.md)
89
+
- [`ALTER TABLE ... MODIFY COLUMN`](/sql-statements/sql-statement-modify-column.md): you can only modify the `default value` and `comment` of the column
Non-replicable DDLs are the DDLs that have a great impact on the business, and might cause data inconsistency between clusters. Non-replicable DDLs cannot be directly replicated to other TiDB clusters in bidirectional replication through TiCDC. Non-replicable DDLs must be executed through specific operations.
105
+
106
+
Non-replicable DDLs include:
107
+
108
+
- [`ALTER DATABASE CHARACTER SET`](/sql-statements/sql-statement-alter-table.md)
109
+
- [`ALTER TABLE ... ADD COLUMN`](/sql-statements/sql-statement-alter-table.md): the column is `not null` and does not have a `default value`
- [`ALTER TABLE ... MODIFY COLUMN`](/sql-statements/sql-statement-modify-column.md): you can modify the attributes of the column except `default value` and `comment`
To solve the problem of replicable DDLs and non-replicable DDLs, TiDB introduces the following BDR roles:
130
+
131
+
- `PRIMARY`: You can execute replicable DDLs, but not non-replicable DDLs. Replicable DDLs executed in a PRIMARY cluster will be replicated to the downstream by TiCDC.
132
+
- `SECONDARY`: You cannot execute replicable DDLs or non-replicable DDLs. However, DDLs executed in a PRIMARY cluster can be replicated to a SECONDARY cluster by TiCDC.
133
+
134
+
When no BDR role is set, you can execute any DDL. However, the changefeed in BDR mode does not replicate any DDL on that cluster.
135
+
136
+
In short, in BDR mode, TiCDC only replicates replicable DDLs in the PRIMARY cluster to the downstream.
137
+
138
+
### Replication scenarios of replicable DDLs
139
+
140
+
1. Choose a TiDB cluster and execute `ADMIN SET BDR ROLE PRIMARY` to set it as the primary cluster.
141
+
142
+
```sql
143
+
ADMIN SET BDR ROLE PRIMARY;
144
+
```
145
+
146
+
```
147
+
Query OK, 0 rows affected
148
+
Time: 0.003s
149
+
150
+
ADMIN SHOW BDR ROLE;
151
+
+----------+
152
+
| BDR_ROLE |
153
+
+----------+
154
+
| primary |
155
+
+----------+
156
+
```
157
+
158
+
2. On other TiDB clusters, execute `ADMIN SET BDR ROLE SECONDARY` to set them as the secondary clusters.
159
+
3. Execute **replicable DDLs** on the primary cluster. The successfully executed DDLs will be replicated to the secondary clusters by TiCDC.
160
+
161
+
> **Note:**
162
+
>
163
+
> To prevent misuse:
164
+
>
165
+
> - If you try to execute **non-replicable DDLs** on the primary cluster, you will get the [Error 8263](/error-codes.md).
166
+
> - If you try to execute **replicable DDLs** or **non-replicable DDLs** on the secondary clusters, you will get the [Error 8263](/error-codes.md).
167
+
168
+
### Replication scenarios of non-replicable DDLs
169
+
170
+
1. Execute `ADMIN UNSET BDR ROLE` on all TiDB clusters to unset the BDR role.
171
+
2. Stop writing data to the tables that need to execute DDLs in all clusters.
172
+
3. Wait until all writes to the corresponding tables in all clusters are replicated to other clusters, and then manually execute all DDLs on each TiDB cluster.
173
+
4. Wait until the DDLs are completed, and then resume writing data.
174
+
5. Follow the steps in [Replication scenarios of replicable DDLs](#replication-scenarios-of-replicable-ddls) to switch back to the replication scenario of replicable DDLs.
175
+
176
+
> **Warning:**
177
+
>
178
+
> After you execute `ADMIN UNSET BDR ROLE` on all TiDB clusters, none of the DDLs are replicated by TiCDC. You need to manually execute the DDLs on each cluster separately.
179
+
>>>>>>> 1e9dac24d7 (ticdc: fix wrong names in bidirectional replication (#21928))
68
180
69
-
## Stop bi-directional replication
181
+
## Stop bidirectional replication
70
182
71
183
After the application has stopped writing data, you can insert a special record into each cluster. By checking the two special records, you can make sure that data in two clusters are consistent.
72
184
185
+
<<<<<<< HEAD
73
186
After the check is completed, you can stop the changefeed to stop bi-directional replication.
74
187
75
188
## Limitations
76
189
77
190
- For the limitations of DDL, see [Execute DDL](#execute-ddl).
191
+
=======
192
+
After the check is completed, you can stop the changefeed to stop bidirectional replication, and execute `ADMIN UNSET BDR ROLE` on all TiDB clusters.
193
+
194
+
## Limitations
195
+
196
+
- Use BDR role only in the following scenarios:
197
+
198
+
- 1 `PRIMARY` cluster and n `SECONDARY` clusters (replication scenarios of replicable DDLs)
199
+
- n clusters that have no BDR roles (replication scenarios in which you can manually execute non-replicable DDLs on each cluster)
200
+
201
+
> **Note:**
202
+
>
203
+
> Do not set the BDR role in other scenarios, for example, setting `PRIMARY`, `SECONDARY`, and no BDR roles at the same time. If you set the BDR role incorrectly, TiDB cannot guarantee data correctness and consistency during data replication.
204
+
205
+
- Usually do not use [`AUTO_INCREMENT`](/auto-increment.md) or [`AUTO_RANDOM`](/auto-random.md) to avoid data conflicts in the replicated tables. If you need to use `AUTO_INCREMENT` or `AUTO_RANDOM`, you can set different `auto_increment_increment` and `auto_increment_offset` for different clusters to ensure that different clusters can be assigned different primary keys. For example, if there are three TiDB clusters (A, B, and C) in bidirectional replication, you can set them as follows:
206
+
207
+
- In Cluster A, set `auto_increment_increment=3` and `auto_increment_offset=2000`
208
+
- In Cluster B, set `auto_increment_increment=3` and `auto_increment_offset=2001`
209
+
- In Cluster C, set `auto_increment_increment=3` and `auto_increment_offset=2002`
210
+
211
+
This way, A, B, and C will not conflict with each other in the implicitly assigned `AUTO_INCREMENT` ID and `AUTO_RANDOM` ID. If you need to add a cluster in BDR mode, you need to temporarily stop writing data of the related application, set the appropriate values for `auto_increment_increment` and `auto_increment_offset` on all clusters, and then resume writing data of the related application.
212
+
>>>>>>> 1e9dac24d7 (ticdc: fix wrong names in bidirectional replication (#21928))
78
213
79
-
- Bi-directional replication clusters cannot detect write conflicts, which might cause undefined behaviors. Therefore, you must ensure that there are no write conflicts from the application side.
214
+
- Bidirectional replication clusters cannot detect write conflicts, which might cause undefined behaviors. Therefore, you must ensure that there are no write conflicts from the application side.
80
215
81
-
- Bi-directional replication supports more than two clusters, but does not support multiple clusters in cascading mode, that is, a cyclic replication like TiDB A -> TiDB B -> TiDB C -> TiDB A. In such a topology, if one cluster fails, the whole data replication will be affected. Therefore, to enable bi-directional replication among multiple clusters, you need to connect each cluster with every other clusters, for example, `TiDB A <-> TiDB B`, `TiDB B <-> TiDB C`, `TiDB C <-> TiDB A`.
216
+
- Bidirectional replication supports more than two clusters, but does not support multiple clusters in cascading mode, that is, a cyclic replication like TiDB A -> TiDB B -> TiDB C -> TiDB A. In such a topology, if one cluster fails, the whole data replication will be affected. Therefore, to enable bidirectional replication among multiple clusters, you need to connect each cluster with every other clusters, for example, `TiDB A <-> TiDB B`, `TiDB B <-> TiDB C`, `TiDB C <-> TiDB A`.
0 commit comments