|
| 1 | +--- |
| 2 | +title: Use Dumpling and TiDB Lightning for Data Backup and Restoration |
| 3 | +summary: Introduce how to use Dumpling and TiDB Lightning to backup and restore full data of TiDB. |
| 4 | +aliases: ['/docs-cn/dev/export-or-backup-using-dumpling/','/zh/tidb/dev/export-or-backup-using-dumpling'] |
| 5 | +--- |
| 6 | + |
| 7 | +# Use Dumpling and TiDB Lightning for Data Backup and Restoration |
| 8 | + |
| 9 | +This document introduces in detail how to use Dumpling and TiDB Lightning to backup and restore full data of TiDB. For incremental backup and replication to downstream, refer to [TiDB Binlog](/tidb-binlog/tidb-binlog-overview.md). |
| 10 | + |
| 11 | +Suppose that the TiDB server information is as follows: |
| 12 | + |
| 13 | +|Server Name|Server Address|Port|User|Password| |
| 14 | +|----|-------|----|----|--------| |
| 15 | +|TiDB|127.0.0.1|4000|root|*| |
| 16 | + |
| 17 | +Use the following tools for data backup and restoration: |
| 18 | + |
| 19 | +- [Dumpling](/dumpling-overview.md): to export data from TiDB |
| 20 | +- [TiDB Lightning](/tidb-lightning/tidb-lightning-overview.md): to import data into TiDB |
| 21 | + |
| 22 | +## Best practices for full backup and restoration using Dumpling/TiDB Lightning |
| 23 | + |
| 24 | +To quickly backup and restore data (especially large amounts of data), refer to the following recommendations: |
| 25 | + |
| 26 | +* Keep the exported data file as small as possible. It is recommended to use the `-F` option of Dumpling to set the file size. If you use TiDB Lightning to restore data, it is recommended that you set the value of `-F` to `256m`. |
| 27 | +* If some of the exported tables have many rows, you can enable concurrency in the table by setting the `-r` option. |
| 28 | + |
| 29 | +## Backup data from TiDB |
| 30 | + |
| 31 | +Use the following `dumpling` command to backup data from TiDB. |
| 32 | + |
| 33 | +{{< copyable "shell-regular" >}} |
| 34 | + |
| 35 | +```bash |
| 36 | +./bin/dumpling -h 127.0.0.1 -P 4000 -u root -t 32 -F 256m -T test.t1 -T test.t2 -o ./var/test |
| 37 | +``` |
| 38 | + |
| 39 | +In this command: |
| 40 | + |
| 41 | +- `-T test.t1 -T test.t2` means that only the two tables `test`.`t1` and `test`.`t2` are exported. For more methods to filter exported data, refer to [Filter exported data](/dumpling-overview.md#filter-the-exported-data). |
| 42 | +- `-t 32` means that 32 threads are used to export the data. |
| 43 | +- `-F 256m` means that a table is partitioned into chunks, and one chunk is 256MB. |
| 44 | + |
| 45 | +Starting from v4.0.0, Dumpling can automatically extends the GC time if it can access the PD address of the TiDB cluster. But for TiDB earlier than v4.0.0, you need to manually modify the GC time. Otherwise, you might bump into the following error: |
| 46 | + |
| 47 | +```log |
| 48 | +Could not read data from testSchema.testTable: GC life time is shorter than transaction duration, transaction starts at 2019-08-05 21:10:01.451 +0800 CST, GC safe point is 2019-08-05 21:14:53.801 +0800 CST |
| 49 | +``` |
| 50 | + |
| 51 | +The steps to manually modify the GC time are as follows: |
| 52 | + |
| 53 | +1. Before executing the `dumpling` command, query the [GC](/garbage-collection-overview.md) value of the TiDB cluster and execute the following statement in the MySQL client to adjust it to a suitable value: |
| 54 | + |
| 55 | + {{< copyable "sql" >}} |
| 56 | + |
| 57 | + ```sql |
| 58 | + SELECT * FROM mysql.tidb WHERE VARIABLE_NAME = 'tikv_gc_life_time'; |
| 59 | + ``` |
| 60 | + |
| 61 | + ```sql |
| 62 | + +-----------------------+------------------------------------------------------------------------------------------------+ |
| 63 | + | VARIABLE_NAME | VARIABLE_VALUE | |
| 64 | + +-----------------------+------------------------------------------------------------------------------------------------+ |
| 65 | + | tikv_gc_life_time | 10m0s | |
| 66 | + +-----------------------+------------------------------------------------------------------------------------------------+ |
| 67 | + 1 rows in set (0.02 sec) |
| 68 | + ``` |
| 69 | + |
| 70 | + {{< copyable "sql" >}} |
| 71 | + |
| 72 | + ```sql |
| 73 | + update mysql.tidb set VARIABLE_VALUE = '720h' where VARIABLE_NAME = 'tikv_gc_life_time'; |
| 74 | + ``` |
| 75 | + |
| 76 | +2. After executing the `dumpling` command, restore the GC value of the TiDB cluster to the initial value in step 1: |
| 77 | + |
| 78 | + {{< copyable "sql" >}} |
| 79 | + |
| 80 | + ```sql |
| 81 | + update mysql.tidb set VARIABLE_VALUE = '10m' where VARIABLE_NAME = 'tikv_gc_life_time'; |
| 82 | + ``` |
| 83 | + |
| 84 | +## Restore data into TiDB |
| 85 | + |
| 86 | +To restore data into TiDB, use TiDB Lightning to import the exported data. See [TiDB Lightning Tutorial](/tidb-lightning/tidb-lightning-tidb-backend.md). |
0 commit comments