Repository navigation
Piping
zfs-autobackup sends the data through a pipe from the source to the target. You can add operations to this pipe.
zfs send changes between no data and large bursts of data. --buffer adds an IO buffer: when zfs send has no data, the buffer continues to send data.
This needs mbuffer on the source and on the target.
For a slow link, use one of these options:
| Option | Compression | CPU use | Notes |
|---|---|---|---|
--compress |
Usually much better | High | Compresses the data before the send. |
--zfs-compressed |
Only blocks that ZFS already compressed | Lowest | Not good if you want a different ZFS compression on the target. |
Do not use the two options together. That usually does not help.
Without a type, --compress uses zstd-fast. Use --compress=TYPE to select a different compressor:
| Type | Command |
|---|---|
pigz-fast |
pigz -3 |
pigz-slow |
pigz -9 |
gzip |
gzip -3 and zcat
|
zstd-fast |
zstdmt -3 |
zstd-slow |
zstdmt -19 |
zstd-adapt |
zstdmt --adapt |
xz |
xz |
lzo |
lzop |
lz4 |
lz4 |
The compressor must be installed on the source and on the target.
Use --rate to limit the data rate. This prevents a full uplink, or decreases the IO load.
This needs mbuffer on the source.
Use --send-pipe and --recv-pipe to send the data through your own commands, for example a different compressor.
When you use all options, the pipes are:
On the source:
zfs send | buffer | custom send pipes | compression | transfer rate limiter | ssh
On the target:
decompression | custom recv pipes | buffer | zfs recv
zfs-autobackup only adds the buffer on the target if the target is a different host.
zfs-autobackup gives a warning for options that do not help. For example: --compress for a local transfer.
zfs-autobackup:
- Introduction (README.md)
- Getting started
- Full manual
- Mounting backup datasets
- Performance tips (recommended)
- Common problems and errors
- Thinning out obsolete snapshots
- Common snapshots, holds and bookmarks
- Handling ZFS encryption
- Transfer buffering, compression and rate limiting.
- Custom Pre- and post-snapshot commands
- Monitoring
Examples: