Skip to content

Performance

DatuX edited this page Nov 13, 2026 · 15 revisions

Performance tips

These tips can make zfs-autobackup much faster.

Speeding up SSH

Use persistent SSH connections. On the server that starts the backup, add this to ~/.ssh/config:

Host *
    ControlPath ~/.ssh/control-master-%r@%h:%p
    ControlMaster auto
    ControlPersist 3600

Thanks @mariusvw :)

Direct TCP network transfer

If SSH encryption is too slow, you can send the data without encryption via TCP. For example, with netcat:

zfs-autobackup ... --send-pipe "nc server_name 8023" --recv-pipe "nc -l -p 8023"

You can use any transfer program in this way. Only the zfs send/zfs recv data goes through this pipe. zfs-autobackup still uses SSH for all other commands.

See also issue #15.

Buffering and compression

  • --buffer: Adds an IO buffer. This makes the bursts of zfs send and zfs recv smoother.
  • --compress: Use this for a slow link and data that compresses well.

See Piping.

Less zfs commands

These options decrease the number of commands and the IO for each snapshot:

  • --no-holds: No hold and release commands.
  • --allow-empty: No commands to find out if a snapshot is empty.
  • --no-guid-check: No GUID check of the common snapshots.
  • --clear-mountpoint: zfs-autobackup does not mount the target datasets.

Disable progress (ZFS bug)

A bug in ZFS 2 adds a delay of 1 second to each transfer. This is a problem when you have many small transfers. Use --no-progress to prevent this.

Some statistics

Test setup: a laptop with an SKHynix_HFS512GD9TNI-L2B0B disk, ZFS 2.0.2, --no-holds, --allow-empty and SSH ControlMaster. The test measures the total time for 100 empty datasets:

Setup Time Speed
No SSH 15 s more than 6 datasets/s
--ssh-target or --ssh-source to localhost 20 s 5 datasets/s
--ssh-target and --ssh-source to localhost 24 s 4 datasets/s

A test with 2500 datasets also had no problems. Thus, zfs-autobackup can work with thousands of datasets.

If you need more performance, tell me.

TCP tuning

By default, Linux uses cubic for TCP congestion control. This is good for almost all situations. But for large transfers over the internet, short congestion (packet loss) can make the transfer slow for a long time.

scalable recovers much faster from packet loss. To use it, make the file /etc/sysctl.d/60-send-bulk-tcp.conf:

# Recovers rapidly from congestion
net.ipv4.tcp_congestion_control = scalable

More information: Scalable TCP. (Thanks @xrobau)

Clone this wiki locally