hole_birth FAQ

Short explanation

hole_birth has/had bugs, the result of which is that, if you do a zfs send -i (or -R, since it uses -i) from an affected dataset, the receiver will not see any checksum or other errors, but will not match the source.

0.6.5.8 and 0.7.0-rc1 (and above) default to ignoring the faulty metadata which causes this issue on the sender side.

FAQ

I have a pool with hole_birth enabled, how do I know if I'm affected?

It's technically possible to calculate whether you have any affected files, but it requires scraping zdb output for each file in each snapshot in each dataset, which is a combinatoric nightmare. (If you really want it, there's a proof of concept here

Is there any less painful way to fix this if we've already received an affected snapshot?

No, the data you need was simply not present in the send stream, unfortunately, and can't feasibly be rewritten in place.

Long explanation

hole_birth is a feature to speed up ZFS send -i - in particular, ZFS used to not store metadata on when "holes" (sparse regions) in files were created, so every zfs send -i needed to include every hole.

hole_birth, as the name implies, added tracking for the txg (transaction group) when a hole was created, so that zfs send -i could only send holes that had a birth_time between (starting snapshot txg) and (ending snapshot txg), and life was wonderful.

Unfortunately, hole_birth had a number of edge cases where it could "forget" to set the birth_time of holes in some cases, causing it to record the birth_time as 0 (the value used prior to hole_birth, and essentially equivalent to "since file creation").

This meant that, when you did a zfs send -i, since zfs send doesn't have any knowledge of the surrounding snapshots when sending a given snapshot, it would see the creation txg as 0, conclude "oh, it's 0, I must have already sent this before", and not include it.

This means that, on the receiving side, it doesn't know those holes should exist, and doesn't create them - leading to differences between what you have on the source and the destination.

0.6.5.8 and 0.7.0-rc1 (and above) default to ignoring this metadata and always sending holes with birth_time 0, configurable using the tunable known as "ignore_hole_birth" or "send_holes_without_birth_time" (the latter is what OpenZFS standardized on; 0.6.5.8 only has the former, but for any ZoL version with send_holes_without_birth_time, they point to the same value, so changing either will work).