Replace MD5 with CRC32 for 2x checksum speed

[adiov]

While running this script on a large pool, I noticed that that the MD5 checksum calculation is taking up to 2-3 minutes on the large video project files. By changing to CRC32, I was able to cut down the checksum time by nearly 50% on most files.

For the purpose of adversarial integrity checking, CRC32 is not adequate, and neither is MD5. However, for the purpose of successful copying and corruption checking, CRC32 is adequate. Using MD5 for this script is burning double the CPU cycles for no gain.

[meo-pill]

For the use of awk i would advocate against because if you have a lot of small files it would have a big impact on the performance.
In my opinion it's better to use the % that is included in Bash

Ps : I speak about the removals of the extension at the end of the "checksum"

[adiov]

When running on 5000 small files, there is indeed a performance difference due to spinning up the awk subprocess, about 16% more time.

./zfs-inplace-rebalancing-noawk.sh --checksum true --passes 1   112.50s user 103.84s system 103% cpu 3:28.98 total
./zfs-inplace-rebalancing.sh --checksum true --passes 1 134.23s user 135.18s system 111% cpu 4:02.25 total

Since both Bash and zsh support it, it makes sense to reclaim that extra CPU time and use the % builtin.

[markusressel]

Thx!