-
Notifications
You must be signed in to change notification settings - Fork 171
[storage] move to crc32c #2591
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[storage] move to crc32c #2591
Conversation
Deploying monorepo with
|
| Latest commit: |
4c340cb
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://7c8d067e.monorepo-eu0.pages.dev |
| Branch Preview URL: | https://storage-faster-crc.monorepo-eu0.pages.dev |
|
Created some benchmarks for the crates directly: https://andresilva.github.io/crc32-bench/report/ Benchmark code is here https://github.com/andresilva/crc32-bench. |
d12da5d to
d80c0b9
Compare
|
crc-fast did a new release that makes it faster at small inputs (https://github.com/awesomized/crc-fast-rust/releases/tag/1.9.0):
|
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
commonware-mcp | 4c340cb | Jan 01 2026, 05:00 PM |
d4bcb4f to
4c340cb
Compare
|
Replaced by #2740. |
Codecov Report❌ Patch coverage is
@@ Coverage Diff @@
## main #2591 +/- ##
=======================================
Coverage 92.62% 92.62%
=======================================
Files 357 358 +1
Lines 102956 103007 +51
=======================================
+ Hits 95366 95415 +49
- Misses 7590 7592 +2
... and 3 files with indirect coverage changes Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
This PR update the storage crate to use CRC32C for checksumming. For 4 KB blocks, CRC-32C (Castagnoli) has higher guaranteed hamming distance than the classic CRC-32 (IEEE). Concretely, CRC-32C guarantees detection of any combination of up to 5 bit errors within a 4 KB block, whereas CRC-32 (IEEE) only guarantees detection of up to ~3-4 bit errors at that size. Both detect all burst errors up to 32 bits, but CRC-32C has fewer undetected multi-bit patterns beyond that.
I ran the full storage benchmark suite against main, crc_fast with CRC32 algorithm and crc_fast with CRC32C algorithm (this PR). Benchmarks were done on a AMD Ryzen 9 9950X3D.
main vs fast-crc32
main vs fast-crc32c (this PR)
fast-crc32 vs fast-crc32c
Closes #2589.