Skip to content

Parallel vectorized Keccak-256/SHA-256 merkle tree hashing#332

Closed
edubart wants to merge 6 commits intorefactor/new-merkle-treefrom
feature/parallel-keccak
Closed

Parallel vectorized Keccak-256/SHA-256 merkle tree hashing#332
edubart wants to merge 6 commits intorefactor/new-merkle-treefrom
feature/parallel-keccak

Conversation

@edubart
Copy link
Collaborator

@edubart edubart commented Jul 24, 2025

This PR:

  • Adds custom Keccak-256/SHA-256 implementation using SIMD instructions optimized for hash operations we use in our hash tree (with fixed size leaves/nodes), it uses compiler vector extensions (GCC/Clang compatible) for a simpler implementation (without compiler intrinsics).
  • Adds specialized code paths leveraging AVX2 and AVX512 instructions on x86_64 using GCC function multiversioning
  • Optimizes pristine page checking with better SIMD instruction utilization
  • Makes hash function configurable (adapted from Add configurable hash tree target support #327) - defaults to Keccak-256, can be set to SHA-256 via --hash-tree=hash_function:sha256
  • Adds new C API hash functions (cm_get_hash and cm_get_concat_hash)
  • Adds Lua hash functions cartesi.keccak256 (renamed from cartesi.kecccak) and cartesi.sha256 using the new C APIs
  • Optimizes hash tree multithreading by reducing atomic operations and improving scheduling

The overall speed boost of periodic sparse hashing at 32K cycles relative to base branch (refactor/new-merkle-tree) is more than 2x in average for must benchmark I run (on my hardware supporting only AVX2 instructions).

@edubart edubart added this to the v0.20.0 milestone Jul 24, 2025
@edubart edubart self-assigned this Jul 24, 2025
@edubart edubart added enhancement New feature or request optimization Optimization labels Jul 24, 2025
@edubart edubart moved this from Todo to In Progress in Machine Unit Jul 31, 2025
@edubart
Copy link
Collaborator Author

edubart commented Sep 19, 2025

this was superseded by #333

@edubart edubart closed this Sep 19, 2025
@github-project-automation github-project-automation bot moved this from In Progress to Done in Machine Unit Sep 19, 2025
@edubart edubart deleted the feature/parallel-keccak branch December 2, 2025 22:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request optimization Optimization

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

1 participant