Skip to content

Major hash tree refactoring#335

Merged
edubart merged 82 commits intomainfrom
share/hash-tree-on-disk
Nov 25, 2025
Merged

Major hash tree refactoring#335
edubart merged 82 commits intomainfrom
share/hash-tree-on-disk

Conversation

@edubart
Copy link
Collaborator

@edubart edubart commented Oct 21, 2025 •
edited
Loading

This PR completely reworks how hash tree are computed:

  • It is now orders magnitude faster to compute root hash
  • Hash tree now supports SHA-256 hasher
  • Hash tree now uses OpenMP and SIMD instructions to parallelize work
  • Hash tree can now persist to disk
  • A new C API was added to help constructing sparse and dense uarch computation hashes
  • Machines can now run mapped directly on disk

Along with many other changes, it combines months of work from Diego and me.

It also solves:

It supersedes:

diegonehab added 30 commits June 4, 2025 14:25
@diegonehab
refactor: unify mock_pma_entry implementations
6c9f9cc
@diegonehab
refactor: remove htif depencency on os.h
1153cc6
@diegonehab
refactor: move TLB management to interpret.cpp
cf6a83c
@diegonehab
refactor: clean up uarch state access mechanism
a464422
@diegonehab
refactor: remove some inline asm from uarch
b3190aa 
Using __builtin_memcpy instead.
@diegonehab
refactor: remove machine bridge
9dd83e0 
The machine class took over the role.
@diegonehab
refactor: split out i-interactive-state-access.h
e83beaa
@diegonehab
refactor: split out i-accept-scoped-note.h
63bc215
@diegonehab
refactor: split out i-prefer-state-access.h
5e11985 
And also i-prefer-uarch-state-access.h
@diegonehab
fix: format, typos, warnings, and cinttypes macros
7087601
@diegonehab
refactor: split out i-counters.h
45f9ea3 
Cleanup several of the DUMP_*.* implementations
@diegonehab
refactor: move uarch-printf to uarch/third-party
2bc0eb4
@diegonehab
refactor: make all DUMP_*.* work and pass linter
7f52135
@diegonehab
refactor: move log out of record state access
c1506b0
@diegonehab
fix: lint and format with new specs
2cf9c3c
@diegonehab
refactor: improve includes
8ba4e79
@diegonehab
refactor: make all state access objects const
32c7f0d
@diegonehab
refactor: make dump_regs work with mcycle and pc
4bab9fb
@diegonehab
refactor: add unique_mmap smart pointer and use it
6423fe5
@diegonehab
refactor: factor machine constructor
c0e7254
@diegonehab
refactor: clean up PMA class strucure
13c5406
@diegonehab
refactor: use NVI pattern on virtio classes
63ae3a6
@diegonehab
refactor: shadow PMAs becomes PMAS memory range
72a9c91
@diegonehab
refactor: split PMA and AR constants
208063b 
PMA is only used for the two-word description of address ranges in memory
All other constants use AR (for address range)
Also simplified machine::read_memory()
@diegonehab
refactor: remove the E flag
c2d4a10 
This increases the space for DID
Checking for empty address range simply checks its length.
Shadow address ranges are neither device nor memory.
This is because they cannot be read/written with read_device/write_device.
They do not implement get_host_memory() either.
@diegonehab
refactor: rename classes with "virtual" in name
aadc552 
i-virtual-machine        -> i-machine
virtual-machine          -> local-machine
jsonrpc-virtual-machine  -> jsonrpc-machine
clua-i-virtual-machine   -> clua-i-machine
@diegonehab
refactor: rename pm-type-name
e494969 
poor-type-name is more legible
@diegonehab
refactor: use std::ranges::lower_bound
207e9d7
@diegonehab
refactor: bump boost to 1.87 on mac
132aa97
@diegonehab
refactor: use attribute((format(printf, 1, 2)))
5ee6fc6
@edubart edubart self-assigned this Oct 21, 2025
@edubart edubart added the enhancement New feature or request label Oct 21, 2025
@edubart edubart added optimization Optimization epic refactor Restructuring code, while not changing its original functionality labels Oct 21, 2025
@edubart edubart added this to the v0.20.0 milestone Oct 21, 2025
@edubart edubart mentioned this pull request Oct 21, 2025
Closed
@edubart edubart requested a review from a team October 21, 2025 14:22
@edubart edubart requested a review from diegonehab October 22, 2025 13:42
@edubart edubart moved this from Todo to Waiting Review in Machine Unit Oct 22, 2025
This was linked to issues Nov 6, 2025
Make snapshots on COW filesystem possible and efficient #277
Closed
Optimize root hash computation while loading and saving snapshots #258
Closed
Support writing TLB with write_memory method #244
Closed
Optimize square root instruction #221
Closed
Snapshot and shared flash drives #203
Closed
This was referenced Nov 6, 2025
Closed
Closed
Closed
@github-project-automation github-project-automation bot moved this from Waiting Review to Waiting Merge in Machine Unit Nov 25, 2025
@edubart edubart merged commit 1cce7ce into main Nov 25, 2025
8 checks passed
@edubart edubart deleted the share/hash-tree-on-disk branch November 25, 2025 15:33
@github-project-automation github-project-automation bot moved this from Waiting Merge to Done in Machine Unit Nov 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@diegonehab diegonehab diegonehab approved these changes

Assignees

@edubart edubart

Labels

enhancement New feature or request epic optimization Optimization refactor Restructuring code, while not changing its original functionality

Projects

Machine Unit
Status: Done

Milestone

v0.20.0

2 participants

@edubart @diegonehab