feat: improve block selection in `stacks-inspect` #6735

brice-stacks · 2025-12-02T15:56:56Z

Improves the UX by merging validate-naka-block and validate-block, improves the lookup time by optimizing the queries, and improves the output for a better experience. Adds --early-exit flag to exit on first error instead of completing all blocks.

Improves the UX by merging `validate-naka-block` and `validate-block,` improves the lookup time by optimizing the queries, and improves the output for a better experience. Adds `--early-exit` flag to exit on first error instead of completing all blocks.

codecov · 2025-12-02T16:50:42Z

Codecov Report

❌ Patch coverage is 1.70213% with 231 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.95%. Comparing base (7ff75a2) to head (48db5d0).
⚠️ Report is 52 commits behind head on develop.

Files with missing lines	Patch %	Lines
contrib/stacks-inspect/src/lib.rs	0.00%	230 Missing ⚠️
stackslib/src/chainstate/stacks/db/blocks.rs	80.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #6735      +/-   ##
===========================================
+ Coverage    70.05%   71.95%   +1.89%     
===========================================
  Files          578      579       +1     
  Lines       358373   360160    +1787     
===========================================
+ Hits        251064   259140    +8076     
+ Misses      107309   101020    -6289

Files with missing lines	Coverage Δ
contrib/stacks-inspect/src/main.rs	`0.00% <ø> (ø)`
stackslib/src/chainstate/stacks/db/blocks.rs	`82.66% <80.00%> (+10.09%)`	⬆️
contrib/stacks-inspect/src/lib.rs	`4.51% <0.00%> (-15.37%)`	⬇️

... and 390 files with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7ff75a2...48db5d0. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

contrib/stacks-inspect/src/lib.rs

…eader_info`

contrib/stacks-inspect/src/lib.rs

francesco-stacks · 2025-12-04T12:51:01Z

contrib/stacks-inspect/src/lib.rs

+fn collect_block_entries_for_selection(
+    db_path: &str,
+    selection: &BlockSelection,
+    chainstate: &StacksChainState,
+) -> Vec<BlockScanEntry> {


sanity check: isn't this method currently returning 2n blocks when called with BlockSelection::Last and BlockSelection::First? seems like it will get the first N and last N blocks from both tables

You're right. I decided to just remove first, since that doesn't really seem like a useful scenario any way, and then fixed last.

I decided to just remove first, since that doesn't really seem like a useful scenario any way

And if someone needs it after all, they can use a height range starting at 0.

francesco-stacks · 2025-12-04T12:57:54Z

contrib/stacks-inspect/src/lib.rs

+                panic!("Failed to open staging blocks DB at {staging_blocks_db_path}: {e}");
+            });
+    let sql = format!(
+        "SELECT index_block_hash, consensus_hash, anchored_block_hash, height FROM staging_blocks {clause}"


nit: since we are only consuming index_block_hash, maybe we can simplity the query as SELECT index_block_hash FROM staging_blocks {clause}

Yeah, I had that there originally because I needed other fields when I was also reusing the new structure in other places, but then ended up deleting that code. I'll delete these now.

francesco-stacks · 2025-12-04T12:58:25Z

contrib/stacks-inspect/src/lib.rs

-    for (i, index_block_hash) in index_block_hashes.iter().enumerate() {
-        if i % 100 == 0 {
-            println!("Checked {i}...");
+    let sql = format!("SELECT index_block_hash, height FROM nakamoto_staging_blocks {clause}");


nit: since we are only consuming index_block_hash, maybe we can simplity the query as SELECT index_block_hash FROM nakamoto_staging_blocks {clause}

francesco-stacks · 2025-12-04T13:33:45Z

contrib/stacks-inspect/src/lib.rs

+    let mut seen = HashSet::new();
+    let mut entries = Vec::new();


suuuuper nit: These two variables hold over 5 million index_block_hash values each when the function is called with BlockSelection::All, stored as 64 bytes hex strings. ~320 MB for each variable (plus additional overhead from the data structures). It's not a concern right now, but we could reduce memory usage by about half if we move the

let index_block_hash = StacksBlockId::from_hex(&index_block_hash_hex).unwrap();

processing inside this function and avoid keeping the hex strings around.

as always, feel free to skip the changes for the nits

This is a good point. Also, a good reason to just remove the seen tracking altogether.

Remove the `seen` tracking since it doesn't really gain us much and uses a lot of space when processing many blocks.

benjamin-stacks · 2025-12-10T16:56:19Z

contrib/stacks-inspect/src/lib.rs

+            if start > end {
+                return Err("<start-block> must be <= <end-block>".into());


If we want to keep end exclusive (see other comment), start == end wouldn't make much sense (it would always be zero blocks), so it would indicate a likely mistake. So in that case maybe we should do

Suggested change

if start > end {

return Err("<start-block> must be <= <end-block>".into());

if start >= end {

return Err("<start-block> must be < <end-block>".into());

instead?

benjamin-stacks · 2025-12-10T17:02:59Z

contrib/stacks-inspect/src/lib.rs

+                let blocks = end.saturating_sub(*start);
+                format!("WHERE orphaned = 0 ORDER BY index_block_hash ASC LIMIT {start}, {blocks}")


This makes end exclusive. We probably want it inclusive for consistency with HeightRange?

Actually even worse, I don't think this even works?

Say the user wants blocks 10 through 20. If there are 10 nakamoto blocks with a hash that is smaller than any eopch2 block, then the first block in the overall result set is the first epoch2 block. But because our OFFSET was 10, we never retrieved that block.

benjamin-stacks · 2025-12-10T17:36:08Z

contrib/stacks-inspect/src/lib.rs

        next_staging_block.commit_burn,
        next_staging_block.sortition_burn,
    );
+    Ok(())


I assume you meant to change replay_block to return a Result<,> and then return that result here?

Returning an unconditional Ok(()) regardless of what happens in replay_block (if it even ever returns -- it has some process::exits in there) doesn't seem like it's the intent.

brice-stacks requested a review from benjamin-stacks December 2, 2025 15:57

fix: clippy warnings

1d9260b

benjamin-stacks reviewed Dec 2, 2025

View reviewed changes

contrib/stacks-inspect/src/lib.rs Outdated Show resolved Hide resolved

contrib/stacks-inspect/src/lib.rs Outdated Show resolved Hide resolved

contrib/stacks-inspect/src/lib.rs Outdated Show resolved Hide resolved

brice-stacks requested a review from francesco-stacks December 2, 2025 17:59

brice-stacks added 3 commits December 2, 2025 14:03

refactor: cleanup block selection query

d461655

fix: panic when duplicate block is found during replay

2d50158

refactor: remove unnecessary mut in `StacksChainState::get_parent_h…

e51f898

…eader_info`

brice-stacks requested a review from benjamin-stacks December 2, 2025 19:06

chore: fix formatting

aa76fe4

francesco-stacks reviewed Dec 4, 2025

View reviewed changes

brice-stacks added 3 commits December 5, 2025 14:15

fix(stacks-inspect): last option and remove first option

1410c4a

chore: remove unnecessary fields from queries

6fd437d

feat: store StacksBlockId directly instead of String

48db5d0

Remove the `seen` tracking since it doesn't really gain us much and uses a lot of space when processing many blocks.

brice-stacks requested a review from francesco-stacks December 5, 2025 19:30

Merge branch 'develop' into feat/stacks-inspect-improvements

35f883e

benjamin-stacks reviewed Dec 10, 2025

View reviewed changes

		if start > end {
		return Err("<start-block> must be <= <end-block>".into());

		let blocks = end.saturating_sub(*start);
		format!("WHERE orphaned = 0 ORDER BY index_block_hash ASC LIMIT {start}, {blocks}")

feat: improve block selection in stacks-inspect #6735

Are you sure you want to change the base?

feat: improve block selection in stacks-inspect #6735

Conversation

brice-stacks commented Dec 2, 2025

Uh oh!

codecov bot commented Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

feat: improve block selection in `stacks-inspect` #6735

feat: improve block selection in `stacks-inspect` #6735

codecov bot commented Dec 2, 2025 •

edited

Loading