Skip to content

perf: optimizing collect_node_signatures function - #967

Merged
anderdc merged 2 commits into
entrius:testfrom
seroperson:scoring-walk-perf
May 8, 2026
Merged

perf: optimizing collect_node_signatures function#967
anderdc merged 2 commits into
entrius:testfrom
seroperson:scoring-walk-perf

Conversation

@seroperson

@seroperson seroperson commented May 4, 2026

Copy link
Copy Markdown
Contributor

Closes #966

Summary

This PR introduces set of optimizations to the collect_node_signatures:

  • Recursive walk_node approach is replaced with TreeCursor (tree.walk()) and a plain while loop.
  • Leaf ("leaf", node_type, text) replaced with ("leaf", node_type, bytes) to strip unnecessary utf-8 decoding.
  • Some micro-optimizations, like double node.text call, list allocations, and sticking to C calls.

Benchmark

Benchmarking e2e scoring:

  | fixture     | before (HEAD~1)   | after (this PR)   | speedup |
  |-------------|-------------------|-------------------|---------|
  | py-tiny     |     9.61us        |     8.39us        | 1.15x   |
  | py-small    |    87.65us        |    67.23us        | 1.30x   |
  | py-medium   |     2626us        |     2030us        | 1.29x   |
  | py-large    |    10827us        |     8192us        | 1.32x   |
  | rs-tiny     |    22.35us        |    18.24us        | 1.23x   |
  | rs-small    |   122.10us        |    97.90us        | 1.25x   |
  | rs-medium   |     2193us        |     1700us        | 1.29x   |
  | rs-large    |     9024us        |     6527us        | 1.38x   |
  | sh-tiny     |     7.68us        |     6.32us        | 1.21x   |
  | sh-small    |    76.20us        |    63.66us        | 1.20x   |
  | sh-medium   |     1633us        |     1198us        | 1.36x   |
  | sh-large    |     6595us        |     4775us        | 1.38x   |

python -m benchmarks.bench_collect_node_signatures (run in the branch with actual changes)

benchmarks/bench_collect_node_signatures.py

Test plan

Added tests/validator/test_collect_node_signatures.py which pins the exact function behavior, and also added more e2e tests to the tests/validator/test_token_scoring_integration.py to pin even more the existing scoring behavior (to not break anything in future).

@xiao-xiao-mao xiao-xiao-mao Bot added the enhancement New feature or request label May 4, 2026
@seroperson
seroperson force-pushed the scoring-walk-perf branch from 2e5c50b to fdcf6af Compare May 6, 2026 23:20

@anderdc anderdc left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good stuff

@anderdc
anderdc merged commit 6f60ed7 into entrius:test May 8, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf: unoptimized collect_node_signatures function

2 participants