Skip to content

ci: add benchmark workflow #1643

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
RodrigoVillar wants to merge 4 commits into
base: main
Choose a base branch
from
Open

ci: add benchmark workflow #1643

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
aee21eb
ci: add benchmark workflow
RodrigoVillar Jan 27, 2026
d792faa
chore: add PR workflow [REMOVE BEFORE MERGING]
RodrigoVillar Jan 27, 2026
3cf75e0
chore: remove PR workflow
RodrigoVillar Jan 27, 2026
e5dce27
Merge branch 'main' into rodrigo/add-benchmark-workflow
RodrigoVillar Jan 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions .github/workflows/benchmarks.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
name: firewood-benchmarks

on:
push:
branches: [main]
workflow_dispatch:

env:
CARGO_TERM_COLOR: always
RUST_BACKTRACE: full

jobs:
bench:
runs-on: ubuntu-latest
Copy link
Contributor

@Elvis339 Elvis339 Jan 28, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this run on a dedicated/self-hosted runner instead of ubuntu-latest? Shared runners have variable performance which can make benchmark results unreliable for tracking regressions across runs.

Copy link
Contributor Author

@RodrigoVillar RodrigoVillar Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What dedicated/self-hosted runners are available to use? Open to switching but I'm unaware of the options.

Copy link
Contributor

@Elvis339 Elvis339 Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For reference, these self-hosted runners are available (managed via ARC):

Runner Spec Storage
avalanche-avalanchego-runner 4 vCPU, 16 GB EBS
avago-runner-m6i-4xlarge-ebs-fast 16 vCPU, 64 GB Fast EBS
avago-runner-i4i-4xlarge-local-ssd 16 vCPU, 128 GB Local NVMe

Worth reaching out to security/GitHub admin for a full list. Not a blocker tho ubuntu-latest is fine to start.

Copy link
Contributor

@Elvis339 Elvis339 Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Full list of self-hosted runners: https://github.com/ava-labs/devops-argocd/blob/main/base/system/actions-runners/action-runner.yaml

steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
with:
shared-key: "maxperf-ethhash-logger"
cache-all-crates: true
cache-workspace-crates: true
- name: Cargo Fetch (run `cargo generate-lockfile` if this fails)
run: cargo fetch --locked --verbose
- name: Build
run: cargo build --frozen --profile maxperf --features ethhash,logger --workspace --benches
- name: Run firewood crate benchmarks
run: cargo bench --frozen --profile maxperf --features ethhash,logger -p firewood --bench hashops -- --noplot
- name: Run storage crate benchmarks
run: cargo bench --frozen --profile maxperf --features ethhash,logger -p firewood-storage --bench serializer -- --noplot
- name: Upload benchmark results
uses: actions/upload-artifact@v4
with:
name: benchmark-results
path: target/criterion