Skip to content
Open
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
966eb57
ci(avalanchego-build)
Elvis339 Sep 25, 2025
ec8fe0e
ci: temp. run custom action on github runners
Elvis339 Sep 25, 2025
36f8420
ci: temp. run custom action on github runners with ubuntu-latest tag
Elvis339 Sep 25, 2025
8a517e5
ci(gh_runners): temp. checkout
Elvis339 Sep 25, 2025
04a9aa8
ci: remove checkout
Elvis339 Sep 25, 2025
1477677
ci: add checkout back
Elvis339 Sep 25, 2025
58784da
ci: remove defaults from coreth and libevm
Elvis339 Sep 25, 2025
bec87c7
debug
Elvis339 Sep 25, 2025
ebeafdc
ci: use $GITHUB_ACTION_PATH
Elvis339 Sep 25, 2025
17af2a7
ci: github runners
Elvis339 Sep 25, 2025
76b7009
ci: github runners use nix and use realpath
Elvis339 Sep 25, 2025
7aafbae
ci: github runners run help from bin
Elvis339 Sep 25, 2025
35e7eb6
ci: self-hosted runners
Elvis339 Sep 25, 2025
2ea5241
ci: self-hosted runners install build-essential
Elvis339 Sep 25, 2025
0e863be
ci: self-hosted runners run & build reexecution bench
Elvis339 Sep 25, 2025
b63ffe4
ci: self-hosted runners run & build reexecution bench
Elvis339 Sep 25, 2025
18e728c
ci(reexec): temp. disable go setup
Elvis339 Sep 25, 2025
3dfbc38
re-run re-exec
Elvis339 Sep 25, 2025
dfa8c81
ci: set permissions
Elvis339 Sep 25, 2025
06708b0
ci(avalanchego-build-action): use composable Firewood action
Elvis339 Sep 26, 2025
fb8a6cf
ci: reexecution bench with firewood custom action
Elvis339 Sep 26, 2025
da4c389
ci: add back ci jobs & remove tmp. jobs
Elvis339 Sep 26, 2025
6a09874
Merge branch 'master' into composable-ci-action
Elvis339 Sep 26, 2025
2c84ad9
ci: uncomment c-chain-reexecution-benchmark-* pull_request directive
Elvis339 Sep 26, 2025
f1a55c7
chore: address copilot PR review
Elvis339 Sep 26, 2025
4702f81
ci(c-chain-reexecution-benchmark)
Elvis339 Sep 26, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 101 additions & 0 deletions .github/actions/avalanchego-build-action/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# AvalancheGo Build Action

## Overview
This action provides composable CI capabilities for building AvalancheGo with custom dependency versions.

### Why this exists?
Solves CI composability problems by enabling repositories to build AvalancheGo with specific dependency versions without complex setup or cross-repository coordination.

## Modes

**BUILD Mode** (target specified): Produces a ready-to-use binary with custom dependencies - for teams that need the executable.

When `target` parameter is provided, the action:
1. Checks out AvalancheGo at specified version
2. Replaces dependencies with custom versions
3. Builds the specified binary
4. Makes binary available via both output parameter AND artifact upload
5. Optionally executes binary with provided args

**SETUP Mode** (no target): Prepares the build environment with custom dependencies - for teams that need custom build workflows.

When `target` parameter is empty, the action:
1. Checks out AvalancheGo at specified version
2. Replaces dependencies with custom versions
3. Sets up build environment for consumer's custom workflow

## Security Model
- Repository Restriction: Only allows dependencies from `github.com/ava-labs/*` repositories
- No Custom Forks: Prevents supply chain attacks from malicious forks
- Commit Validation: Validates dependency versions reference ava-labs repositories only

## Inputs

| Input | Description | Required | Default |
|-------|-------------|----------|---------|
| `target` | Which binary to build (`avalanchego`, `reexecution`). Determines BUILD vs SETUP mode | No | `''` |
| `args` | Command-line arguments for target executable (BUILD mode only) | No | `''` |
| `checkout-path` | Directory path where AvalancheGo will be checked out | No | `'avalanchego'` |
| `avalanchego` | AvalancheGo version (commit SHA, branch, tag) | No | `'main'` |
| `firewood` | Firewood version. Consumer should run Firewood shared workflow first | No | `''` |
| `coreth` | Coreth version (commit SHA, branch, tag) | No | `'main'` |
| `libevm` | LibEVM version (commit SHA, branch, tag) | No | `'main'` |

## Outputs

| Output | Description |
|--------|-------------|
| `binary-path` | Absolute path to built binary (BUILD mode only) |

## Usage Examples

### BUILD Mode - Binary Available for Consumer

```yaml
- name: Build AvalancheGo
id: build
uses: ./.github/actions/avalanchego-build-action
with:
target: "avalanchego"
coreth: "v0.12.5"
libevm: "v1.0.0"

- name: Use binary via output parameter
run: ${{ steps.build.outputs.binary-path }} --network-id=local

- name: Or download as artifact
uses: actions/download-artifact@v4
with:
name: avalanchego-avalanchego_main-coreth_v0.12.5-libevm_v1.0.0

- name: Use downloaded artifact
run: ./avalanchego --network-id=local
```
### SETUP Mode - Custom Workflow
```yaml
- name: Setup AvalancheGo with custom dependencies
uses: ./.github/actions/avalanchego-build-action
with:
checkout-path: "build/avalanchego"
coreth: "my-feature-branch"
libevm: "experimental-branch"

- name: Run custom build commands
run: |
cd build/avalanchego
./scripts/run_task.sh reexecute-cchain-range
./scripts/run_task.sh my-custom-task
Comment on lines +85 to +89
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would not work because the current task definition uses go run, so it would not use the the compiled test binary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It works here.

```
## Artifact Naming
Artifacts are named with the complete dependency matrix for full traceability:
**Format:** `{target}-avalanchego_{version}-coreth_{version}-libevm_{version}[-firewood_{version}]`

**Examples:**
- `avalanchego-avalanchego_main-coreth_main-libevm_main` (default versions)
- `avalanchego-avalanchego_v1.11.0-coreth_v0.12.5-libevm_v1.0.0-firewood_ffi%2Fv0.0.13` (with firewood)
- `reexecution-avalanchego_my-branch-coreth_main-libevm_experimental-firewood_abc123` (mixed versions)
120 changes: 120 additions & 0 deletions .github/actions/avalanchego-build-action/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
name: 'AvalancheGo Build Action'
description: 'Build AvalancheGo with custom dependencies. Dual mode: BUILD (with target) creates binary, SETUP (no target) prepares environment.'

inputs:
target:
description: 'Binary to build (avalanchego, reexecution). If provided: BUILD mode. If empty: SETUP mode.'
required: false
default: ''
args:
description: 'Arguments for target executable (BUILD mode only)'
required: false
default: ''
checkout-path:
description: 'Directory path where AvalancheGo will be checked out'
required: false
default: 'avalanchego'
avalanchego:
description: 'AvalancheGo version (commit SHA, branch name, or tag)'
required: false
default: ${{ github.sha }}
firewood:
description: 'Firewood version (commit SHA, branch, tag, or ffi/vX.Y.Z for pre-built)'
required: false
default: ''
coreth:
description: 'Coreth version (commit SHA, branch name, or tag)'
required: false
default: ''
libevm:
description: 'LibEVM version (commit SHA, branch name, or tag)'
required: false
default: ''

outputs:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we also output the artifact name (rather than solely relying on the convention described in README.md)

binary-path:
description: 'Absolute path to built binary (BUILD mode only)'
value: ${{ steps.build.outputs.binary-path }}

runs:
using: 'composite'
steps:
- name: Checkout AvalancheGo
uses: actions/checkout@v4
with:
repository: 'ava-labs/avalanchego'
ref: ${{ inputs.avalanchego }}
path: ${{ inputs.checkout-path }}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does AvalancheGo use checkout path and everything else just uses the name of the repo it's checking out?

Would it make sense to take in one parameter to use as a working directory and then check out each required remote as a subdirectory?

Copy link
Contributor Author

@Elvis339 Elvis339 Sep 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The avalanchego checkout path is the main workspace where we build the final binary. It needs to be configurable because consumers might want to integrate it into existing directory structures.

Firewood has some storage requirements its database operations benefit from NVMe for performance, but compilation can happen anywhere. The Firewood action handles its own optimal storage detection internally.
A single working directory would force all components to use the same storage type.

The current approach lets each component use optimal storage:

  • Firewood runtime: NVMe when available (handled by its action)
  • AvalancheGo: Standard storage (sufficient for compilation)
  • Coreth/LibEVM: Handled by Go's dependency system

- name: Setup Go for project
uses: ./.github/actions/setup-go-for-project # Note: If Nix-specific functionality is needed, consumer should install Nix as a prerequisite to this action
- name: Setup Firewood FFI
if: inputs.firewood != ''
id: firewood
uses: ava-labs/firewood/.github/actions/build-action@composable-ci-action
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this action defined? I don't see it on Firewood main

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's defined here composable-ci-action branch. I used it to test these things can work.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit hard to verify as a reviewer since it's not on main and doesn't have a reference to the branch where it's available. How does this work for the workflow runs linked in the PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added link to Firewood action in PR description, thanks for pointing it out.

This run is using Firewood action along with AvalancheGo to run reexecution test on self-hosted runners. You can find the commit here. (I'll attach commits in the PR description as well).

The workflow in the commit proves that:

  1. Firewood build action is:
  2. AvalancheGo build action is linking dependencies and running reexecution benchmark in composable way

with:
version: ${{ inputs.firewood }}
- name: Checkout Coreth
if: inputs.coreth != '' && inputs.coreth != 'master'
uses: actions/checkout@v4
with:
repository: 'ava-labs/coreth'
ref: ${{ inputs.coreth }}
path: 'coreth'
- name: Checkout LibEVM
if: inputs.libevm != '' && inputs.libevm != 'main'
uses: actions/checkout@v4
with:
repository: 'ava-labs/libevm'
ref: ${{ inputs.libevm }}
path: 'libevm'
- name: Replace dependencies with local checkouts
if: ${{ inputs.firewood != '' || inputs.coreth != '' || inputs.libevm != '' }}
shell: bash
working-directory: ./${{ inputs.checkout-path }}
run: |
# Replace Firewood FFI if provided
if [ "${{ inputs.firewood }}" != "" ]; then
echo "Replacing Firewood FFI with: ${{ steps.firewood.outputs.ffi-path }}"
go mod edit -replace github.com/ava-labs/firewood-go-ethhash/ffi=${{ steps.firewood.outputs.ffi-path }}
fi
# Replace Coreth if provided and not default
if [ "${{ inputs.coreth }}" != "" ] && [ "${{ inputs.coreth }}" != "master" ]; then
echo "Replacing Coreth with local checkout: ../coreth"
go mod edit -replace github.com/ava-labs/coreth=../coreth
fi
# Replace LibEVM if provided and not default
if [ "${{ inputs.libevm }}" != "" ] && [ "${{ inputs.libevm }}" != "main" ]; then
echo "Replacing LibEVM with local checkout: ../libevm"
go mod edit -replace github.com/ava-labs/libevm=../libevm
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given we are checking out versions that are available in GitHub, we can just use go get with the updated version rather than checking them each out locally. If we wanted to checkout and compile a specific version of Firewood, we'd need the source code, but currently this is just using the published golang ffi bindings of Firewood as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's good input, I'll run go get instead of go mod edit and repo checkout.

The Firewood workflow handles building form source and using pre-built FFI.
https://github.com/ava-labs/firewood/blob/composable-ci-action/.github/actions/build-action/action.yml

fi
go mod tidy
go mod download
Comment on lines +93 to +94
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't this leave a diff in the resulting checked out code that may cause future actions to find an unexpected diff and fail? Should that be documented or cleaned up?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it will, it should probably be cleaned up. Will handle that case.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could probably do less here and only support setup rather than adding a script that builds different binaries once setup has completed.

- name: Build and run target (BUILD mode)
if: inputs.target != ''
id: build
shell: bash
working-directory: ./${{ inputs.checkout-path }}
run: |
if [ -n "${{ inputs.args }}" ]; then
OUTPUT=$(./scripts/build_target.sh "${{ inputs.target }}" ${{ inputs.args }})
else
OUTPUT=$(./scripts/build_target.sh "${{ inputs.target }}")
fi
# Extract binary path from script output
BINARY_PATH=$(echo "$OUTPUT" | grep "BINARY_PATH=" | cut -d'=' -f2)
# Convert to absolute path to avoid relative pathing issues
ABSOLUTE_BINARY_PATH=$(realpath "$BINARY_PATH")
# Set output for consumer use
echo "binary-path=$ABSOLUTE_BINARY_PATH" >> $GITHUB_OUTPUT
- name: Upload binary artifact (BUILD mode)
if: inputs.target != ''
uses: actions/upload-artifact@v4
with:
name: ${{ inputs.target }}-avalanchego_${{ inputs.avalanchego }}-coreth_${{ inputs.coreth }}-libevm_${{ inputs.libevm }}${{ inputs.firewood != '' && format('-firewood_{0}', inputs.firewood) || '' }}
path: ${{ steps.build.outputs.binary-path }}
52 changes: 52 additions & 0 deletions .github/actions/avalanchego-build-action/build_target.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
#!/usr/bin/env bash
# Build AvalancheGo target binary and optionally execute with arguments
# Usage: ./scripts/build_target.sh <target> [args...]

set -euo pipefail

if [[ $# -eq 0 ]]; then
echo "Usage: $0 <target> [args...]"
echo "Valid targets: avalanchego, reexecution"
exit 1
fi

TARGET="$1"
shift # Remove target from arguments, remaining args are for execution

case "$TARGET" in
"avalanchego")
if [[ ! -f "./scripts/run_task.sh" || ! -x "./scripts/run_task.sh" ]]; then
echo "Error: ./scripts/run_task.sh not found or not executable"
exit 1
fi
./scripts/run_task.sh build
EXECUTABLE="./build/avalanchego"

if [[ ! -f "$EXECUTABLE" ]]; then
echo "Error: Binary $EXECUTABLE was not created"
exit 1
fi

echo "BINARY_PATH=$PWD/$EXECUTABLE"

if [[ $# -gt 0 ]]; then
"$EXECUTABLE" "$@"
fi
;;
"reexecution")
# Compile reexecution benchmark test into binary
go test -c github.com/ava-labs/avalanchego/tests/reexecute/c -o reexecute-benchmark
EXECUTABLE="./reexecute-benchmark"

if [[ ! -f "$EXECUTABLE" ]]; then
echo "Error: Binary $EXECUTABLE was not created"
exit 1
fi

echo "BINARY_PATH=$PWD/$EXECUTABLE"
;;
*)
echo "Error: Invalid target '$TARGET'. Valid targets: avalanchego, reexecution"
exit 1
;;
esac
1 change: 1 addition & 0 deletions .github/actions/c-chain-reexecution-benchmark/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ runs:
prometheus_password: ${{ inputs.prometheus-password }}
grafana_dashboard_id: 'Gl1I20mnk/c-chain'
runtime: "" # Set runtime input to empty string to disable log collection
prometheus_url: ${{ inputs.prometheus-push-url }}

- name: Compare Benchmark Results
uses: benchmark-action/github-action-benchmark@v1
Expand Down