-
Notifications
You must be signed in to change notification settings - Fork 802
ci: composable avalanchego action #4341
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from 25 commits
966eb57
ec8fe0e
36f8420
8a517e5
04a9aa8
1477677
58784da
bec87c7
ebeafdc
17af2a7
76b7009
7aafbae
35e7eb6
2ea5241
0e863be
b63ffe4
18e728c
3dfbc38
dfa8c81
06708b0
fb8a6cf
da4c389
6a09874
2c84ad9
f1a55c7
4702f81
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,101 @@ | ||
# AvalancheGo Build Action | ||
|
||
## Overview | ||
This action provides composable CI capabilities for building AvalancheGo with custom dependency versions. | ||
|
||
### Why this exists? | ||
Solves CI composability problems by enabling repositories to build AvalancheGo with specific dependency versions without complex setup or cross-repository coordination. | ||
|
||
## Modes | ||
|
||
**BUILD Mode** (target specified): Produces a ready-to-use binary with custom dependencies - for teams that need the executable. | ||
|
||
When `target` parameter is provided, the action: | ||
1. Checks out AvalancheGo at specified version | ||
2. Replaces dependencies with custom versions | ||
3. Builds the specified binary | ||
4. Makes binary available via both output parameter AND artifact upload | ||
5. Optionally executes binary with provided args | ||
|
||
**SETUP Mode** (no target): Prepares the build environment with custom dependencies - for teams that need custom build workflows. | ||
|
||
When `target` parameter is empty, the action: | ||
1. Checks out AvalancheGo at specified version | ||
2. Replaces dependencies with custom versions | ||
3. Sets up build environment for consumer's custom workflow | ||
|
||
## Security Model | ||
- Repository Restriction: Only allows dependencies from `github.com/ava-labs/*` repositories | ||
- No Custom Forks: Prevents supply chain attacks from malicious forks | ||
- Commit Validation: Validates dependency versions reference ava-labs repositories only | ||
|
||
## Inputs | ||
|
||
| Input | Description | Required | Default | | ||
|-------|-------------|----------|---------| | ||
| `target` | Which binary to build (`avalanchego`, `reexecution`). Determines BUILD vs SETUP mode | No | `''` | | ||
| `args` | Command-line arguments for target executable (BUILD mode only) | No | `''` | | ||
| `checkout-path` | Directory path where AvalancheGo will be checked out | No | `'avalanchego'` | | ||
| `avalanchego` | AvalancheGo version (commit SHA, branch, tag) | No | `'main'` | | ||
| `firewood` | Firewood version. Consumer should run Firewood shared workflow first | No | `''` | | ||
| `coreth` | Coreth version (commit SHA, branch, tag) | No | `'main'` | | ||
| `libevm` | LibEVM version (commit SHA, branch, tag) | No | `'main'` | | ||
|
||
## Outputs | ||
|
||
| Output | Description | | ||
|--------|-------------| | ||
| `binary-path` | Absolute path to built binary (BUILD mode only) | | ||
|
||
## Usage Examples | ||
|
||
### BUILD Mode - Binary Available for Consumer | ||
|
||
```yaml | ||
- name: Build AvalancheGo | ||
id: build | ||
uses: ./.github/actions/avalanchego-build-action | ||
with: | ||
target: "avalanchego" | ||
coreth: "v0.12.5" | ||
libevm: "v1.0.0" | ||
|
||
- name: Use binary via output parameter | ||
run: ${{ steps.build.outputs.binary-path }} --network-id=local | ||
|
||
- name: Or download as artifact | ||
uses: actions/download-artifact@v4 | ||
with: | ||
name: avalanchego-avalanchego_main-coreth_v0.12.5-libevm_v1.0.0 | ||
|
||
- name: Use downloaded artifact | ||
run: ./avalanchego --network-id=local | ||
``` | ||
### SETUP Mode - Custom Workflow | ||
```yaml | ||
- name: Setup AvalancheGo with custom dependencies | ||
uses: ./.github/actions/avalanchego-build-action | ||
with: | ||
checkout-path: "build/avalanchego" | ||
coreth: "my-feature-branch" | ||
libevm: "experimental-branch" | ||
|
||
- name: Run custom build commands | ||
run: | | ||
cd build/avalanchego | ||
./scripts/run_task.sh reexecute-cchain-range | ||
./scripts/run_task.sh my-custom-task | ||
``` | ||
## Artifact Naming | ||
Artifacts are named with the complete dependency matrix for full traceability: | ||
**Format:** `{target}-avalanchego_{version}-coreth_{version}-libevm_{version}[-firewood_{version}]` | ||
|
||
**Examples:** | ||
- `avalanchego-avalanchego_main-coreth_main-libevm_main` (default versions) | ||
- `avalanchego-avalanchego_v1.11.0-coreth_v0.12.5-libevm_v1.0.0-firewood_ffi%2Fv0.0.13` (with firewood) | ||
- `reexecution-avalanchego_my-branch-coreth_main-libevm_experimental-firewood_abc123` (mixed versions) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,120 @@ | ||
name: 'AvalancheGo Build Action' | ||
description: 'Build AvalancheGo with custom dependencies. Dual mode: BUILD (with target) creates binary, SETUP (no target) prepares environment.' | ||
|
||
inputs: | ||
target: | ||
description: 'Binary to build (avalanchego, reexecution). If provided: BUILD mode. If empty: SETUP mode.' | ||
required: false | ||
default: '' | ||
args: | ||
description: 'Arguments for target executable (BUILD mode only)' | ||
required: false | ||
default: '' | ||
checkout-path: | ||
description: 'Directory path where AvalancheGo will be checked out' | ||
required: false | ||
default: 'avalanchego' | ||
avalanchego: | ||
description: 'AvalancheGo version (commit SHA, branch name, or tag)' | ||
required: false | ||
default: ${{ github.sha }} | ||
firewood: | ||
description: 'Firewood version (commit SHA, branch, tag, or ffi/vX.Y.Z for pre-built)' | ||
required: false | ||
default: '' | ||
coreth: | ||
description: 'Coreth version (commit SHA, branch name, or tag)' | ||
required: false | ||
default: '' | ||
libevm: | ||
description: 'LibEVM version (commit SHA, branch name, or tag)' | ||
required: false | ||
default: '' | ||
|
||
outputs: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we also output the artifact name (rather than solely relying on the convention described in README.md) |
||
binary-path: | ||
description: 'Absolute path to built binary (BUILD mode only)' | ||
value: ${{ steps.build.outputs.binary-path }} | ||
|
||
runs: | ||
using: 'composite' | ||
steps: | ||
- name: Checkout AvalancheGo | ||
uses: actions/checkout@v4 | ||
with: | ||
repository: 'ava-labs/avalanchego' | ||
ref: ${{ inputs.avalanchego }} | ||
path: ${{ inputs.checkout-path }} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why does AvalancheGo use checkout path and everything else just uses the name of the repo it's checking out? Would it make sense to take in one parameter to use as a working directory and then check out each required remote as a subdirectory? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The avalanchego checkout path is the main workspace where we build the final binary. It needs to be configurable because consumers might want to integrate it into existing directory structures. Firewood has some storage requirements its database operations benefit from NVMe for performance, but compilation can happen anywhere. The Firewood action handles its own optimal storage detection internally. The current approach lets each component use optimal storage:
|
||
- name: Setup Go for project | ||
uses: ./.github/actions/setup-go-for-project # Note: If Nix-specific functionality is needed, consumer should install Nix as a prerequisite to this action | ||
- name: Setup Firewood FFI | ||
if: inputs.firewood != '' | ||
id: firewood | ||
uses: ava-labs/firewood/.github/actions/build-action@composable-ci-action | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Where is this action defined? I don't see it on Firewood main There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It's defined here There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is a bit hard to verify as a reviewer since it's not on main and doesn't have a reference to the branch where it's available. How does this work for the workflow runs linked in the PR? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added link to Firewood action in PR description, thanks for pointing it out. This run is using Firewood action along with AvalancheGo to run reexecution test on self-hosted runners. You can find the commit here. (I'll attach commits in the PR description as well). The workflow in the commit proves that:
|
||
with: | ||
version: ${{ inputs.firewood }} | ||
- name: Checkout Coreth | ||
if: inputs.coreth != '' && inputs.coreth != 'master' | ||
uses: actions/checkout@v4 | ||
with: | ||
repository: 'ava-labs/coreth' | ||
ref: ${{ inputs.coreth }} | ||
path: 'coreth' | ||
- name: Checkout LibEVM | ||
if: inputs.libevm != '' && inputs.libevm != 'main' | ||
uses: actions/checkout@v4 | ||
with: | ||
repository: 'ava-labs/libevm' | ||
ref: ${{ inputs.libevm }} | ||
path: 'libevm' | ||
- name: Replace dependencies with local checkouts | ||
if: ${{ inputs.firewood != '' || inputs.coreth != '' || inputs.libevm != '' }} | ||
shell: bash | ||
working-directory: ./${{ inputs.checkout-path }} | ||
run: | | ||
# Replace Firewood FFI if provided | ||
if [ "${{ inputs.firewood }}" != "" ]; then | ||
echo "Replacing Firewood FFI with: ${{ steps.firewood.outputs.ffi-path }}" | ||
go mod edit -replace github.com/ava-labs/firewood-go-ethhash/ffi=${{ steps.firewood.outputs.ffi-path }} | ||
fi | ||
# Replace Coreth if provided and not default | ||
if [ "${{ inputs.coreth }}" != "" ] && [ "${{ inputs.coreth }}" != "master" ]; then | ||
echo "Replacing Coreth with local checkout: ../coreth" | ||
go mod edit -replace github.com/ava-labs/coreth=../coreth | ||
fi | ||
# Replace LibEVM if provided and not default | ||
if [ "${{ inputs.libevm }}" != "" ] && [ "${{ inputs.libevm }}" != "main" ]; then | ||
echo "Replacing LibEVM with local checkout: ../libevm" | ||
go mod edit -replace github.com/ava-labs/libevm=../libevm | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Given we are checking out versions that are available in GitHub, we can just use go get with the updated version rather than checking them each out locally. If we wanted to checkout and compile a specific version of Firewood, we'd need the source code, but currently this is just using the published golang ffi bindings of Firewood as well. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That's good input, I'll run The Firewood workflow handles building form source and using pre-built FFI. |
||
fi | ||
go mod tidy | ||
go mod download | ||
Comment on lines
+93
to
+94
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Won't this leave a diff in the resulting checked out code that may cause future actions to find an unexpected diff and fail? Should that be documented or cleaned up? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes it will, it should probably be cleaned up. Will handle that case. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we could probably do less here and only support setup rather than adding a script that builds different binaries once setup has completed. |
||
- name: Build and run target (BUILD mode) | ||
if: inputs.target != '' | ||
id: build | ||
shell: bash | ||
working-directory: ./${{ inputs.checkout-path }} | ||
run: | | ||
if [ -n "${{ inputs.args }}" ]; then | ||
OUTPUT=$(./scripts/build_target.sh "${{ inputs.target }}" ${{ inputs.args }}) | ||
else | ||
OUTPUT=$(./scripts/build_target.sh "${{ inputs.target }}") | ||
fi | ||
# Extract binary path from script output | ||
BINARY_PATH=$(echo "$OUTPUT" | grep "BINARY_PATH=" | cut -d'=' -f2) | ||
# Convert to absolute path to avoid relative pathing issues | ||
ABSOLUTE_BINARY_PATH=$(realpath "$BINARY_PATH") | ||
# Set output for consumer use | ||
echo "binary-path=$ABSOLUTE_BINARY_PATH" >> $GITHUB_OUTPUT | ||
- name: Upload binary artifact (BUILD mode) | ||
if: inputs.target != '' | ||
uses: actions/upload-artifact@v4 | ||
with: | ||
name: ${{ inputs.target }}-avalanchego_${{ inputs.avalanchego }}-coreth_${{ inputs.coreth }}-libevm_${{ inputs.libevm }}${{ inputs.firewood != '' && format('-firewood_{0}', inputs.firewood) || '' }} | ||
path: ${{ steps.build.outputs.binary-path }} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
#!/usr/bin/env bash | ||
# Build AvalancheGo target binary and optionally execute with arguments | ||
# Usage: ./scripts/build_target.sh <target> [args...] | ||
|
||
set -euo pipefail | ||
|
||
if [[ $# -eq 0 ]]; then | ||
echo "Usage: $0 <target> [args...]" | ||
echo "Valid targets: avalanchego, reexecution" | ||
exit 1 | ||
fi | ||
|
||
TARGET="$1" | ||
shift # Remove target from arguments, remaining args are for execution | ||
|
||
case "$TARGET" in | ||
"avalanchego") | ||
Elvis339 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
if [[ ! -f "./scripts/run_task.sh" || ! -x "./scripts/run_task.sh" ]]; then | ||
echo "Error: ./scripts/run_task.sh not found or not executable" | ||
exit 1 | ||
fi | ||
./scripts/run_task.sh build | ||
EXECUTABLE="./build/avalanchego" | ||
|
||
if [[ ! -f "$EXECUTABLE" ]]; then | ||
echo "Error: Binary $EXECUTABLE was not created" | ||
exit 1 | ||
fi | ||
|
||
echo "BINARY_PATH=$PWD/$EXECUTABLE" | ||
Elvis339 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
if [[ $# -gt 0 ]]; then | ||
"$EXECUTABLE" "$@" | ||
fi | ||
;; | ||
"reexecution") | ||
# Compile reexecution benchmark test into binary | ||
go test -c github.com/ava-labs/avalanchego/tests/reexecute/c -o reexecute-benchmark | ||
EXECUTABLE="./reexecute-benchmark" | ||
|
||
if [[ ! -f "$EXECUTABLE" ]]; then | ||
echo "Error: Binary $EXECUTABLE was not created" | ||
exit 1 | ||
fi | ||
|
||
echo "BINARY_PATH=$PWD/$EXECUTABLE" | ||
;; | ||
*) | ||
echo "Error: Invalid target '$TARGET'. Valid targets: avalanchego, reexecution" | ||
exit 1 | ||
;; | ||
esac | ||
Elvis339 marked this conversation as resolved.
Show resolved
Hide resolved
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This would not work because the current task definition uses
go run
, so it would not use the the compiled test binary.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It works here.