Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions .github/actions/e2e-cold-start/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: E2E cold-start
description: >
Bring up a kurtosis-pos devnet from source. Installs the kurtosis CLI,
starts the engine, clones kurtosis-pos at the requested ref, scales EL
resource limits down to fit a 2-vCPU / 7-GB ubuntu-latest runner, and
runs the enclave from the consumer's args file.

This is the escape hatch for iterating on a kurtosis-pos branch that
hasn't been snapshotted yet. ~10 minute wall-time vs the ~30 second
snapshot path; consumers default to the snapshot and reach for cold
start only when they need an unpublished kurtosis-pos commit.

inputs:
kurtosis_pos_ref:
description: >
Git ref of 0xPolygon/kurtosis-pos to clone (branch, tag, or SHA).
required: false
default: main
args_file:
description: >
Path to the kurtosis args file in the consumer's checkout (relative
to GITHUB_WORKSPACE). Each consumer ships its own — the args dictate
L1 backend, contract deployments, and chain IDs that the enclave
brings up.
required: false
default: kurtosis-params.yml
work_dir:
description: >
Scratch dir for the kurtosis-pos clone. Default `/tmp/kurtosis-pos`.
required: false
default: /tmp/kurtosis-pos

runs:
using: composite
steps:
- name: Cold-start kurtosis-pos enclave
shell: bash
env:
KURTOSIS_POS_REF: ${{ inputs.kurtosis_pos_ref }}
ARGS_FILE: ${{ inputs.args_file }}
WORK_DIR: ${{ inputs.work_dir }}
run: bash "${GITHUB_ACTION_PATH}/cold-start.sh"
117 changes: 117 additions & 0 deletions .github/actions/e2e-cold-start/cold-start.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
#!/usr/bin/env bash
#
# Cold-start a kurtosis-pos devnet from source.
#
# Used by consumers who need to test against an unpublished kurtosis-pos
# branch (the snapshot publisher only refreshes weekly + only off `main`).
# Total wall-time: ~10 minutes vs ~30s for the snapshot path. The snapshot
# is the default; this is the escape hatch.
#
# # Inputs (env vars, set by the composite action with documented defaults)
#
# KURTOSIS_POS_REF git ref of 0xPolygon/kurtosis-pos to clone.
# ARGS_FILE kurtosis args file in the consumer's checkout
# (resolved against GITHUB_WORKSPACE).
# WORK_DIR scratch dir for the clone.
#
# # After this script exits 0
#
# `kurtosis enclave inspect pos` reports RUNNING. Downstream steps reach
# the chain via the kurtosis-emitted env vars (each consumer's e2e suite
# already has its own kurtosis adapter — this script doesn't export
# anything to GITHUB_ENV).

set -euo pipefail

KURTOSIS_POS_REF="${KURTOSIS_POS_REF:-main}"
ARGS_FILE="${ARGS_FILE:-kurtosis-params.yml}"
WORK_DIR="${WORK_DIR:-/tmp/kurtosis-pos}"

REPO_ROOT="${GITHUB_WORKSPACE:-$PWD}"
ARGS_FILE_ABS="${REPO_ROOT}/${ARGS_FILE}"
if [ ! -f "$ARGS_FILE_ABS" ]; then

Check failure on line 32 in .github/actions/e2e-cold-start/cold-start.sh

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.

See more on https://sonarcloud.io/project/issues?id=0xPolygon_pipelines&issues=AZ6P38nyZGkXtdJ08eqQ&open=AZ6P38nyZGkXtdJ08eqQ&pullRequest=43
echo "[e2e-cold-start] ERROR: args file not found at ${ARGS_FILE_ABS}" >&2
echo "[e2e-cold-start] Pass the path relative to your repo root via the action's args_file input." >&2
exit 1
fi

##############################################################################
# 1. Install kurtosis CLI
##############################################################################
if command -v kurtosis >/dev/null 2>&1; then
echo "[e2e-cold-start] kurtosis already installed: $(kurtosis version | head -n1)"
else
echo "[e2e-cold-start] installing kurtosis-cli via apt"
echo "deb [trusted=yes] https://apt.fury.io/kurtosis-tech/ /" \
| sudo tee /etc/apt/sources.list.d/kurtosis.list >/dev/null
sudo apt-get update -qq
sudo apt-get install -y --no-install-recommends kurtosis-cli
# Kurtosis prompts for analytics consent on first interactive use; disable
# explicitly so the CLI never blocks on stdin in a non-TTY shell.
kurtosis analytics disable
fi

##############################################################################
# 2. Start the kurtosis engine
##############################################################################
echo "[e2e-cold-start] starting kurtosis engine"
kurtosis engine start

##############################################################################
# 3. Clone kurtosis-pos and run the enclave
##############################################################################
if kurtosis enclave inspect pos 2>/dev/null | grep -q RUNNING; then
echo "[e2e-cold-start] enclave 'pos' already running"
exit 0
fi

if [ ! -d "$WORK_DIR" ]; then

Check failure on line 68 in .github/actions/e2e-cold-start/cold-start.sh

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.

See more on https://sonarcloud.io/project/issues?id=0xPolygon_pipelines&issues=AZ6P38nyZGkXtdJ08eqR&open=AZ6P38nyZGkXtdJ08eqR&pullRequest=43
echo "[e2e-cold-start] cloning kurtosis-pos@${KURTOSIS_POS_REF} into ${WORK_DIR}"
git clone --depth 1 --branch "$KURTOSIS_POS_REF" \
https://github.com/0xPolygon/kurtosis-pos.git "$WORK_DIR"
fi

# kurtosis-pos hardcodes `MAX_CPU = 4000, MAX_MEM = 16384` for EL services,
# which exceeds what private-repo GitHub-hosted runners provide (2 vCPU,
# 7 GB). Scale down in place so the enclave fits on ubuntu-latest. On a
# developer machine with more headroom this rewrite is a no-op from the
# suite's perspective — the services just use less of the available
# capacity. Remove once kurtosis-pos exposes these as inputs.
#
# This patch is brittle by nature: it depends on the file path and the
# variable names. `kurtosis_pos_ref` is consumer-overridable, so an
# unexpected ref could rename either and make the sed a silent no-op —
# the enclave would then request 16 GB and the runner OOM-kills it with a
# confusing failure. Verify the override actually landed and fail loud if
# not, so the cause is unambiguous.
shared_star="$WORK_DIR/src/el/shared.star"
if [ ! -f "$shared_star" ]; then

Check failure on line 88 in .github/actions/e2e-cold-start/cold-start.sh

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.

See more on https://sonarcloud.io/project/issues?id=0xPolygon_pipelines&issues=AZ6P38nyZGkXtdJ08eqS&open=AZ6P38nyZGkXtdJ08eqS&pullRequest=43
echo "[e2e-cold-start] ERROR: expected ${shared_star} not found — kurtosis-pos@${KURTOSIS_POS_REF} may have moved the EL resource config. Update cold-start.sh for this ref." >&2
exit 1
fi
sed -i.bak -E \
-e 's/^MAX_CPU = [0-9]+.*/MAX_CPU = 1800 # CI override (ubuntu-latest 2-vCPU private-repo runner)/' \
-e 's/^MAX_MEM = [0-9]+.*/MAX_MEM = 4096 # CI override (ubuntu-latest 7-GB private-repo runner)/' \
"$shared_star"
rm -f "${shared_star}.bak"
# Confirm both overrides applied — grep for the marker comment the sed adds.
# Two matches expected (MAX_CPU + MAX_MEM); anything less means a var was
# renamed and we'd silently run at full resource request.
override_count="$(grep -c 'CI override (ubuntu-latest' "$shared_star" || true)"
if [ "$override_count" -ne 2 ]; then

Check failure on line 101 in .github/actions/e2e-cold-start/cold-start.sh

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.

See more on https://sonarcloud.io/project/issues?id=0xPolygon_pipelines&issues=AZ6P38nyZGkXtdJ08eqT&open=AZ6P38nyZGkXtdJ08eqT&pullRequest=43
echo "[e2e-cold-start] ERROR: EL resource scale-down did not apply (${override_count}/2 markers found in ${shared_star}). kurtosis-pos@${KURTOSIS_POS_REF} likely renamed MAX_CPU/MAX_MEM — the runner would OOM. Update the sed patterns for this ref." >&2
exit 1
fi

echo "[e2e-cold-start] running enclave 'pos' (this takes ~5–10m)"
( cd "$WORK_DIR" && kurtosis run --enclave pos --args-file "$ARGS_FILE_ABS" . )

# Export the enclave name so the consumer's e2e suite can locate it via the
# kurtosis CLI in cold-start mode (symmetric with snapshot mode exporting
# E2E_SNAPSHOT_ADDRESSES_JSON). Lets a consumer write a single env-driven
# adapter instead of hardcoding the enclave name per mode.
if [ -n "${GITHUB_ENV:-}" ]; then

Check failure on line 113 in .github/actions/e2e-cold-start/cold-start.sh

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.

See more on https://sonarcloud.io/project/issues?id=0xPolygon_pipelines&issues=AZ6P38nyZGkXtdJ08eqU&open=AZ6P38nyZGkXtdJ08eqU&pullRequest=43
echo "E2E_KURTOSIS_ENCLAVE=pos" >> "$GITHUB_ENV"
fi

echo "[e2e-cold-start] ready"
40 changes: 40 additions & 0 deletions .github/actions/e2e-dump/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: E2E devnet dump
description: >
Capture devnet state into /tmp/devnet-dump for post-mortem after an e2e
failure. Snapshot mode dumps the docker-compose stack's ps + logs and the
compose file; cold-start mode dumps the kurtosis enclave. Best-effort —
individual failures are swallowed so a half-restored run still yields
whatever artefacts it can.

Bundled as a composite (not a reusable-workflow inline step or a
.github/scripts/ helper) so the shell logic is locally runnable AND
reachable when apps-e2e.yml runs in a consumer's checkout — a raw script
under .github/scripts/ is not on disk cross-repo; ${{ github.action_path }}
is.

inputs:
use_snapshot:
description: '"1" for snapshot mode (compose dump), "0" for cold-start (enclave dump).'
required: true
compose_file:
description: >
Snapshot mode: path to the restored docker-compose.yaml (the restore
composite's compose_file_path output). May be empty if the restore
failed before writing it — the dump guards for that.
required: false
default: ''
enclave:
description: 'Cold-start mode: kurtosis enclave name to dump.'
required: false
default: pos

runs:
using: composite
steps:
- name: Dump devnet state
shell: bash
env:
E2E_USE_SNAPSHOT: ${{ inputs.use_snapshot }}
COMPOSE_FILE: ${{ inputs.compose_file }}
E2E_KURTOSIS_ENCLAVE: ${{ inputs.enclave }}
run: bash "${GITHUB_ACTION_PATH}/dump-devnet.sh"
40 changes: 40 additions & 0 deletions .github/actions/e2e-dump/dump-devnet.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
#!/usr/bin/env bash
#
# Capture devnet state into /tmp/devnet-dump for post-mortem after an e2e
# failure. Invoked by the e2e-dump composite action. Two modes, picked by
# E2E_USE_SNAPSHOT:
#
# - Snapshot mode (=1): dump the docker-compose stack's `ps` + `logs` and
# copy the compose file itself. COMPOSE_FILE is the path the restore
# composite emitted; it may be unset/absent if the restore failed before
# writing it, so every step is guarded.
# - Cold-start mode (=0): dump the kurtosis enclave named by
# E2E_KURTOSIS_ENCLAVE (default `pos`).
#
# Best-effort throughout: a half-restored run should still yield whatever
# artefacts it can, so individual failures are swallowed with `|| true`
# rather than aborting the dump.
#
# Environment (set by action.yml):
# E2E_USE_SNAPSHOT "1" for snapshot mode, "0" for cold-start.
# COMPOSE_FILE snapshot mode: path to the restored docker-compose.
# E2E_KURTOSIS_ENCLAVE cold-start mode: enclave name (default `pos`).

set -euo pipefail

DUMP_DIR="/tmp/devnet-dump"
mkdir -p "$DUMP_DIR"

if [ "${E2E_USE_SNAPSHOT:-1}" = "1" ]; then

Check failure on line 28 in .github/actions/e2e-dump/dump-devnet.sh

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.

See more on https://sonarcloud.io/project/issues?id=0xPolygon_pipelines&issues=AZ6P38n6ZGkXtdJ08eqV&open=AZ6P38n6ZGkXtdJ08eqV&pullRequest=43
compose_file="${COMPOSE_FILE:-}"
if [ -n "$compose_file" ] && [ -f "$compose_file" ]; then

Check failure on line 30 in .github/actions/e2e-dump/dump-devnet.sh

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.

See more on https://sonarcloud.io/project/issues?id=0xPolygon_pipelines&issues=AZ6P38n6ZGkXtdJ08eqW&open=AZ6P38n6ZGkXtdJ08eqW&pullRequest=43

Check failure on line 30 in .github/actions/e2e-dump/dump-devnet.sh

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.

See more on https://sonarcloud.io/project/issues?id=0xPolygon_pipelines&issues=AZ6P38n6ZGkXtdJ08eqX&open=AZ6P38n6ZGkXtdJ08eqX&pullRequest=43
docker compose --file "$compose_file" ps > "${DUMP_DIR}/compose-ps.txt" 2>&1 || true
docker compose --file "$compose_file" logs --no-color > "${DUMP_DIR}/compose-logs.txt" 2>&1 || true
cp "$compose_file" "${DUMP_DIR}/docker-compose.yaml" || true
else
echo "[dump-devnet] no compose file at '${compose_file:-<unset>}' — restore likely failed before writing it" >&2
fi
else
enclave="${E2E_KURTOSIS_ENCLAVE:-pos}"
kurtosis enclave dump "$enclave" "${DUMP_DIR}/${enclave}" || true
fi
88 changes: 88 additions & 0 deletions .github/actions/e2e-snapshot-restore/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
name: E2E snapshot restore
description: >
Restore a kurtosis-pos devnet from a published snapshot image. Pulls the image
from GHCR, extracts volumes + docker-compose + addresses sidecar, patches the
pos-anvil host-port mapping, brings the devnet up, and replays the captured
anvil state. Exports E2E_SNAPSHOT_ADDRESSES_JSON to GITHUB_ENV so consumer
steps inherit the path to the addresses sidecar.

inputs:
image:
description: >
Snapshot image reference (full registry/repo:tag). Default points at the
lst-api-published snapshot. Pin to a SHA tag (e.g. ":22c449550213") to
lock to a specific kurtosis-pos commit.
required: false
default: ghcr.io/0xpolygon/lst-api-e2e-snapshot:latest
kurtosis_pos_ref:
description: >
Git ref of 0xPolygon/kurtosis-pos to clone for the upstream extract.sh /
restore.sh helpers. Should match the ref used to build the snapshot —
the extract/restore scripts are tightly paired with their snapshot
contents. Default 'main' tracks the snapshot publisher's default.
required: false
default: main
out_dir:
description: >
Directory the snapshot is extracted into, relative to the consumer's
checkout root. Default './tmp/e2e-snapshot' avoids clobbering anything
else under ./tmp.
required: false
default: ./tmp/e2e-snapshot
registry:
description: >
Container registry to authenticate against. Default 'ghcr.io'. Override
only when the snapshot image lives elsewhere.
required: false
default: ghcr.io
registry_username:
description: >
Username for the registry login. Defaults to the workflow actor; override
when calling with a service account or PAT.
required: false
default: ${{ github.actor }}
registry_password:
description: >
Token for the registry login. Defaults to GITHUB_TOKEN, which works for
same-org GHCR pulls when the calling workflow grants `packages: read`.
required: false
default: ${{ github.token }}

outputs:
addresses_json_path:
description: Absolute path to the extracted addresses sidecar JSON.
value: ${{ steps.restore.outputs.addresses_json_path }}
compose_file_path:
description: Absolute path to the bundled docker-compose.yaml.
value: ${{ steps.restore.outputs.compose_file_path }}

runs:
using: composite
steps:
- name: Log in to container registry
shell: bash
env:
REGISTRY: ${{ inputs.registry }}
REGISTRY_USERNAME: ${{ inputs.registry_username }}
REGISTRY_PASSWORD: ${{ inputs.registry_password }}
run: echo "$REGISTRY_PASSWORD" | docker login "$REGISTRY" -u "$REGISTRY_USERNAME" --password-stdin

- name: Restore devnet from snapshot
id: restore
shell: bash
env:
IMAGE: ${{ inputs.image }}
KURTOSIS_POS_REF: ${{ inputs.kurtosis_pos_ref }}
OUT_DIR: ${{ inputs.out_dir }}
run: bash "${GITHUB_ACTION_PATH}/restore.sh"

- name: Export snapshot addresses path to GITHUB_ENV
shell: bash
env:
ADDRS_PATH: ${{ steps.restore.outputs.addresses_json_path }}
run: |
# The consuming repo's e2e suite reads this env var to switch from the
# kurtosis CLI path to the sidecar JSON path. Writing to GITHUB_ENV
# propagates it to every subsequent step in the calling job without
# the consumer having to know the magic name.
echo "E2E_SNAPSHOT_ADDRESSES_JSON=${ADDRS_PATH}" >> "$GITHUB_ENV"
Comment on lines +83 to +88
Loading
Loading