When custom dependencies (firewood-ref or libevm-ref) are used,
go.mod/go.sum are modified. This causes github-action-benchmark
to fail when checking out gh-pages.

New input skips comparison step only when custom deps are present,
preserving summary display for normal PR runs.

Would it make sense to make this skip automatic with an appropriate warning if the tree is dirty?

The inputs already tell us when the tree will be dirty if firewood-ref or libevm-ref is provided, polyrepo/go get will modify go.mod. A runtime git status check would just confirm what we already know statically.

Putting the detection logic inside the action would also conflict with Aaron's constraint he was okay with the action accepting skip-benchmark-comparison as an input, but the decision of when to skip should be made by the caller, not the action itself.

@aaronbuchwald Why is it desirable to have to pass in a synthesized variable instead of just detecting this condition locally?

Adding context on the design. Polyrepo is a composable layer that sits before the benchmark action, setting up deps. The base action runs benchmarks on whatever code is present, then uploads artifacts (Firewood will track performance on their own gh-pages).

Aaron's constraint as I understood it is that the action should run reexecution for the environment that's present not be responsible for in-place modification or inferring dependency state.

This matters because if the action detects dirty tree and decides to skip, it's making decisions about dependency state without knowing why the tree is dirty. The caller (workflow) knows it ran polyrepo → knows the tree is dirty → passes skip-benchmark-comparison: true.

This keeps concerns separated dep setup in workflow, benchmarking in action.

I might be wrong in how I inferred Aaron's constraint happy for him to correct me. Open to doing whatever to progress with this PR.

The original constraint is that the custom action should not need to know about the special case of firewood or overwriting dependencies. This seems a better separation of responsibilities to me than to pass in firewood-ref, libevm-ref, etc. which imo is better handled as a separate step.

I am not sure why skip-benchmark-comparison needs to be added as opposed to simply setting push-github-action-benchmark to false.

The issue is that github-action-benchmark fails entirely when the tree is dirty - it tries to checkout gh-pages and errors on the dirty working tree. Setting push-github-action-benchmark: false still runs the step, which fails. We need to skip the step entirely.

Would it make sense to make this skip automatic with an appropriate warning if the tree is dirty?

Maybe use env vars for both args so that the caller has a consistent way of specifying both values?

That was my initial implementation :D

Re-added: a29c22c

Why are these values the trigger instead of the dirty tree state?

Two separate concerns:

• push-github-action-benchmark - Controls whether results get pushed to gh-pages for historical tracking. Only enabled for scheduled runs on master with default deps.
• skip-benchmark-comparison - Controls whether the comparison step runs at all. When custom deps are used, go.mod/go.sum are modified, causing the benchmark action to fail on gh-pages checkout (dirty tree error).

Why input values instead of dirty tree check?
We know statically that if firewood-ref or libevm-ref is provided, the tree will be dirty. Runtime check would confirm what inputs already tell us.

I'm okay with adding automatic dirty tree detection inside c-chain-reexecution-benchmark action if Aaron agrees. From my conversation with him, I inferred the action should stay focused on running benchmarks and not infer/handle dependency state the caller knows it modified deps, so the caller decides whether to skip comparison.