feat(ci): add changeset validation to solidity analysis workflows

[yorhodes]

Summary

Add changeset validation to the Solidity bytecode, interface, and storage analysis CI workflows. This ensures developers include appropriate changesets when modifying contracts.

Changeset Requirements

Analysis	Change Type	Required Changeset
Bytecode	Any change	patch or higher
Interface	Addition (new functions, events, errors)	minor or higher
Interface	Removal or modification	major
Storage	Addition (new storage slots)	minor or higher
Storage	Removal	major

Changes

• Add .github/scripts/check-solidity-changeset.sh - validates @hyperlane-xyz/core changesets at required bump level
• Add .github/scripts/check-diff-changeset.sh - classifies diffs (additions/removals) and calls changeset validator
• Update bytecode-analysis.yml to use shared script
• Update interface-analysis.yml to check changeset based on change type
• Update storage-analysis.yml to use shared script and add skip condition for changeset-release PRs
• Update solidity/interface.sh to return exit code 2 for additions-only (distinguishing from removals)
• Update solidity/README.md with changeset requirements table

Manual Testing

check-solidity-changeset.sh tests

# No changeset for @hyperlane-xyz/core
$ .github/scripts/check-solidity-changeset.sh patch
No changeset found for @hyperlane-xyz/core (required: patch or higher)

To fix this, run 'pnpm changeset' and select '@hyperlane-xyz/core' with a 'patch' (or higher) bump.
Exit code: 1

# With patch changeset
$ .github/scripts/check-solidity-changeset.sh patch
Found @hyperlane-xyz/core changeset with 'patch' bump (required: patch or higher)
Exit code: 0

$ .github/scripts/check-solidity-changeset.sh minor
Found @hyperlane-xyz/core changeset with 'patch' bump, but 'minor' or higher is required

To fix this, run 'pnpm changeset' and select '@hyperlane-xyz/core' with a 'minor' (or higher) bump.
Exit code: 1

# With minor changeset
$ .github/scripts/check-solidity-changeset.sh minor
Found @hyperlane-xyz/core changeset with 'minor' bump (required: minor or higher)
Exit code: 0

$ .github/scripts/check-solidity-changeset.sh major
Found @hyperlane-xyz/core changeset with 'minor' bump, but 'major' or higher is required

To fix this, run 'pnpm changeset' and select '@hyperlane-xyz/core' with a 'major' (or higher) bump.
Exit code: 1

check-diff-changeset.sh tests

# No changes
$ .github/scripts/check-diff-changeset.sh bytecode /tmp/base-test /tmp/head-test
No bytecode changes detected.
Exit code: 0

# Bytecode additions (requires patch)
$ .github/scripts/check-diff-changeset.sh bytecode /tmp/base-test /tmp/head-test
Detected bytecode changes:
...
Bytecode changes detected.

No changeset found for @hyperlane-xyz/core (required: patch or higher)
...
ERROR: Bytecode changes require a changeset for @hyperlane-xyz/core with at least a 'patch' bump.
Exit code: 1

# Storage additions (requires minor)
$ .github/scripts/check-diff-changeset.sh storage /tmp/base-test /tmp/head-test
Detected storage changes:
...
Storage layout additions detected.

No changeset found for @hyperlane-xyz/core (required: minor or higher)
...
ERROR: Storage layout additions require a changeset for @hyperlane-xyz/core with at least a 'minor' bump.
Exit code: 1

# Storage removals (requires major)
$ .github/scripts/check-diff-changeset.sh storage /tmp/base-test /tmp/head-test
Detected storage changes:
...
Storage layout removals (breaking change) detected.

No changeset found for @hyperlane-xyz/core (required: major or higher)
...
ERROR: Storage layout removals (breaking change) require a changeset for @hyperlane-xyz/core with at least a 'major' bump.
Exit code: 1

Related

This will fix the bytecode check failure on #7796 once a changeset is added.

---

Note

Introduces centralized changeset validation for Solidity contract modifications and wires it into CI.

• Adds ./.github/scripts/check-solidity-changeset.sh to verify @hyperlane-xyz/core bump level and ./.github/scripts/check-diff-changeset.sh to classify diffs (additions/removals) for bytecode and storage
• Updates bytecode-analysis.yml, interface-analysis.yml, and storage-analysis.yml to use shared validation, require patch|minor|major bumps as appropriate, and skip on changeset-release/main
• Enhances solidity/interface.sh to distinguish additions (exit code 2) vs removals (exit code 1) and improve output messaging
• Extends tests in solidity/test/interface.test.ts to assert new exit codes and messages; documents requirements in solidity/README.md

^{Written by Cursor Bugbot for commit 1c141df. This will update automatically on new commits. Configure here.}

Summary by CodeRabbit

• Chores

  • Enhanced CI/CD pipelines to enforce changeset validation for Solidity contract modifications, categorizing changes by type (bytecode, interface, storage) and requiring appropriate version bump levels.

• Documentation

  • Added contributing guidelines detailing CI-enforced changeset requirements with a reference table mapping change types to required version bumps and instructions for generating changesets.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[changeset-bot]

⚠️ No Changeset found

Latest commit: 1c141df

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[paulbalaji]

use spaces not tabs

[claude]

Claude encountered an error after 1m 3s —— View job

---

Reviewing PR #7801

Analyzing changeset validation scripts for security and code quality...

• Gather context and understand the PR changes
• Address review feedback (use spaces not tabs)
• Security review of shell scripts
• Review workflow file changes
• Check for best practices and coding standards
• Provide final summary

[coderabbitai]

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

📝 Walkthrough

Walkthrough

This PR introduces centralized shell scripts to validate that Solidity contract changes include appropriate changesets (patch, minor, or major) based on change type. The scripts are integrated into CI workflows for bytecode, interface, and storage analysis, replacing inline validation logic with reusable, scriptable checks. Documentation is added to guide contributors on changeset requirements.

Changes

Cohort / File(s)	Summary
New Changeset Validation Scripts .github/scripts/check-diff-changeset.sh, .github/scripts/check-solidity-changeset.sh	Added two new shell scripts: check-diff-changeset.sh (compares directory trees and determines required changeset level based on analysis type—bytecode always requires patch; storage treats removals as major and additions as minor) and check-solidity-changeset.sh (validates that a changeset at the required level exists for @hyperlane-xyz/core). Includes error handling and exit code preservation.
Workflow Updates Using New Scripts .github/workflows/bytecode-analysis.yml, .github/workflows/storage-analysis.yml	Replaced inline diff-and-exit logic with centralized script calls to check-diff-changeset.sh. Added early-exit skip condition in storage workflow for changeset-version PRs. Reduces duplication and standardizes changeset enforcement.
Interface Analysis Workflow .github/workflows/interface-analysis.yml	Replaced simple "fail on removal" check with expanded conditional logic: captures exit codes from interface comparison (0 = no changes, 1 = removals requiring major bump, 2 = additions requiring minor bump), integrates check-solidity-changeset.sh calls, and provides explicit messaging for each path.
Interface Detection Script solidity/interface.sh	Modified reporting and exit codes: aggregates new-contract additions, defers reporting until decision point, exits with code 2 when additions exist without removals (instead of no exit), exits with code 1 on removals (breaking change). Renamed output messages ("ABI entries added" vs. prior variants).
Documentation solidity/README.md	Added Contributing section detailing CI-enforced changeset requirements with a table mapping change types (Bytecode, Interface, Storage) to required changeset levels and guidance on running pnpm changeset.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• fix(ci): skip bytecode check on release PRs and add foundry to CLI install #7588: Modifies CI workflow guards (bytecode-analysis.yml) with the same changeset-release/main skip condition.
• fix: storage layout compatibility #7078: Updates storage-analysis workflow comparison and failure handling logic (removals vs. additions).
• feat: add bytecode analysis GitHub action #7500: Introduces bytecode analysis workflow and bytecode generation/diff logic that this PR now centralizes into new scripts.

Suggested reviewers

• ltyu
• larryob
• paulbalaji

Poem

🧅 Like an onion with layers, these scripts unfold—
Each workflow checks its changeset, all neat and bold,
No more scattered logic, just scripts clean and tight,
Validation flows smooth from left to right,
Now contracts bump proper, and CI sleeps right! 🚀

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: adding changeset validation to Solidity analysis workflows, which is the core focus of this PR.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Description check	✅ Passed	The PR description includes all required sections: comprehensive summary of changes, detailed changeset requirements table, specific file modifications listed, and thorough manual testing examples demonstrating expected behavior.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch feat/solidity-changeset-validation

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[claude]

Claude finished @yorhodes's task in 1m 25s —— View job

---

Reviewing PR #7801

• Gather context and understand the PR changes
• Address review feedback (use spaces not tabs) — Fixed in commit 550e8dfee
• Security review of shell scripts
• Review workflow file changes
• Check for best practices and coding standards
• Provide final summary

---

Summary

This PR adds changeset validation to Solidity analysis CI workflows, ensuring developers include appropriate version bumps when modifying contracts. The implementation is well-structured and ready to merge.

Security Review

Shell Script Security ✅

The scripts follow security best practices:

Practice	Status
set -euo pipefail	✅ Both scripts use strict mode
Input validation	✅ Arguments validated before use
No command injection	✅ Variables properly quoted where needed
No eval or dynamic execution	✅ No dangerous constructs
Path handling	✅ Uses ${BASH_SOURCE[0]} for reliable script location

Minor observations (non-blocking):

1. .github/scripts/check-solidity-changeset.sh:38 — The sed and grep pipeline for YAML parsing is fragile for edge cases (e.g., if the package name has special regex characters). However, since @hyperlane-xyz/core is hardcoded and the input is controlled changeset files, this is acceptable.

2. solidity/interface.sh:143 — Using find output in an array with IFS manipulation works but could be more robust with mapfile -t. However, this follows existing patterns in the codebase.

Workflow Review

Changes are correct and well-designed:

• ✅ All three workflows (bytecode-analysis.yml, interface-analysis.yml, storage-analysis.yml) properly skip on changeset-release/main branch
• ✅ Exit code semantics are clear: 0 = no changes, 1 = breaking changes, 2 = additions only
• ✅ Appropriate bump levels enforced per change type
• ✅ Consistent use of shared scripts reduces duplication

Changeset requirements matrix is sensible:

Analysis	Change Type	Required Bump	Rationale
Bytecode	Any	patch	Any bytecode change should be versioned
Interface	Addition	minor	New ABI entries are backward compatible
Interface	Removal	major	Breaking for consumers
Storage	Addition	minor	New slots are safe
Storage	Removal	major	Breaking for upgrades

Code Quality

• ✅ Good documentation in PR description and solidity/README.md
• ✅ Clear variable names and comments
• ✅ Comprehensive manual testing documented
• ✅ Tabs-to-spaces issue resolved in latest commit

Note on Changesets for this PR

This PR itself modifies CI scripts (not Solidity contracts), so it does not require a @hyperlane-xyz/core changeset. The changeset-bot warning can be safely ignored.

---

Verdict: Approve 👍

The implementation is clean, follows shell best practices, and the validation logic correctly maps semantic versioning to contract change types.

[claude]

Claude finished @yorhodes's task in 1m 42s —— View job

---

Trail of Bits Security Review — PR #7801

• Gather context and understand PR changes
• Analyze shell scripts for command injection vulnerabilities
• Review diff classification logic for security bypass potential
• Verify codex review feedback fixes (new file/removal detection)
• Check workflow files for security concerns
• Provide security summary with severity ratings

---

Security Analysis Summary

This PR adds changeset validation to Solidity analysis CI workflows. Overall, the implementation is secure and follows shell scripting best practices.

Shell Script Security ✅

Check	Status	Details
Strict mode	✅	Both scripts use set -euo pipefail
Input validation	✅	Arguments validated before use (lines 15-18 in both scripts)
Command injection	✅	No user-controlled input reaches command execution
Path traversal	✅	Paths come from workflow, not user input
Dangerous constructs	✅	No eval, source, or dynamic execution

Note on YAML parsing (.github/scripts/check-solidity-changeset.sh:38): The sed/grep pipeline is intentionally simple since it parses controlled changeset files with a hardcoded package name. This is acceptable for CI but wouldn't be suitable for arbitrary YAML.

Codex Feedback — Now Addressed ✅

Both concerns raised by codex have been fixed in recent commits:

1. New/removed file detection — Commit 685793392 added handling for Only in <dir> lines in check-diff-changeset.sh (lines 46-53). This correctly classifies file additions and removals, not just line-level changes.

2. New contracts as interface additions — Commit 84f8fbc3f fixed solidity/interface.sh to add new contracts to ADDED_ITEMS (line 105), ensuring exit code 2 is returned and a minor changeset is required.

Workflow Security ✅

Aspect	Status	Details
Secret exposure	✅	No secrets used in changed workflows
Branch protection bypass	✅	Workflows skip changeset-release/main appropriately
Checkout safety	✅	Uses pull_request event (not pull_request_target)
Script injection	✅	No interpolation of PR titles/bodies in scripts

Bypass Analysis

Q: Can a malicious PR bypass changeset requirements?

No practical bypass vectors exist:

• CI scripts run in the workflow context, not with PR author privileges
• The changeset files are checked from the PR branch itself
• A PR author would need to include a valid changeset to pass CI
• The changeset bot independently verifies changesets

Minor Observations (Informational)

1. solidity/interface.sh:143 — Uses $(find ...) in array assignment with IFS manipulation. This works but mapfile -t would be more idiomatic. Not a security issue, follows existing codebase patterns.

2. Regex escaping — The grep patterns like ^Only in ${BASE_DIR} assume directory paths don't contain regex special characters. This is safe in practice since paths come from workflow variables, not user input.

---

Hyperlane-Specific Security Review

This PR modifies CI/infrastructure only, not smart contracts. However, the changeset policy correctly maps semantic versioning to contract change types:

Analysis	Change Type	Bump	Security Rationale
Bytecode	Any	patch	Any bytecode change should be tracked
Interface	Addition	minor	New ABI entries are backward compatible
Interface	Removal	major	Breaking — affects downstream integrators
Storage	Addition	minor	New storage slots are safe for upgrades
Storage	Removal	major	Breaking — can corrupt upgraded contracts

This policy aligns with Hyperlane's upgrade safety requirements.

---

Verdict: APPROVE ✅

The implementation is secure, addresses prior review feedback, and correctly enforces semver for contract changes.

Severity	Finding Count
Critical	0
High	0
Medium	0
Low	0
Informational	2

No blocking issues. Ready to merge.

[paulbalaji]

ci will go green once you pull the latest commit on main which fixes it

[cursor]

This is the final PR Bugbot will review for you during this billing cycle

Your free Bugbot reviews will reset on February 13

Details

You are on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle.

To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 77.02%. Comparing base (d90525a) to head (1c141df).
⚠️ Report is 8 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #7801   +/-   ##
=======================================
  Coverage   77.02%   77.02%           
=======================================
  Files         117      117           
  Lines        2651     2651           
  Branches      244      244           
=======================================
  Hits         2042     2042           
  Misses        593      593           
  Partials       16       16           

Components	Coverage Δ
core	87.80% <ø> (ø)
hooks	71.86% <ø> (ø)
isms	81.10% <ø> (ø)
token	86.67% <ø> (ø)
middlewares	84.98% <ø> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[claude]

Claude finished @yorhodes's task in 1m 2s —— View job

---

Reviewing PR #7801

• Gather context and understand the PR changes
• Review Cursor Bugbot feedback on unexpected exit codes
• Address the directory existence check issue
• Implement fix for unexpected exit code handling in interface workflow
• Final security summary

---

Trail of Bits Security Review Summary

This PR adds changeset validation to Solidity analysis CI workflows. The implementation is secure and all previous review feedback has been addressed.

Cursor Bugbot Feedback — Addressed ✅

Both issues raised by Cursor Bugbot have been fixed in commit fe1cf5efa:

Issue	Severity	Status	Fix
Silent success when diff directories don't exist	Low	✅ Fixed	Lines 21-28 in check-diff-changeset.sh now validate directory existence before running diff
Script errors treated as interface removals	Medium	✅ Fixed	Lines 100-104 in interface-analysis.yml now properly fail on unexpected exit codes

Shell Script Security Analysis ✅

Check	Status	Details
Strict mode	✅	Both scripts use set -euo pipefail
Input validation	✅	Arguments validated before use
Directory validation	✅	check-diff-changeset.sh:21-28 validates paths exist
Command injection	✅	No user-controlled input reaches command execution
Exit code handling	✅	interface-analysis.yml:100-104 properly handles unexpected codes

Workflow Security ✅

Aspect	Status	Details
Secret exposure	✅	No secrets used in changed workflows
Checkout safety	✅	Uses pull_request event (not pull_request_target)
Script injection	✅	No interpolation of PR titles/bodies in scripts
Branch skip	✅	All workflows skip changeset-release/main appropriately

Changeset Requirements Policy

The semver mapping is correct for smart contract changes:

Analysis	Change Type	Required Bump	Rationale
Bytecode	Any	patch	Any bytecode change should be versioned
Interface	Addition	minor	New ABI entries are backward compatible
Interface	Removal	major	Breaking for downstream integrators
Storage	Addition	minor	New storage slots are safe
Storage	Removal	major	Breaking for contract upgrades

Minor Observations (Informational)

1. check-solidity-changeset.sh:38 — The sed/grep YAML parsing is simple but adequate since it only parses controlled changeset files with a hardcoded package name.

2. paulbalaji's suggestion (.github/workflows/interface-analysis.yml:67) — Converting the inline shell logic to a separate script would improve consistency with bytecode/storage workflows. This is optional but recommended for maintainability.

---

Verdict: APPROVE ✅

Severity	Finding Count
Critical	0
High	0
Medium	0 (fixed)
Low	0 (fixed)
Informational	2

All blocking issues have been addressed. CI is running and no further changes are required.

[claude]

Claude finished @yorhodes's task in 55s —— View job

---

Reviewing PR #7801

• Gather context and understand the PR changes
• Security review of shell scripts (Trail of Bits analysis)
• Review workflow file changes
• Verify all review feedback has been addressed
• Provide final summary

---

Trail of Bits Security Review Summary

This PR adds changeset validation to Solidity analysis CI workflows. The implementation is secure and follows shell scripting best practices.

Shell Script Security Analysis ✅

Check	Status	Details
Strict mode	✅	Both scripts use set -euo pipefail (lines 9, 7)
Input validation	✅	Arguments validated before use
Directory validation	✅	check-diff-changeset.sh:21-28 validates paths exist
Command injection	✅	No user-controlled input reaches command execution
Exit code handling	✅	interface-analysis.yml:100-104 properly handles unexpected codes

All Review Feedback Addressed ✅

Issue	Source	Status	Fix
Use spaces not tabs	paulbalaji	✅ Fixed	Commit 550e8dfee
New/removed file detection	codex	✅ Fixed	Commit 685793392 — Lines 56-63 in check-diff-changeset.sh
New contracts as interface additions	codex	✅ Fixed	Commit 84f8fbc3f — Line 105 in interface.sh
Silent success when directories don't exist	cursor	✅ Fixed	Commit fe1cf5efa — Lines 21-28 in check-diff-changeset.sh
Script errors treated as interface removals	cursor	✅ Fixed	Commit 88088f891 — Lines 100-104 in interface-analysis.yml

Workflow Security ✅

Aspect	Status	Details
Secret exposure	✅	No secrets used in changed workflows
Checkout safety	✅	Uses pull_request event (not pull_request_target)
Script injection	✅	No interpolation of PR titles/bodies in scripts
Branch skip	✅	All workflows skip changeset-release/main appropriately

Changeset Requirements Policy

The semver mapping is correct for smart contract changes:

Analysis	Change Type	Required Bump	Rationale
Bytecode	Any	patch	Any bytecode change should be versioned
Interface	Addition	minor	New ABI entries are backward compatible
Interface	Removal	major	Breaking for downstream integrators
Storage	Addition	minor	New storage slots are safe
Storage	Removal	major	Breaking for contract upgrades

Minor Observations (Informational)

1. check-solidity-changeset.sh:38 — The sed/grep YAML parsing is simple but adequate since it only parses controlled changeset files with a hardcoded package name.

2. solidity/interface.sh:143 — Uses $(find ...) in array assignment with IFS manipulation. This works correctly and follows existing codebase patterns.

Note on This PR's Changeset

This PR modifies CI scripts only (not Solidity contracts), so it does not require a @hyperlane-xyz/core changeset. The changeset-bot warning can be safely ignored.

---

Verdict: APPROVE ✅

Severity	Finding Count
Critical	0
High	0
Medium	0
Low	0
Informational	2

All blocking issues have been addressed. The implementation is clean, follows shell best practices, and the validation logic correctly maps semantic versioning to contract change types. Ready to merge.

[claude]

Claude encountered an error after 1m 38s —— View job

---

Reviewing PR #7801

Analyzing changeset validation scripts...

• Gather context and understand the PR changes
• Review latest commit changes
• Security review of shell scripts
• Fix: tabs reintroduced in check-solidity-changeset.sh (latest commit)
• Verify all review feedback has been addressed
• Provide final summary