Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NET-11150] ci: fix conditional skip and add safeguard #637

Merged
merged 1 commit into from
Sep 25, 2024

Conversation

zalimeni
Copy link
Member

@zalimeni zalimeni commented Sep 24, 2024

Fix CI skip workflow by adopting tj-actions/changed-files (used by several other hashicorp repos) and putting in safeguards to avoid false negatives on changed files due to inappropriate workflow triggers.

Reproduced bug (skipped checks on Dockerfile change): https://github.com/hashicorp/consul-dataplane/actions/runs/10984338025/job/30494819791?pr=630#step:3:14

Debugging 🧵 for posterity, which has some notes w/ more background on our previous script and potential gotchas: https://go.hashi.co/net-11150-debugging-slack

PRs for other repos:

Testing

pull_request retrieves changes between base and feature branch: https://github.com/hashicorp/consul-dataplane/actions/runs/11017376755/job/30595124657?pr=637#step:4:64

Prints both skippable and non-skippable files: https://github.com/hashicorp/consul-dataplane/actions/runs/11017376755/job/30595124657?pr=637#step:5:12

Skip test: https://github.com/hashicorp/consul-dataplane/actions/runs/11018306352/job/30598259268?pr=638#step:6:6

Safeguard test: https://github.com/hashicorp/consul-dataplane/actions/runs/11016637261/job/30592629656?pr=636#step:2:14

Simulated push with non-skippable commit (diff is just latest commit): https://github.com/hashicorp/consul-dataplane/actions/runs/11020606567/job/30605759628#step:3:62

Simulated push with skippable commit (diff is just latest commit): https://github.com/hashicorp/consul-dataplane/actions/runs/11020613882/job/30605785364#step:4:12

@zalimeni zalimeni added pr/no-changelog This PR does not introduce a user-facing change that should be reflected in the changelog backport/1.1 Changes are backported to 1.1 backport/1.3 backport/1.4 Changes are backported to 1.4 backport/1.5 Changes are backported to 1.5 backport/1.6 Changes are backported to 1.6 labels Sep 24, 2024
@zalimeni zalimeni force-pushed the zalimeni/fix-ci-skip-action branch 4 times, most recently from f764902 to 835293b Compare September 24, 2024 18:03
@zalimeni zalimeni marked this pull request as ready for review September 24, 2024 19:30
@zalimeni zalimeni requested a review from a team as a code owner September 24, 2024 19:30
Comment on lines +14 to +19
on:
push:
branches:
- main
- release/**
pull_request:
Copy link
Member Author

@zalimeni zalimeni Sep 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This ensures we're using the triggers expected by the workflow. Mixing pull_request and push triggers for PRs means we can get falsely passing required checks due to diff'ing only the latest commit (the expected behavior on push), because there's no base_ref or "magic GH-only merge commit" to rely on since the push trigger is in the context of the feature branch in isolation rather than a PR merge branch (pull_request). More on this here: https://github.com/orgs/community/discussions/26243

The safeguard added to the reusable workflow enforces this by checking for either base_ref or protected branch status (what we'd expect on push).

- name: Check for skippable file changes
id: changed-files
uses: tj-actions/changed-files@v45
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This action is used in other hashicorp repos and is allow-listed:

❯ tsccr-helper gha allowlist -any-version | grep changed-files
tj-actions/changed-files@*
tj-actions/verify-changed-files@*

@zalimeni zalimeni changed the title ci: fix conditional skip and add safeguard [NET-11150] ci: fix conditional skip and add safeguard Sep 24, 2024
Copy link
Contributor

@NiniOak NiniOak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great and easy enough to understand and update for the future, thanks again for looking into this.

@zalimeni zalimeni force-pushed the zalimeni/fix-ci-skip-action branch 2 times, most recently from 9d6e228 to 135c745 Compare September 24, 2024 19:50
# The ability to do this is ultimately determined by the triggers of the calling
# workflow, since `base_ref` (the target branch of a PR) is only available in
# `pull_request` events, not `push`.
- name: Error if conditional check is not allowed
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Post-review note: updated this job name for clarity, since it's expected to be skipped if things are working correctly.

zalimeni added a commit to hashicorp/consul that referenced this pull request Sep 25, 2024
Adopt a third-party action to avoid script bugs, and to fix a current
issue where the script fails to detect all changes when processing push
events on PR branches.

Adapted from hashicorp/consul-dataplane#637. See that PR for testing
details and background context.
zalimeni added a commit to hashicorp/consul that referenced this pull request Sep 25, 2024
Adopt a third-party action to avoid script bugs, and to fix a current
issue where the script fails to detect all changes when processing push
events on PR branches.

Adapted from hashicorp/consul-dataplane#637. See that PR for testing
details and background context.
zalimeni added a commit to hashicorp/consul-k8s that referenced this pull request Sep 25, 2024
Adopt a third-party action to avoid script bugs, and to fix a current
issue where the script fails to detect all changes when processing push
events on PR branches.

Adapted from hashicorp/consul-dataplane#637. See that PR for testing
details and background context.
zalimeni added a commit to hashicorp/consul-k8s that referenced this pull request Sep 25, 2024
Adopt a third-party action to avoid script bugs, and to fix a current
issue where the script fails to detect all changes when processing push
events on PR branches.

Adapted from hashicorp/consul-dataplane#637. See that PR for testing
details and background context.
zalimeni added a commit to hashicorp/consul-k8s that referenced this pull request Sep 25, 2024
Adopt a third-party action to avoid script bugs, and to fix a current
issue where the script fails to detect all changes when processing push
events on PR branches.

Adapted from hashicorp/consul-dataplane#637. See that PR for testing
details and background context.
zalimeni added a commit to hashicorp/consul that referenced this pull request Sep 25, 2024
Adopt a third-party action to avoid script bugs, and to fix a current
issue where the script fails to detect all changes when processing push
events on PR branches.

Adapted from hashicorp/consul-dataplane#637. See that PR for testing
details and background context.
Adopt a third-party action to avoid script bugs, and to fix a current
issue where the script fails to detect all changes when processing push
events on PR branches.
@zalimeni
Copy link
Member Author

Updated changed-files action to pin latest allow-listed SHA.

@zalimeni zalimeni merged commit a81f552 into main Sep 25, 2024
41 checks passed
@zalimeni zalimeni deleted the zalimeni/fix-ci-skip-action branch September 25, 2024 16:44
zalimeni added a commit to hashicorp/consul that referenced this pull request Sep 25, 2024
ci: fix conditional skip and add safeguard

Adopt a third-party action to avoid script bugs, and to fix a current
issue where the script fails to detect all changes when processing push
events on PR branches.

Adapted from hashicorp/consul-dataplane#637. See that PR for testing
details and background context.
zalimeni added a commit to hashicorp/consul-k8s that referenced this pull request Sep 25, 2024
ci: fix conditional skip and add safeguard

Adopt a third-party action to avoid script bugs, and to fix a current
issue where the script fails to detect all changes when processing push
events on PR branches.

Adapted from hashicorp/consul-dataplane#637. See that PR for testing
details and background context.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport/1.1 Changes are backported to 1.1 backport/1.4 Changes are backported to 1.4 backport/1.5 Changes are backported to 1.5 backport/1.6 Changes are backported to 1.6 pr/no-changelog This PR does not introduce a user-facing change that should be reflected in the changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants