Merge Queue: Github does not cancel workflows in merge queue even when they become irrelevant

[avx-ctan]

Select Topic Area

Question

Body

We noticed a behavior of Github workflow handling that leads to hidden charges for users if they are not aware of it. In essense, Github does not cancel workflows in merge queues even when they become irrelevant. And as far as we can tell there is no documentation on this.

Here's an example:
Assume there are 3 PRs --- PR1, PR2 and PR3, and there is a workflow to run to get the required checks. When all PRs queued in the merge queue, 3 workflows started.
workflow 1 running on main + PR1
workflow 2 running on main + PR1 + PR2
workflow 3 running on main + PR1 + PR2 + PR3

Now, if we want to move PR3 to be at the head of the queue, 3 new workflows will start:
workflow 4 running on main + PR3
workflow 5 running on main + PR3 + PR1
workflow 6 running on main + PR3 + PR1 + PR2

When this happens, workflow 1, 2 and 3 should be cancelled automatically because the results are irrelevant at this point.
However, currently this is not done automatically. This not only leads to huge waste of money for the unawared users, but also slow down the merge velocity because of the extra runner concurrency consumed.

In addition, currently Github APIs do not support users implementing this behavior themselves. We have tried numerous ways of setting the concurrency group to get the behavior we'd like. However, no information in merge_queue event can be used and no feature in the expression evaluation is useful in this case.

Is there anyone else who is also facing this issue? We hope someone at Github can take a look and address this (or tell us what can we do to achieve the desired behavior).

[AVMiraga]

Have you tried Github actions to do such a task?
I mean you have a bunch of control over workflow runs, if what you asking is doing what you are experiencing, no 100% thing will do it, you at least need to implement it yourself. Consider checking this API: Cancel Workflow Run

Continous work of the first three "un-essential" workflows, can be sometimes useful, especially if you need to trace something.

[isker]

However, no information in merge_queue event can be used and no feature in the expression evaluation is useful in this case.

I've faced this problem and have seemingly resolved it with this (remember, JSON is valid YAML 🌞):

  "concurrency": {
    "group": "${{ github.workflow }}-${{ github.event.merge_group.head_commit.message }}",
    "cancel-in-progress": true
  }

This works because we use "squash" as the merge queue commit method, and configure squash to only use the PR title as the commit message. In addition to the PR title, GitHub also adds the PR number to that generated commit message. So, it is a roundabout way of getting the PR number into the concurrency group.

I couldn't find any documentation on length/valid character limitations in group, but things are not broken with PR titles in group as far as I can tell.

If you don't use those squash and commit message settings, I'm not sure whether this approach can work.

[github-actions[bot]]

🕒 Discussion Activity Reminder 🕒

This Discussion has been labeled as dormant by an automated system for having no activity in the last 60 days. Please consider one the following actions:

1️⃣ Close as Out of Date: If the topic is no longer relevant, close the Discussion as out of date at the bottom of the page.

2️⃣ Provide More Information: Share additional details or context — or let the community know if you've found a solution on your own.

3️⃣ Mark a Reply as Answer: If your question has been answered by a reply, mark the most helpful reply as the solution.

Note: This dormant notification will only apply to Discussions with the Question label. To learn more, see our recent announcement.

Thank you for helping bring this Discussion to a resolution! 💬