Doc: how to make the CI look green

[LecrisUT]

This is orthogonal with what this action does, but it is a common consequence of using this action, namely how do we report on the top page that the current CI was successful? Some candidates:

• is there a badge to report the specific alls-green status? This is not ideal because the commit is still reported with a cross
• document using steps.<step_id>.outcome + another step with exit 0 to overwrite the exit status. Haven't confirmed that this works, but it could be useful to overwrite the CI status when on.push, while when on on.pull_request keep the normal report. This is still a lost cause as far as I've tested
• within a reusable workflow, could one use all-green to overwrite the job outcome of the workflow job that calls uses? When I've tried to use it, the workflow diagram can get a bit mesy, even though the documentation suggests that they should be compressed. Some tips for people using this design would be good

Upvote & Fund

• We're using Polar.sh so you can upvote and help fund this issue.
• We receive the funding once the issue is completed & confirmed by you.
• Thank you in advance for helping prioritize & fund our backlog.

[webknjaz]

• is there a badge to report the specific alls-green status? This is not ideal because the commit is still reported with a cross

If you use a normal GitHub CI/CD workflow badge for a workflow that has alls-green as the final job (and no following jobs), that badge will show the outcome of the entire workflow, which would essentially be whatever alls-green sets. It's just an action. If you have a step with the action in a job and it succeeds, the workflow will succeed. Though, the individual job statuses may show up as red, but that's not something GitHub allows to override. I suppose, there could be hacks, but they'd be very involved and super ugly anyway.

[webknjaz]

• within a reusable workflow, could one use all-green to overwrite the job outcome of the workflow job that calls uses? When I've tried to use it, the workflow diagram can get a bit mesy, even though the documentation suggests that they should be compressed. Some tips for people using this design would be good

I don't think so. Use alls-green within the calling workflow. Though, you can still use it additionally inside the reusable ones, if you have a use-case, of course.

[LecrisUT]

Indeed, so far the only thing that works is displaying a badge on the top page. The commit status are still displayed as red cross, which is the main complaint I've received when trying to implement this.

I have tried in vain any possible ugly way to mask the red cross, but they all seem to be blocked by set::neutral and the myriad of issues that try to request it ¹. The issue is that even if we can read the context object, we cannot edit it back.

Footnotes

1. https://github.com/actions/runner/issues/2347 ↩

[webknjaz]

Yep, it's been like this ever since GHA got first released for GA (maybe even since its private beta!). I've seen countless requests to handle this with zero influence. One of the common requests used to be a Travis CI style allow-failures (which uses a separate yellow/orange color there). This never got implemented. But if the runner were to implement the above command, it'd be much easier.

FWIW another workaround I had in mind was setting a special output: in the previous jobs and suppressing the statuses on the step level (a continue-on-failure status would have a red cross in the job, but the the job would be "green", one of the following steps would analyze the outcome of the allowed-to-fail job and set that magic output — but it's quite messy and I wouldn't encourage people to copy-and-paste this across projects).

[webknjaz]

@LecrisUT do you want to send a PR with the badge suggestion? Does the issue still need to be open?

[LecrisUT]

FWIW another workaround I had in mind was setting a special output: in the previous jobs and suppressing the statuses on the step level

That is the ugly approach I was trying to do where I got stuck at overwriting context. Since the only action control there is is setFailed, you can only add failed states via exit 1, not subtract them via exit 0. So unless the action itself supports allow-failure input, there is no generic way to work around it.

@LecrisUT do you want to send a PR with the badge suggestion? Does the issue still need to be open?

Yeah, I'll try to write a few words about that. Just ping me if I didn't get to it within 2 weeks.

[webknjaz]

That is the ugly approach I was trying to do where I got stuck at overwriting context. Since the only action control there is is setFailed, you can only add failed states via exit 1, not subtract them via exit 0. So unless the action itself supports allow-failure input, there is no generic way to work around it.

The action has a separate allow-failures input, but that won't work for filtering out the individual matrix jobs. My point was that you'd have to use continue-on-failure on the step level, analyze that step's result and write a new output. Then, you'd have to convert those outputs into the structure that this action understands (after all, it only works with the input people provide, not directly with GHA runner).

@LecrisUT do you want to send a PR with the badge suggestion? Does the issue still need to be open?

Yeah, I'll try to write a few words about that. Just ping me if I didn't get to it within 2 weeks.

Sure! I'll assign you, then.

[LecrisUT]

My point was that you'd have to use continue-on-failure on the step level, analyze that step's result and write a new output.

😮 I did not see that you can do continue-on-failure on step level. I will need to experiment with that. Thanks a lot for that.