Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] update link checker and handle status 429 (too many requests) #356

Merged
merged 5 commits into from
Oct 16, 2024

Conversation

egpbos
Copy link
Member

@egpbos egpbos commented Oct 16, 2024

This PR:

  • Updates checkout action to version 4
  • Updates the lychee action to version 2
  • Makes lychee accept status 429 (too many requests) as a successful check. This makes some sense: a 429 may very well be a live link, just one that currently you are not allowed to view. If it were a broken link, the server should return 404 instead. I'm not 100% sure if all webservers work this way (i.e. if they first check for 404 and only then apply throttling to give 429), but in any case it seems the best we can do, because we cannot otherwise control request rate from GitHub Actions, as far as I know.
  • Adds a manual run trigger to the link checker
  • Removes the workflow part where we only fail if there are 10 or more broken links. I think we can get to 0 broken links by using lychee options, like indeed the one above for 429 and handling other broken links properly as well or excluding them explicitly. I think this is preferable to leaving up to 9 broken links for possibly a long time.

Fixes workflow warning.
Especially gnu.org is giving a lot of 429 statuses (too many requests), seems like it doesn't like GitHub spam. There doesn't seem to be much one can do about this, so let's just ignore this code (add it to the accept list) and hope people run the link checker on the GNU links locally every now and then. In a way, accepting this code is not wrong, because a benevolent server will not give 429 if the URL is invalid (it will give 404 instead), so in a way 429 may signify that the URL is valid (it just won't load for you right now).
Handling all broken links does not seem unfeasible, especially when handling 429 statuses. Lychee has some other options like excluding certain domains, that we could use in exceptional cases.
Copy link
Member

@c-martinez c-martinez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks @egpbos !

@egpbos egpbos merged commit c77f660 into NLeSC:main Oct 16, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants