Workflow and download source cache enhancement

[henderkes]

https://github.com/static-php/spc-packages/actions/runs/17536273315

Today it took 50 minutes to run bin/spc download. 45 minutes of those because of GNU libraries. Maybe we should switch to a mirror by default?

[crazywhalecc]

I'm not sure the download should be switched to alt first, but we could add time limit for single download item just in case.

[henderkes]

If we choose a sensible time limit like 1 minute by default, it would always cancel gnu downloads. Might as well default to using a faster mirror then.

[crazywhalecc]

Considering the reliability of the source, I think using the mirror by default is feasible, but I am not sure if there is any limit on the pulling frequency of digitalocean spaces.

I'll decide after evaluation, but for now, I'd prefer adding an option to prioritize mirrors (or alts) for download.

[crazywhalecc]

Okay. The spaces oss has 1024GB bandwidth for free, and excess amount: 0.01$/GB. I'm not sure how much of a strain this will put on pulls, but I'm currently using an average of around 500GB of bandwidth per month. There is no problem with the current sponsorship coverage of NativePHP and BeyondCode.

Or we could mirror GitHub Release, just like pre-built, but this would also be subject to GitHub's limitations.

[henderkes]

Do pulls from one workflow to another count into that?

I might need to rework the package repositories into a self hosted one if that's the case. I run 18 gcc rpm, 6 zig rpm and 6 gcc deb package runners daily. That's 30 * 1gb per day * 30 days = 900gb per month.

[jingjingxyk]

https://ftpmirror.gnu.org/

https://www.gnu.org/prep/ftp.en.html#gnu_mirror_list

[crazywhalecc]

Do pulls from one workflow to another count into that?

All the GitHub things (GitHub Release assets, tarballs, GitHub Actions caches) are free for public repositories, normally we don't hit the limit, except matrix builds or reusable runners.

I might need to rework the package repositories into a self hosted one if that's the case. I run 18 gcc rpm, 6 zig rpm and 6 gcc deb package runners daily. That's 30 * 1gb per day * 30 days = 900gb per month.

So for GitHub itself, we should use cache as much as possible when building from Actions. This not only saves the cost of dl.static-php.dev, but also avoids the frequency limit of GitHub itself as much as possible. But I think we might need to take a moment to focus on all the caching and current actions issues.

Moreover, the current dl.static-php.dev also has too many redundant files, and the truly useful files do not exceed 200. The crawler may also consume traffic, although it may be negligible. We should also find time to clean that up.

[henderkes]

But I think we might need to take a moment to focus on all the caching and current actions issues.

Yes, but in order to make caching possible, we need version checks for downloads. Otherwise the rpms may be built based on old source code and wouldn't push updates fast enough for security fixes.