Caching

[ArneTR]

From @rajbos

I'm trying to understand the discussions in various places about caching. Piecing that story and reasoning together failed, so I'll do an attempt here so future folks have an overview as well.

Javascript route

As far as I understand you tried to use a Javascript / Typescript action because of the pre and post options, but that would not let you use caching for the ML module you are using from Python? Was there no option to use everything from Javascript?

Chosen solution

The solution that has been chosen is a composite action with different parameters for the stage you want to execute, which nicely gives you the benefit of using actions/cache to store the used Python files in the repo's cache as well as an option to measure specific steps

Question: why not use Docker for this?

I think using Docker would make the action a lot faster, since the caching happens inside of the Docker image (provided you push the image to GitHub Container Registry and not use a local Dockerfile that gets rebuild every time). That way, not every job that uses this action will download all files (across go, python, and even spec-power-model) And why compile the reporter each and every time?

All those things can be versioned properly. Assuming the base image we use is small, downloading the image from gcr.io will be faster then all the steps that happen at the moment.

Passing in the three parameters with a Docker based action still works the same way.

If we want to make things as efficient (and thus green) as possible, than this would save a lot of download and install traffic / CPU.

Or is it then an issue to grab the CPU utilization from the host, which will not work inside of the Docker based action?

[dan-mm]

Hi @rajbos , just wanted to give you an update on this regarding the Docker caching as we've had an internal discussion about it regarding what makes the most sense to use and wanted to fill you in

At the moment, we don't believe that moving it over to a docker image would save that much computation power/speed. Most of our caching needs is network traffic based, pulling mainly python, our xgboost model, and a single go library.

Both the cached pip packages and our xgboost repository already exist within githubs internal network, and this is the majority of the network traffic we pull. Whether we cache as we do now, or make an image, we expect the same amount of MB of traffic to have to be pulled on each run, all from somewhere within github's internal network. For that reason, we don't really expect much of a speed or network savings.

As for the rest, compiling the cpu reporter is very minimal (less than 100ms of compute time), as is pulling the ascii graph library (in cpu time and traffic).

Additionally, in terms of overall energy savings, we expect a bit of overhead using docker as that would be a new image that needs to be created and stored somewhere, whereas the python pip packages are already cached in github's internal caching network. Though granted, that would be quite small in comparison to overall traffic shared across all repositories using this action. But still, we think the network traffic would be about the same.

For this reason we're not keen on spending the dev time at the moment to rewrite the action, though if you have any more insights into this we'd be happy to hear them

[rajbos]

Thanks for taking the time to explain, makes sense. The difference is probably not large enough.

And then the main reason for not doing this in Javascript is that there is completely no option to cache, right?

[ArneTR]

Jap, my research showed that it is way more complicated than doing it in node.

See an example here: https://github.com/newrelic/newrelic-ruby-agent/blob/f0290ab6468ad205dd014d63c794883dc47eebe7/.github/actions/build-ruby/index.js#L357

It includes require('@actions/cache') and uses it's methods ...

To my knowledge is no other option and this one seemed quite more work than just using bash and composites actions