On demand runner

[sverhoeven]

At the moment the runner service must be kept online so it can accept jobs (GitHub Action workflow jobs). This requires a server to be running, doing nothing most of time, except when a job needs to be run. This is very wasteful.

We should have a setup that will start up a server (and runner) only when a job needs to be run and shutdown the server when no jobs need to run.

[sverhoeven]

For example https://040code.github.io/2020/05/25/scaling-selfhosted-action-runners uses AWS lambda functions.

[HannoSpreeuw]

But it is mostly offline, by default, right? I mean, after being active for CI, it only remains idle for a couple of minutes. So why wasteful?

Running something like ubuntu-virtualbox/playbook-set-up-runner.yml will turn its state from offline to idle. That can be run by default before any CI task.

[sverhoeven]

The packer-Ubuntu-18 is really offline.

When you start the runner it keeps polling to GitHub server indefinitely see https://help.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners#communication-between-self-hosted-runners-and-github

[sverhoeven]

Started in https://github.com/ci-for-science/on-demand-runner on a function to spin up/down servers. with runners on demand