Balance jobs between two ARCs

[blackliner]

We do have two locations where we run our self hosted runners:

• onprem datacenter kubernetes cluster
• AWS EKS cluster

The goal is to have as many jobs running on the onprem cluster, before spilling over to AWS. How can this be achieved with ARC? (both ARCs provide the same runner label!)

EDIT: I found High availability and automatic failover, which states:

If both runner scale sets are online, jobs assigned to them will be distributed arbitrarily (assignment race). You cannot configure the job assignment algorithm.

That is sad to hear. It would be very helpful to do something similar to what we are using right now with Gitlab CI runners, where we set the polling time to 1s on the onprem cluster, and 10s on the cloud cluster.