Increase limits_cpu for the short_running_workers (#615)

Increase limits_cpu for the short_running_workers

I am monitoring the hard time limits exceptions since a while, seems like that when we have spikes both in the short running workers and in the long running we sometimes get an hard time limit exception for the celery tasks.
We have spike above the requested limits (as you can see in the today picture) mainly for the short running workers.
We have exceptions for the spikes even when the single worker cpu usage is below the limits_cpu and I am wondering if this limit applies to the sum of all our replicas or just for the single replica?
I would say it is the limit for all the replicas together and in this case we need to increase much more the limits_cpu both for the long running and the short running pods (I would put there 1G of limits_cpu for both kind of workers in this case).


One of the highest cpu requests (for just a single short running worker) I registered:

Reviewed-by: Matej Focko

playbooks/deploy.yml

@@ -250,7 +250,7 @@
         worker_requests_memory: "320Mi"
         worker_requests_cpu: "80m"
         worker_limits_memory: "640Mi"
-        worker_limits_cpu: "400m"
+        worker_limits_cpu: "700m"
       ansible.builtin.include_tasks: tasks/k8s.yml
       loop:
         - "{{ lookup('template', '{{ project_dir }}/openshift/packit-worker.yml.j2') }}"

playbooks/roles/deploy/tasks/main.yml

@@ -206,7 +206,7 @@
     worker_requests_memory: "320Mi"
     worker_requests_cpu: "80m"
     worker_limits_memory: "640Mi"
-    worker_limits_cpu: "400m"
+    worker_limits_cpu: "700m"
   ansible.builtin.include_tasks: tasks/k8s.yml
   loop:
     - "{{ lookup('template', '{{ project_dir }}/openshift/packit-worker.yml.j2') }}"