Kepler recording a lot of metrics possibly adding runtime overhead

[jharriga]

What happened?

On x86 system running container-kepler v0.7.10 bare-metal:
root# curl localhost:8888/metrics | grep kepler | wc -l
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 163k 0 163k 0 0 15.9M 0 --:--:-- --:--:-- --:--:-- 15.9M
1213

Pick a process and search for metrics per PID (ie. PID 986 = NetworkManager)
root# cat HOLD.txt | grep 986 | wc -l
25

Break down record count by record-type:
root# cat HOLD.txt | grep 986 | grep rapl | wc -l
12
root# cat HOLD.txt | grep 986 | grep cgroup | wc -l
4
root@# cat HOLD.txt | grep 986 | grep bpf | wc -l
9

Twenty-five records per PID. That's a lot of metrics and a large payload.

What did you expect to happen?

Raising concern on potential overhead of recording so many metrics per PID

How can we reproduce it (as minimally and precisely as possible)?

See cmd sequence above

Anything else we need to know?

No response

Kepler image tag

root# dnf list installed | grep kepler container-kepler.noarch 0.7.10-1 @System

Kubernetes version

bare-metal env

Cloud provider or bare metal

bare-metal root# uname -r 5.14.0-452.el9.x86_64

OS version

# On Linux:
$ cat /etc/os-release
PRETTY_NAME="CentOS Stream 9"

$ uname -a
Linux nuc7 5.14.0-452.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Install tools

root# wget https://github.com/sustainable-computing-io/kepler/releases/download/v0.7.10/kepler.rpm.tar.gz 2024-05-22 14:01:04 (9.44 MB/s) - ‘kepler.rpm.tar.gz’ saved root# tar xzvf kepler.rpm.tar.gz RPMS/ RPMS/noarch/ RPMS/noarch/container-kepler-0.7.10-1.noarch.rpm root# rpm -Uvh ./RPMS/noarch/container-kepler-0.7.10-1.noarch.rpm

Kepler deployment config

For on kubernetes:

$ KEPLER_NAMESPACE=kepler

# provide kepler configmap
$ kubectl get configmap kepler-cfm -n ${KEPLER_NAMESPACE} 
# paste output here

# provide kepler deployment description
$ kubectl describe deployment kepler-exporter -n ${KEPLER_NAMESPACE} 

For standalone:

put your Kepler command argument here

Container runtime (CRI) and version (if applicable)

Related plugins (CNI, CSI, ...) and versions (if applicable)