Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Variables not passing through - FIXED #69

Open
m00nyONE opened this issue Feb 1, 2024 · 1 comment
Open

Variables not passing through - FIXED #69

m00nyONE opened this issue Feb 1, 2024 · 1 comment

Comments

@m00nyONE
Copy link

m00nyONE commented Feb 1, 2024

the {{.name}} parameter in the configMap does not get passed through to prometheus.
When i open the metrics tab i get no results back:
image

so i started investigating and wrote a very simple http-logger. When i look at the chrome console, it tells me that the correct request against argocd-metrics-server is fired:
https://argo.*******.***/extensions/metrics/api/applications/http-debugger/groupkinds/pod/rows/pod/graphs/pod_cpu_line?name=http-debugger-5b54d4556c-7hp2p.*&namespace=default&application_name=http-debugger&project=administration&uid=b6e323fe-a997-431a-b81a-41f1fcbb99a0&duration=1h

so the {{.name}} is clearly there in form of "http-debugger-5b54d4556c-7hp2p.*" which is correct.

when i look at the logs of my http-logger it shows me this:

Path: /api/v1/query_range
Headers:
Host: http-debugger.default.svc.cluster.local:8080
User-Agent: Go-http-client/1.1
Content-Length: 279
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip
Body:
end=1706789730.0744565&query=sum(rate(container_cpu_usage_seconds_total{pod=~"http-debug-fixed-test",+image!="",+container!="POD",+container!="",+container_name!="POD"}[5m]))+by+(pod)&start=1706786130.074456&step=60
INFO:root:10.244.13.63 - - [01/Feb/2024 12:15:30] "POST /api/v1/query_range HTTP/1.1" 200 -
INFO:root:POST request,
Path: /api/v1/query_range
Headers:
Host: http-debugger.default.svc.cluster.local:8080
User-Agent: Go-http-client/1.1
Content-Length: 259
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip
Body:
end=1706789730.0778503&query=sum(rate(container_cpu_usage_seconds_total{pod=~"",+container!="POD",+image!="",+container!="",+container_name!="POD"}[5m]))+by+(pod)&start=1706786130.07785&step=60
INFO:root:10.244.13.63 - - [01/Feb/2024 12:15:30] "POST /api/v1/query_range HTTP/1.1" 200 -
INFO:root:POST request,

The first one is a FIXED name in the config. i created a new row just to test if its even working. And yes, it works when i hardcode the podname into the config.

The second one is with {{.name}} variable. And here we can see that it did not get passed through to prometheus ( Thanos in my case here )

So i pinned down the problem to the argocd metrics server. Then i looked into the code here but I'm not a go expert anymore :D Maybe the query parameter does not get parsed correctly?
I was curious and investigated further.. i found that no placeholder whatsoever gets parsed. No namespace, no name, no whatever.

After i had a look inside the cluster to check if there is something wrong with my template, i found the solution:

Helm was the problem. I deploy all my applications via argocd and helm. So the {{.name}} got parsed by helm and replaced with "nothing"

So if anyone else has this problem, have a look here: helm/helm#2798 (comment)

it should look something like this:

                      {
                        "name": "pod_cpu_line",
                        "title": "CPU",
                        "description": "",
                        "graphType": "line",
                        "metricName": "pod",
                        "queryExpression": "sum(rate(container_cpu_usage_seconds_total{pod=~\"{{ "{{" }}.name{{ "}}" }}\", image!=\"\", container!=\"POD\", container!=\"\", container_name!=\"POD\"}[5m])) by (pod)"
                      },

i hope this can help someone :-)

@Tchoupinax
Copy link

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants