Skip to content

Commit

Permalink
feat: added basic metrics to track pod image ages over time and highl…
Browse files Browse the repository at this point in the history
…ight namespaces to updated
  • Loading branch information
hebestreit committed Oct 6, 2024
1 parent e767415 commit 1b3a2eb
Show file tree
Hide file tree
Showing 15 changed files with 862 additions and 47 deletions.
7 changes: 6 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,12 @@ k3d-import: docker-build

.PHONY: helm-install
helm-install: k3d-import
helm upgrade --install pod-image-aging charts/pod-image-aging --set image.tag="latest" --set dockerAuthSecretName=pod-image-aging-docker-auth
helm upgrade --install pod-image-aging charts/pod-image-aging --set image.tag="latest" --set dockerAuthSecretName=pod-image-aging-docker-auth --set metrics.enabled=true --set metrics.serviceMonitor.enabled=true --set dashboards.enabled=true

.PHONY: helm-install-monitoring
helm-install-monitoring:
kubectl create namespace monitoring || true
helm upgrade --install -n monitoring kube-prometheus-stack prometheus-community/kube-prometheus-stack --set prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues=false

##@ Build

Expand Down
51 changes: 50 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,9 @@ EOF
Create the secret from the above file:

```shell
kubectl -n $NAMESPACE create secret generic pod-image-aging-docker-auth --type=kubernetes.io/dockerconfigjson --from-file=.dockerconfigjson=.dockerconfigjson
kubectl -n $NAMESPACE create secret generic pod-image-aging-docker-auth \
--type=kubernetes.io/dockerconfigjson \
--from-file=.dockerconfigjson=.dockerconfigjson
```

### Install using Helm
Expand Down Expand Up @@ -151,6 +153,53 @@ NAMESPACE="default"
helm uninstall -n $NAMESPACE pod-image-aging
```

## Metrics

The `pod-image-aging` controller exposes a few metrics which can be scraped by Prometheus to visualize the image age
over time or to define alerts based on your company's policies.

Recommended configuration to use with the Prometheus Operator:

```yaml
metrics:
enabled: true
serviceMonitor:
enabled: true
dashboards:
enabled: true
```
You can enable the metrics by setting the `metrics.enabled` property to `true`.

If enabled the below metrics will be evaluated in an interval of 30 minutes. It's designed in this way to reduce the
load on the Kubernetes API and it's more likely that the pods are less frequently updated. You can change the interval
by setting the `metrics.interval` to a lower or higher value.

| Metric | Description | Labels |
|------------------------------------|---------------------------------------|----------------------|
| `pod_image_aging_youngest_seconds` | Age of the youngest image in seconds. | `exported_namespace` |
| `pod_image_aging_oldest_seconds` | Age of the oldest image in seconds. | `exported_namespace` |
| `pod_image_aging_average_seconds` | Average age of all images in seconds. | `exported_namespace` |

### ServiceMonitor

If you're using the Prometheus Operator
a [ServiceMonitor](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/api.md#servicemonitor)
resource can be created to automatically scrape the metrics by setting `metrics.serviceMonitor.enabled=true`.

### Grafana Dashboard

When using the official Grafana Helm chart you can automatically import the dashboard by setting
`dashboards.enabled=true`. This will create a ConfigMap with the dashboard definition which will be imported by the
sidecar container `grafana-sc-dashboard`.

https://github.com/grafana/helm-charts/tree/main/charts/grafana#sidecar-for-dashboards

Otherwise if you want to import the file manually take a look inside
the [charts/pod-image-aging/dashboards](charts/pod-image-aging/dashboards) folder for the JSON file.

![grafana-dashboard](docs/assets/grafana-dashboard.png)

# Development

Contributions are welcome!
Expand Down
Loading

0 comments on commit 1b3a2eb

Please sign in to comment.