-
Notifications
You must be signed in to change notification settings - Fork 1
feat(helm)!: Update 📊 Observability: Prometheus (major) #656
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
homebot-0
wants to merge
1
commit into
main
Choose a base branch
from
renovate/major-observability-prometheus
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+2
−2
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
🏁 Script executed:
Repository: jlengelbrecht/prox-ops
Length of output: 140
🏁 Script executed:
Repository: jlengelbrecht/prox-ops
Length of output: 6918
🏁 Script executed:
Repository: jlengelbrecht/prox-ops
Length of output: 1756
🏁 Script executed:
Repository: jlengelbrecht/prox-ops
Length of output: 422
🌐 Web query:
kube-prometheus-stack 81.0.0 breaking changes release notes💡 Result:
kube-prometheus-stack
81.0.0— breaking / upgrade-impacting changesMajor chart bump is driven by Prometheus Operator upgrade to
v0.88.0. (newreleases.io)Practical implication: expect CRD/schema and admission-webhook validation changes consistent with Prometheus Operator
v0.88.0.Prometheus Operator
v0.88.0notable “CHANGE” items (most likely to bite during upgrades):v0.85.0. (github.com)EC2/Lightsailservice discovery inScrapeConfigwhen Prometheus is>= 3.8.0. (github.com)CRD upgrade warning (operational breaking change): Helm upgrades commonly don’t upgrade CRDs automatically, which can cause failed upgrades / validation errors unless you update CRDs separately. (github.com)
Citations:
🌐 Web query:
kube-prometheus-stack v81 failurePolicy Prometheus Operator v0.88.0💡 Result:
In kube-prometheus-stack v81.0.0 (which bumps Prometheus Operator to v0.88.0) the admission webhook behavior is controlled by:
Fail/Ignore/IgnoreOnInstallOnlyforfailurePolicy(withIgnoreOnInstallOnlymeaning “Ignore on install, Fail afterwards”). [2]failed calling webhook ... x509: certificate signed by unknown authority/ webhook not reachable, the common mitigation is settingfailurePolicy: Ignore(orIgnoreOnInstallOnly) so the API server won’t block CR updates when the webhook is temporarily broken/unreachable. [3]Also note: in this chart, if you explicitly set
prometheusOperator.admissionWebhooks.failurePolicy, that value is used (even when the webhook patch job is enabled). [4]Sources: [1] [2] [3] [4]
🌐 Web query:
forceDeployDatasources deprecated kube-prometheus-stack 81💡 Result:
In the upstream prometheus-community/kube-prometheus-stack v81.x releases (e.g. 81.0.0 on Jan 16, 2026) there isn’t an announced removal of
grafana.forceDeployDatasources; it’s still a supported value in the chart and is meant to “create datasource configmap even if grafana deployment has been disabled.” [1][2]So if you’re seeing “deprecated” warnings, they’re most likely coming from a wrapper/consumer chart (or a values-schema/linter in your platform), not from kube-prometheus-stack itself.
What to do:
grafana.enabled: falsegrafana.forceDeployDatasources: true(and similarlyforceDeployDashboardsif needed) [2]grafana.enabled: true), you typically don’t needforceDeployDatasources(leave itfalse). [2]Sources
[1] Release list showing kube-prometheus-stack 81.0.0/81.x (dates/versions) — releasealert.dev
[2] kube-prometheus-stack values.yaml changelog snippet documenting
forceDeployDatasourcespurpose — baptiste.is-a.dev🌐 Web query:
serviceMonitorSelectorNilUsesHelmValues deprecated removed v81💡 Result:
In the prometheus-community/kube-prometheus-stack Helm chart,
prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues(and the other*SelectorNilUsesHelmValuesflags) were deprecated in the 62.x → 63.x upgrade, with a documented migration path. The chart maintainers recommend replacing them by explicitly setting the selector, and usingmatchLabels: nullto force an “empty selector” behavior. [1]What to do (migration)
If you previously had:
Change to:
This same pattern applies to the related flags (
podMonitorSelectorNilUsesHelmValues,probeSelectorNilUsesHelmValues,ruleSelectorNilUsesHelmValues,scrapeConfigSelectorNilUsesHelmValues, etc.). [1]If you’re seeing failures on chart v81.x, it’s consistent with these deprecated flags no longer being accepted by the chart/templates you’re upgrading to—so removing them and switching to explicit selectors as above is the intended fix. [1]
Source
[1] kube-prometheus-stack upgrade notes (62.x → 63.x), deprecating
*SelectorNilUsesHelmValuesand thematchLabels: nullreplacement. (gist.github.com)Citations:
Migrate deprecated
*SelectorNilUsesHelmValuespatterns before upgrading to 81.x.The config uses deprecated selector flags introduced in v62→63 that are no longer accepted in v81:
serviceMonitorSelectorNilUsesHelmValues: false(line 64)podMonitorSelectorNilUsesHelmValues: false(line 65)ruleSelectorNilUsesHelmValues: false(line 66)Replace with explicit selectors:
Also verify CRD updates: v81.0.0 bundles Prometheus Operator v0.88.0 with narrower StatefulSet informer selectors. Ensure
crds: CreateReplacein the upgrade strategy (already configured) triggers a CRD refresh, since Helm doesn't auto-upgrade CRDs by default.🤖 Prompt for AI Agents