Enable conditional setting of unhealthyPodEvictionPolicy on PDBs #4093
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This will ensure that pods in CrashLoopBackOff don't impact pool scaledowns (since otherwise, Karpenter will refuse to consolidate nodes due to PDBs) This is not enabled by default due to our legacy clusters, but will be toggled at a per-cluster level through SystemPaastaConfig. This PR is also somewhat spicy due to these legacy clusters, so we'll definitely need to do some testing to make sure that the upgraded k8s clientlib doesn't cause chaos there (or we'll have to take more drastic action). That said, I've upgraded the k8s clientlib version to a somewhat more modern version to match the oldest non-legacy cluster we have and therefore had to make a number of changes (e.g., V1Subject was renamed to RbacV1Subject, etc)
| if load_system_paasta_config().get_enable_unhealthy_pod_eviction(): | ||
| spec = V1PodDisruptionBudgetSpec( | ||
| max_unavailable=max_unavailable, | ||
| unhealthy_pod_eviction_policy="AlwaysAllow", # XXX: should this be configurable? |
There was a problem hiding this comment.
I would always use unhealthy_pod_eviction_policy by reading value from config with default value ("IfHealthyBudget")
There was a problem hiding this comment.
But.. I forgot about k8s clusters with older version. Do we pin old PaaSTA version there?
|
Forked this into a new PR #4146 with some changes. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This will ensure that pods in CrashLoopBackOff don't impact pool scaledowns (since otherwise, Karpenter will refuse to consolidate nodes due to PDBs)
This is not enabled by default due to our legacy clusters, but will be toggled at a per-cluster level through SystemPaastaConfig.
This PR is also somewhat spicy due to these legacy clusters, so we'll definitely need to do some testing to make sure that the upgraded k8s clientlib doesn't cause chaos there (or we'll have to take more drastic action).
That said, I've upgraded the k8s clientlib version to a somewhat more modern version to match the oldest non-legacy cluster we have and therefore had to make a number of changes (e.g., V1Subject was renamed to RbacV1Subject, etc)
TODO