🐛 Perform draining and volume detachment once until completion #11590
Conversation
k8s-ci-robot
added
the
cncf-cla: yes
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
size/L
|
/area bootstrap |
k8s-ci-robot
added
area/bootstrap
Danil-Grigorev
force-pushed
the
drain-once-until-completion
branch
from
df2f637 to
76d0706
Compare
Danil-Grigorev
changed the title
Danil-Grigorev
force-pushed
the
drain-once-until-completion
branch
from
76d0706 to
f472e1a
Compare
k8s-ci-robot
added
size/M
| @@ -657,14 +657,26 @@ func (r *Reconciler) isNodeDrainAllowed(m *clusterv1.Machine) bool { | |||
| return false | |||
| } | |||
|
|
|||
| if conditions.IsTrue(m, clusterv1.DrainingSucceededCondition) { | |||
There was a problem hiding this comment.
We should not start to rely on this condition to be set as it is because its gonna be deprecated/removed in v1beta2.
Also propably a change in the behavior for all VMs (previously followed reconciliation may also run again through drain).
There was a problem hiding this comment.
I unified the check under the PreTerminateDeleteHookSucceededCondition, which allows for both methods to run until completion. It is not listed in the proposal as a part of deprecation/removal AFAICT
Signed-off-by: Danil-Grigorev <[email protected]>
Danil-Grigorev
force-pushed
the
drain-once-until-completion
branch
from
f472e1a to
1021f8e
Compare
|
Thanks for review @chrischdi 👍🏼 |
|
/lgtm A backport would be very much appreciated 🙏 |
k8s-ci-robot
added
the
lgtm
|
LGTM label has been added. Git tree hash: 0bcc510758d20e84eb7b2d56a889fcf26def4356
|
|
/area machine |
k8s-ci-robot
added
the
area/machine
What this PR does / why we need it:
The draining logic continues to access the cluster even after the hook is removed or completed, causing a deadlock on machine removal in a non KCP backed CP provider implementation.
In a cluster with a kubelet local mode, competed draining and etcd membership removal causes inability to access API server externally, so the operation continues to error out even after success. This does not allow underlying machine to be removed.
The solution makes the
PreTeminateHookagnostic to the provider, and ensures that completion of the operation does not lead to further attempts to access the cluster.Which issue(s) this PR fixes (optional, in
fixes #<issue number>(, fixes #<issue_number>, ...)format, will close the issue(s) when PR gets merged):Fixes #11591
/area machine