Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to disable KeyRotation #659

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

black-dragon74
Copy link
Member

This patch adds the feature to disable key
rotation by annotating any of NS, SC or PVC.

The annotation to be used is: keyrotation.csiaddons.openshift.io/disable=true

@black-dragon74
Copy link
Member Author

black-dragon74 commented Sep 3, 2024

Testing

Steps

❯ oc annotate sc/rook-ceph-block "keyrotation.csiaddons.openshift.io/disable=true"
storageclass.storage.k8s.io/rook-ceph-block annotated

❯ oc get encryptionkeyrotationcronjob
No resources found in rook-ceph namespace.

❯ oc annotate sc/rook-ceph-block "keyrotation.csiaddons.openshift.io/disable-"
storageclass.storage.k8s.io/rook-ceph-block annotated

❯ oc get encryptionkeyrotationcronjob
NAME                 SCHEDULE   SUSPEND   ACTIVE   LASTSCHEDULE   AGE
rbd-pvc-1725353742   @weekly                                      2s

Logs

// Add disable annotation
2024-09-03T08:57:55.639Z        INFO    key rotation is disabled, exiting reconcile     {"controller": "persistentvolumeclaim", "controllerGroup": "", "controllerKind": "PersistentVolumeClaim", "PersistentVolumeClaim": {"name":"rbd-pvc","namespace":"rook-ceph"}, "namespace": "rook-ceph", "name": "rbd-pvc", "reconcileID": "4e2de5e8-dbe9-4f6a-a474-dede37f470d5", "EncryptionKeyrotationCronJobName": "rbd-pvc-1725353795"}
2024-09-03T08:57:55.639Z        INFO    Annotation not set, exiting reconcile   {"controller": "persistentvolumeclaim", "controllerGroup": "", "controllerKind": "PersistentVolumeClaim", "PersistentVolumeClaim": {"name":"rbd-pvc","namespace":"rook-ceph"}, "namespace": "rook-ceph", "name": "rbd-pvc", "reconcileID": "4e2de5e8-dbe9-4f6a-a474-dede37f470d5", "EncryptionKeyrotationCronJobName": "rbd-pvc-1725353795"}
2024-09-03T08:57:55.642Z        INFO    key rotation is disabled, exiting reconcile     {"controller": "persistentvolumeclaim", "controllerGroup": "", "controllerKind": "PersistentVolumeClaim", "PersistentVolumeClaim": {"name":"rbd-pvc","namespace":"rook-ceph"}, "namespace": "rook-ceph", "name": "rbd-pvc", "reconcileID": "8acbfc1b-b671-44c5-a65b-02c75f6fa608"}
2024-09-03T08:57:55.642Z        INFO    encryptionkeyrotationcronjob resource not found {"controller": "encryptionkeyrotationcronjob", "controllerGroup": "csiaddons.openshift.io", "controllerKind": "EncryptionKeyRotationCronJob", "EncryptionKeyRotationCronJob": {"name":"rbd-pvc-1725353795","namespace":"rook-ceph"}, "namespace": "rook-ceph", "name": "rbd-pvc-1725353795", "reconcileID": "fb043d65-8c02-4926-b7d9-d41b52f8c325"}
2024-09-03T08:57:55.642Z        INFO    Annotation not set, exiting reconcile   {"controller": "persistentvolumeclaim", "controllerGroup": "", "controllerKind": "PersistentVolumeClaim", "PersistentVolumeClaim": {"name":"rbd-pvc","namespace":"rook-ceph"}, "namespace": "rook-ceph", "name": "rbd-pvc", "reconcileID": "8acbfc1b-b671-44c5-a65b-02c75f6fa608"}

// Remove disable annotation
2024-09-03T08:56:35.118Z        INFO    Adding keyrotation annotation to the pvc        {"controller": "persistentvolumeclaim", "controllerGroup": "", "controllerKind": "PersistentVolumeClaim", "PersistentVolumeClaim": {"name":"rbd-pvc","namespace":"rook-ceph"}, "namespace": "rook-ceph", "name": "rbd-pvc", "reconcileID": "0d37b24c-80e4-4fe3-9ab2-021fe6443ff4", "KeyRotationSchedule": "@weekly", "annotation": "{\"metadata\":{\"annotations\":{\"keyrotation.csiaddons.openshift.io/cronjob\":\"rbd-pvc-1725353795\",\"keyrotation.csiaddons.openshift.io/schedule\":\"@weekly\"}}}"}
2024-09-03T08:56:35.156Z        INFO    successfully created new encryptionkeyrotationcronjob   {"controller": "persistentvolumeclaim", "controllerGroup": "", "controllerKind": "PersistentVolumeClaim", "PersistentVolumeClaim": {"name":"rbd-pvc","namespace":"rook-ceph"}, "namespace": "rook-ceph", "name": "rbd-pvc", "reconcileID": "0d37b24c-80e4-4fe3-9ab2-021fe6443ff4", "KeyRotationSchedule": "@weekly"}
2024-09-03T08:56:35.156Z        INFO    Annotation not set, exiting reconcile   {"controller": "persistentvolumeclaim", "controllerGroup": "", "controllerKind": "PersistentVolumeClaim", "PersistentVolumeClaim": {"name":"rbd-pvc","namespace":"rook-ceph"}, "namespace": "rook-ceph", "name": "rbd-pvc", "reconcileID": "0d37b24c-80e4-4fe3-9ab2-021fe6443ff4", "KeyRotationSchedule": "@weekly"}
2024-09-03T08:56:35.166Z        INFO    no upcoming schedule, requeue with delay until next run {"controller": "encryptionkeyrotationcronjob", "controllerGroup": "csiaddons.openshift.io", "controllerKind": "EncryptionKeyRotationCronJob", "EncryptionKeyRotationCronJob": {"name":"rbd-pvc-1725353795","namespace":"rook-ceph"}, "namespace": "rook-ceph", "name": "rbd-pvc-1725353795", "reconcileID": "fc1371ca-b25b-4844-af08-0c3d9ec50b1c", "now": "2024-09-03T08:56:35.166Z", "nextRun": "2024-09-08T00:00:00.000Z"}
2024-09-03T08:56:35.168Z        INFO    Annotation not set, exiting reconcile   {"controller": "persistentvolumeclaim", "controllerGroup": "", "controllerKind": "PersistentVolumeClaim", "PersistentVolumeClaim": {"name":"rbd-pvc","namespace":"rook-ceph"}, "namespace": "rook-ceph", "name": "rbd-pvc", "reconcileID": "f539df5b-9f6e-4117-83c7-62c9feb621af", "EncryptionKeyrotationCronJobName": "rbd-pvc-1725353795", "KeyRotationSchedule": "@weekly"}
2024-09-03T08:56:35.177Z        INFO    no upcoming schedule, requeue with delay until next run {"controller": "encryptionkeyrotationcronjob", "controllerGroup": "csiaddons.openshift.io", "controllerKind": "EncryptionKeyRotationCronJob", "EncryptionKeyRotationCronJob": {"name":"rbd-pvc-1725353795","namespace":"rook-ceph"}, "namespace": "rook-ceph", "name": "rbd-pvc-1725353795", "reconcileID": "93175055-cb07-4c9f-b9ad-85b813cd8eff", "now": "2024-09-03T08:56:35.177Z", "nextRun": "2024-09-08T00:00:00.000Z"}
2024-09-03T08:56:35.184Z        INFO    successfully updated encryptionkeyrotationcronjob       {"controller": "persistentvolumeclaim", "controllerGroup": "", "controllerKind": "PersistentVolumeClaim", "PersistentVolumeClaim": {"name":"rbd-pvc","namespace":"rook-ceph"}, "namespace": "rook-ceph", "name": "rbd-pvc", "reconcileID": "90551e91-e95b-4c32-9673-97bf15b7f7da", "EncryptionKeyrotationCronJobName": "rbd-pvc-1725353795", "KeyRotationSchedule": "@weekly"}
2024-09-03T08:56:35.184Z        INFO    Annotation not set, exiting reconcile   {"controller": "persistentvolumeclaim", "controllerGroup": "", "controllerKind": "PersistentVolumeClaim", "PersistentVolumeClaim": {"name":"rbd-pvc","namespace":"rook-ceph"}, "namespace": "rook-ceph", "name": "rbd-pvc", "reconcileID": "90551e91-e95b-4c32-9673-97bf15b7f7da", "EncryptionKeyrotationCronJobName": "rbd-pvc-1725353795", "KeyRotationSchedule": "@weekly"}

Comment on lines 737 to 742
val, ok := annotations[annotation]
if ok {
return strings.ToLower(val) == "true", nil
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a helper function for it as we have same check in 3 places

@@ -65,6 +65,7 @@ var (
rsCSIAddonsDriverAnnotation = "reclaimspace." + csiaddonsv1alpha1.GroupVersion.Group + "/drivers"

krcJobScheduleTimeAnnotation = "keyrotation." + csiaddonsv1alpha1.GroupVersion.Group + "/schedule"
krcJobDisableAnnotation = "keyrotation." + csiaddonsv1alpha1.GroupVersion.Group + "/disable"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/enable instead of disable. This is the one we are considering for reclaimspace as well.

Comment on lines 806 to 810
logger.Error(err, "failed to delete child encryptionkeyrotationcronjob")

return fmt.Errorf("failed to delete child encryptionkeyrotationcronjob: %w", err)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
logger.Error(err, "failed to delete child encryptionkeyrotationcronjob")
return fmt.Errorf("failed to delete child encryptionkeyrotationcronjob: %w", err)
errMsg:=""failed to delete child encryptionkeyrotationcronjob"
logger.Error(err, errMsg)
return fmt.Errorf(%s: %w",errMsg, err)

This patch adds the feature to disable key
rotation by annotating either of NS, SC or PVC.

The annotation to be used is:
`keyrotation.csiaddons.openshift.io/disable=true`

Signed-off-by: Niraj Yadav <[email protected]>
@@ -65,6 +65,7 @@ var (
rsCSIAddonsDriverAnnotation = "reclaimspace." + csiaddonsv1alpha1.GroupVersion.Group + "/drivers"

krcJobScheduleTimeAnnotation = "keyrotation." + csiaddonsv1alpha1.GroupVersion.Group + "/schedule"
krcJobDisableAnnotation = "keyrotation." + csiaddonsv1alpha1.GroupVersion.Group + "/enable"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name contains Disable, but the value ends with /enable. This is a little confusing.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. I'll edit the name and logic accordingly.

@@ -717,6 +723,62 @@ func (r *PersistentVolumeClaimReconciler) findChildEncryptionKeyRotationCronJob(
return activeJob, nil
}

// checkDisabledAnnotation checks if the annotation is set in the
// PVC, namespace or the storage class. It returns true if the
// annotation is set to `false`.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quite a confusing description if someone does not know the disabled annotation ends with /enabled. Please read it back to yourself, while you forgot the actual annotation key.


// Check StorageClass
sc := &storagev1.StorageClass{}
err = r.Client.Get(ctx, types.NamespacedName{Name: *pvc.Spec.StorageClassName}, sc)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

StorageClasses are not namespaced, is there an alternative to types.NamespacedName?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we can use client.ObjectKeyFromObject or client.ObjectKey instead.

@nixpanic
Copy link
Collaborator

Also don't forget to add the new annotation to the documentation!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants