Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generates VolRepClasses based on the RamenDR replication schedule resource #177

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

raaizik
Copy link
Contributor

@raaizik raaizik commented Jul 2, 2024

Changess

  • Creates VolumeReplicationClass in response to changes made to DRClusterConfig -- the RamenDR replication schedule CR. VRCs are generated per each sched interval detailed in DRClusterConfig, for both flatten and non-flatten mode VRC types.
  • Handles VRCs deletion in case of removed replication scheduling intervals
  • kubernetes-csi-addons version bump

RHSTOR-5753

Copy link

openshift-ci bot commented Jul 2, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: raaizik
Once this PR has been reviewed and has the lgtm label, please assign nb-ohad for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@raaizik raaizik changed the title Generates VRCs based upon the RamenDR replication schedule resource Generates VRCs based on the RamenDR replication schedule resource Jul 2, 2024
@raaizik raaizik force-pushed the RHSTOR-5753 branch 2 times, most recently from 597255c to 7a6fc7b Compare July 21, 2024 11:36
@raaizik raaizik force-pushed the RHSTOR-5753 branch 2 times, most recently from 626d539 to f0ceee7 Compare July 28, 2024 10:01
@raaizik raaizik force-pushed the RHSTOR-5753 branch 3 times, most recently from 08910fd to 8ee12ce Compare October 28, 2024 15:10
@raaizik raaizik changed the title Generates VRCs based on the RamenDR replication schedule resource Generates VolRepClass based on the RamenDR replication schedule resource Nov 5, 2024
@raaizik raaizik force-pushed the RHSTOR-5753 branch 3 times, most recently from 66dca4f to 27ee9da Compare November 5, 2024 15:05
@raaizik raaizik changed the title Generates VolRepClass based on the RamenDR replication schedule resource Generates VolRepClasses based on the RamenDR replication schedule resource Nov 5, 2024
@raaizik
Copy link
Contributor Author

raaizik commented Nov 5, 2024

Depends on #2727

@rewantsoni
Copy link
Member

rewantsoni commented Nov 13, 2024

Testing:

Before creating drClusterConfig
[rewantsoni scripts] k get drclusterconfig -oyaml && k get volumereplicationclasses.replication.storage.openshift.io -oyaml
apiVersion: v1
items: []
kind: List
metadata:
  resourceVersion: ""
apiVersion: v1
items: []
kind: List
metadata:
  resourceVersion: ""
  
After creating drClusterConfig
[rewantsoni scripts] k get drclusterconfig -oyaml && k get volumereplicationclasses.replication.storage.openshift.io -oyaml
apiVersion: v1
items:
- apiVersion: ramendr.openshift.io/v1alpha1
  kind: DRClusterConfig
  metadata:
    creationTimestamp: "2024-11-13T05:45:15Z"
    generation: 1
    name: example
    resourceVersion: "2016433"
    uid: 1c89ffd8-b149-42d9-9207-f31707bb4b36
  spec:
    replicationSchedules:
    - 1m
    - 2m
kind: List
metadata:
  resourceVersion: ""
apiVersion: v1
items:
- apiVersion: replication.storage.openshift.io/v1alpha1
  kind: VolumeReplicationClass
  metadata:
    annotations:
      ocs.openshift.io/storageclaim: ocs-storagecluster-ceph-rbd
      replication.storage.openshift.io/is-default-class: "true"
    creationTimestamp: "2024-11-13T05:45:15Z"
    generation: 1
    labels:
      ramendr.openshift.io/replicationid: 01bf33de038b327272ad441a6836e389
      ramendr.openshift.io/storageID: 88db0eb5300a85c435ac0a0013075bee
    name: ocs-storagecluster-ceph-rbd-1m
    resourceVersion: "2016434"
    uid: a2d320b4-46a5-4a79-9505-6121fae98487
  spec:
    parameters:
      clusterID: 81fd58def37a35d1b4d4bee7ba35c3c3
      mirroringMode: snapshot
      replication.storage.openshift.io/replication-secret-name: ceph-client-provisioner-88db0eb5300a85c435ac0a0013075bee
      replication.storage.openshift.io/replication-secret-namespace: openshift-storage
      schedulingInterval: 1m
    provisioner: openshift-storage.rbd.csi.ceph.com
- apiVersion: replication.storage.openshift.io/v1alpha1
  kind: VolumeReplicationClass
  metadata:
    annotations:
      ocs.openshift.io/storageclaim: ocs-storagecluster-ceph-rbd
    creationTimestamp: "2024-11-13T05:45:15Z"
    generation: 1
    labels:
      ramendr.openshift.io/replicationid: 01bf33de038b327272ad441a6836e389
      ramendr.openshift.io/storageID: 88db0eb5300a85c435ac0a0013075bee
      replication.storage.openshift.io/flatten-mode: force
    name: ocs-storagecluster-ceph-rbd-1m-flatten
    resourceVersion: "2016436"
    uid: 884f5c2f-602a-453e-83ab-9af429aeddae
  spec:
    parameters:
      clusterID: 81fd58def37a35d1b4d4bee7ba35c3c3
      flattenMode: force
      mirroringMode: snapshot
      replication.storage.openshift.io/replication-secret-name: ceph-client-provisioner-88db0eb5300a85c435ac0a0013075bee
      replication.storage.openshift.io/replication-secret-namespace: openshift-storage
      schedulingInterval: 1m
    provisioner: openshift-storage.rbd.csi.ceph.com
- apiVersion: replication.storage.openshift.io/v1alpha1
  kind: VolumeReplicationClass
  metadata:
    annotations:
      ocs.openshift.io/storageclaim: ocs-storagecluster-ceph-rbd
      replication.storage.openshift.io/is-default-class: "true"
    creationTimestamp: "2024-11-13T05:45:15Z"
    generation: 1
    labels:
      ramendr.openshift.io/replicationid: 01bf33de038b327272ad441a6836e389
      ramendr.openshift.io/storageID: 88db0eb5300a85c435ac0a0013075bee
    name: ocs-storagecluster-ceph-rbd-2m
    resourceVersion: "2016435"
    uid: 839167c8-51a0-4730-9724-331917f6c00b
  spec:
    parameters:
      clusterID: 81fd58def37a35d1b4d4bee7ba35c3c3
      mirroringMode: snapshot
      replication.storage.openshift.io/replication-secret-name: ceph-client-provisioner-88db0eb5300a85c435ac0a0013075bee
      replication.storage.openshift.io/replication-secret-namespace: openshift-storage
      schedulingInterval: 2m
    provisioner: openshift-storage.rbd.csi.ceph.com
- apiVersion: replication.storage.openshift.io/v1alpha1
  kind: VolumeReplicationClass
  metadata:
    annotations:
      ocs.openshift.io/storageclaim: ocs-storagecluster-ceph-rbd
    creationTimestamp: "2024-11-13T05:45:15Z"
    generation: 1
    labels:
      ramendr.openshift.io/replicationid: 01bf33de038b327272ad441a6836e389
      ramendr.openshift.io/storageID: 88db0eb5300a85c435ac0a0013075bee
      replication.storage.openshift.io/flatten-mode: force
    name: ocs-storagecluster-ceph-rbd-2m-flatten
    resourceVersion: "2016437"
    uid: c5821a1a-b6e7-44bb-b75d-8577bedf015e
  spec:
    parameters:
      clusterID: 81fd58def37a35d1b4d4bee7ba35c3c3
      flattenMode: force
      mirroringMode: snapshot
      replication.storage.openshift.io/replication-secret-name: ceph-client-provisioner-88db0eb5300a85c435ac0a0013075bee
      replication.storage.openshift.io/replication-secret-namespace: openshift-storage
      schedulingInterval: 2m
    provisioner: openshift-storage.rbd.csi.ceph.com
kind: List
metadata:
  resourceVersion: ""

@Madhu-1 could you verify that vrc has the right parameters/labels/annotations?

@Madhu-1
Copy link
Member

Madhu-1 commented Nov 13, 2024

@rewantsoni parameters looks good, am not sure about the annotations/labels

@@ -424,6 +469,44 @@ func (r *StorageClaimReconciler) reconcilePhases() (reconcile.Result, error) {
if err != nil {
return reconcile.Result{}, fmt.Errorf("failed to create or update VolumeSnapshotClass: %s", err)
}
case "VolumeReplicationClass":
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens when the drClusterConfig is deleted? Will the VRC's get deleted as well?

}
// check if DRClusterConfig has been removed
if len(drclusterconfigs.Items) == 0 {
vrcs := &replicationv1alpha1.VolumeReplicationClassList{}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just set the owneRef of VolumeReplicationClass as the drclusterconfig as both are clusterscoped resouces?

}
}
drclusterconfig := &drclusterconfigs.Items[0]
for interval := range drclusterconfig.Spec.ReplicationSchedules {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what will happen when specific ReplicationSchedules are deleted?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missed that, added ✔️

if err != nil {
return reconcile.Result{}, fmt.Errorf("failed to get VolumeReplicationClassList: %v", err)
}
drclusterconfig := &drclusterconfigs.Items[0]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will fail if the len(drclusterconfigs.Items) is zero

@@ -141,8 +192,9 @@ func (r *StorageClaimReconciler) SetupWithManager(mgr ctrl.Manager) error {
//+kubebuilder:rbac:groups=snapshot.storage.k8s.io,resources=volumesnapshotclasses,verbs=get;list;watch;create;delete
//+kubebuilder:rbac:groups=core,resources=persistentvolumes,verbs=get;list;watch
//+kubebuilder:rbac:groups=snapshot.storage.k8s.io,resources=volumesnapshotcontents,verbs=get;list;watch
//+kubebuilder:rbac:groups=ramendr.openshift.io,resources=drclusterconfigs,verbs=get;list;watch
//+kubebuilder:rbac:groups=replication.storage.openshift.io,resources=volumereplicationclass,verbs=get;list;watch;create;delete
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you run make bundle after updating the rbac?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

Comment on lines 497 to 501
if len(drclusterconfigs.Items) == 0 {
return reconcile.Result{}, fmt.Errorf("no DRClusterConfig found")
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not an error, it means that we don't need to create VRC.

go.mod Show resolved Hide resolved
internal/controller/storageclaim_controller.go Outdated Show resolved Hide resolved
enqueueStorageClaim := handler.EnqueueRequestsFromMapFunc(
func(_ context.Context, _ client.Object) []reconcile.Request {
sclaims := &v1alpha1.StorageClaimList{}
err := r.list(sclaims)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pls use the supplied context r.List(ctx, ...) as the ctx on reconciler may be empty at this point and when cancelled this may panic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean use r.Client.List(). This won't be consistent with the rest of the code that does use the aux wrapper functions.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Client is embedded inside reconciler and all methods on that will be available to reconciler, anyways I don't have a problem w/ how that'll be referred.

I'm not taking about consistency but of correctness, you can see the wrapper functions are used inside Reconcile and at the start of it r.ctx will be defined but in here that isn't the case.

Comment on lines 496 to 502
err = r.list(drclusterconfigs)
if err != nil {
return reconcile.Result{}, fmt.Errorf("failed to get DRClusterConfigList: %v", err)
}
if len(drclusterconfigs.Items) == 0 {
continue
}
if len(drclusterconfigs.Items) > 1 {
return reconcile.Result{}, fmt.Errorf("there's more than one DRClusterConfig attached to the cluster")
}
Copy link
Contributor

@leelavg leelavg Nov 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
err = r.list(drclusterconfigs)
if err != nil {
return reconcile.Result{}, fmt.Errorf("failed to get DRClusterConfigList: %v", err)
}
if len(drclusterconfigs.Items) == 0 {
continue
}
if len(drclusterconfigs.Items) > 1 {
return reconcile.Result{}, fmt.Errorf("there's more than one DRClusterConfig attached to the cluster")
}
if err := r.list(drclusterconfigs) && err != nil {
return reconcile.Result{}, fmt.Errorf("failed to get DRClusterConfigList: %v", err)
} else if len(drclusterconfigs.Items) == 0 {
continue
} else if len(drclusterconfigs.Items) > 1 {
return reconcile.Result{}, fmt.Errorf("there's more than one DRClusterConfig attached to the cluster")
}

these conditions seems to be all inclusive and could be grouped.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Surely you meant: (can't use else keyword here at all)

				if err := r.list(drclusterconfigs); err != nil {
					return reconcile.Result{}, fmt.Errorf("failed to get DRClusterConfigList: %v", err)
				}
				if len(drclusterconfigs.Items) == 0 {
					continue
				}
				if len(drclusterconfigs.Items) > 1 {
					return reconcile.Result{}, fmt.Errorf("there's more than one DRClusterConfig attached to the cluster")
				}

Copy link
Contributor

@leelavg leelavg Nov 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant what I suggested, didn't get what you mean by else part though. I mentioned else-if as the conditions are all inclusive, ex: when err is nil items len can't be 0 and > 1 at the same time.

internal/controller/storageclaim_controller.go Outdated Show resolved Hide resolved
internal/controller/storageclaim_controller.go Outdated Show resolved Hide resolved
internal/controller/storageclaim_controller.go Outdated Show resolved Hide resolved
- Creates VolRepClass in response to changes made to DRClusterConfig
- k8s csi addons bump

Signed-off-by: raaizik <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants