Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Link PVCs and PVs in VolumeGroupSnapshot and VolumeGroupSnapshotContent #1069

Merged
merged 1 commit into from
May 17, 2024

Conversation

leonardoce
Copy link
Contributor

@leonardoce leonardoce commented Apr 18, 2024

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespaces from that line:

/kind api-change
/kind bug
/kind cleanup
/kind design
/kind documentation
/kind failing-test
/kind feature
/kind flake

What this PR does / why we need it:

Following #1068 and the discussion in #969, this PR improves the snapshot controller and the CSI snapshotter sidecar to track which PVC and PVs are snapshotted by a VolumeGroupSnapshot.

With this PR applied, a VolumeGroupSnapshot will look like:

apiVersion: groupsnapshot.storage.k8s.io/v1alpha1
kind: VolumeGroupSnapshot
metadata:
  finalizers:
  - groupsnapshot.storage.kubernetes.io/volumegroupsnapshot-bound-protection
  generation: 1
  name: new-groupsnapshot
  namespace: default
  resourceVersion: "64488"
  uid: 02f28293-ff7e-4d1c-8ea8-c64fb2b45a26
spec:
  source:
    selector:
      matchLabels:
        cnpg.io/instanceName: cluster-example-1
  volumeGroupSnapshotClassName: csi-hostpath-groupsnapclass
status:
  boundVolumeGroupSnapshotContentName: groupsnapcontent-02f28293-ff7e-4d1c-8ea8-c64fb2b45a26
  creationTime: "2024-05-14T10:02:50Z"
  pvcVolumeSnapshotRefList:
  - persistentVolumeClaimRef:
      name: cluster-example-1
    volumeSnapshotRef:
      name: snapshot-219cbb76259445b14e5a1f0e11008cb976aacfb23bada2e4346d7d4da81d2d2b-2024-05-14-10.2.52
  - persistentVolumeClaimRef:
      name: cluster-example-1-wal
    volumeSnapshotRef:
      name: snapshot-8b65beb9db8573e67e31c3120401d13f1ccd83bf4974b546970799ce0b0016c3-2024-05-14-10.2.52
  readyToUse: true

The corresponding VolumeGroupSnapshotContent will look like:

apiVersion: groupsnapshot.storage.k8s.io/v1alpha1
kind: VolumeGroupSnapshotContent
metadata:
  creationTimestamp: "2024-05-14T10:02:50Z"
  finalizers:
  - groupsnapshot.storage.kubernetes.io/volumegroupsnapshotcontent-bound-protection
  generation: 1
  name: groupsnapcontent-02f28293-ff7e-4d1c-8ea8-c64fb2b45a26
  resourceVersion: "64482"
  uid: 7671dd61-0af3-4e53-a389-d7528fca5260
spec:
  deletionPolicy: Delete
  driver: hostpath.csi.k8s.io
  source:
    volumeHandles:
    - f4416324-1165-11ef-91b4-22776546f286
    - f440f487-1165-11ef-91b4-22776546f286
  volumeGroupSnapshotClassName: csi-hostpath-groupsnapclass
  volumeGroupSnapshotRef:
    apiVersion: groupsnapshot.storage.k8s.io/v1alpha1
    kind: VolumeGroupSnapshot
    name: new-groupsnapshot
    namespace: default
    resourceVersion: "64440"
    uid: 02f28293-ff7e-4d1c-8ea8-c64fb2b45a26
status:
  creationTime: 1715680970153290824
  pvVolumeSnapshotContentList:
  - persistentVolumeRef:
      name: pvc-e92d8114-160f-4a59-8c29-bb6fb42e345f
    volumeSnapshotContentRef:
      name: snapcontent-219cbb76259445b14e5a1f0e11008cb976aacfb23bada2e4346d7d4da81d2d2b-2024-05-14-10.2.52
  - persistentVolumeRef:
      name: pvc-7b7b185e-2c35-4973-959e-15593d5f74a5
    volumeSnapshotContentRef:
      name: snapcontent-8b65beb9db8573e67e31c3120401d13f1ccd83bf4974b546970799ce0b0016c3-2024-05-14-10.2.52
  readyToUse: true
  volumeGroupSnapshotHandle: 1f3e6247-11d9-11ef-a7bc-b25da50ff257

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Testing

To test this patch, I used the CSI host path driver.
Having two PVCs and creating a VolumeGroupSnapshot matching them is enough to trigger the process.

Does this PR introduce a user-facing change?:

Link the snapshotted PVCs and the corresponding PVs in VolumeGroupSnapshot and VolumeGroupSnapshotContent to make restoring data easier.

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/feature Categorizes issue or PR as related to a new feature. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Apr 18, 2024
@k8s-ci-robot
Copy link
Contributor

Hi @leonardoce. Thanks for your PR.

I'm waiting for a kubernetes-csi member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Apr 18, 2024
@xing-yang
Copy link
Collaborator

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Apr 18, 2024
@xing-yang
Copy link
Collaborator

/assign

@nixpanic
Copy link
Contributor

Once successfully created, maybe remove the groupsnapshot.storage.kubernetes.io/info annotation?

@leonardoce
Copy link
Contributor Author

Once successfully created, maybe remove the groupsnapshot.storage.kubernetes.io/info annotation?

Yes. Nice idea, @nixpanic.

All the information in that annotation is in the status section of both VolumeGroupSnapshot and VolumeGroupSnapshotStatus. But this would require a new API server call.

More importantly, updateGroupSnapshotContentStatus and updateGroupSnapshotStatus won't be idempotent anymore as the annotation will be missing after removal. I need more experience in this codebase to judge this properly.

What's your opinion @xing-yang?

@leonardoce
Copy link
Contributor Author

Thinking aloud, the VolumeGroupSnapshotContent resource would be more beautiful without that annotation...

@xing-yang
Copy link
Collaborator

/hold

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 19, 2024
@xing-yang
Copy link
Collaborator

I'd prefer the annotation gets removed.

@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels May 14, 2024
@leonardoce
Copy link
Contributor Author

I rebased it on the new master, removed the annotation, and updated the description of this PR with the latest resource definitions.

@xing-yang
Copy link
Collaborator

xing-yang commented May 15, 2024

Can you update the PR description? It still talks about the groupsnapshot.storage.kubernetes.io/info annotation. Can you also update the "Special notes for your reviewer" section?

@xing-yang
Copy link
Collaborator

I see that the annotation is still in the code. Can you please squash the commits? It will be easier to see what code is removed this way.

pkg/utils/util.go Outdated Show resolved Hide resolved
pkg/common-controller/groupsnapshot_controller_helper.go Outdated Show resolved Hide resolved
pkg/common-controller/groupsnapshot_controller_helper.go Outdated Show resolved Hide resolved
pkg/common-controller/groupsnapshot_controller_helper.go Outdated Show resolved Hide resolved
@leonardoce
Copy link
Contributor Author

You're correct, @xing-yang.
The annotation is still used; it is only removed when the VolumeGroupSnapshot is marked ready.
By having that annotation, the patch avoids hitting the API server.

I've another implementation that doesn't use annotation and queries the API server when needed.
Do you prefer that one?

@xing-yang
Copy link
Collaborator

You're correct, @xing-yang.
The annotation is still used; it is only removed when the VolumeGroupSnapshot is marked ready.
By having that annotation, the patch avoids hitting the API server.

I've another implementation that doesn't use annotation and queries the API server when needed.
Do you prefer that one?

Yes, I'd prefer not to use the annotation. Thanks.

@leonardoce
Copy link
Contributor Author

Done @xing-yang. Thank you.

pkg/utils/pvs.go Outdated Show resolved Hide resolved
pkg/utils/pvs_test.go Outdated Show resolved Hide resolved
pkg/common-controller/groupsnapshot_controller_helper.go Outdated Show resolved Hide resolved
pkg/common-controller/groupsnapshot_controller_helper.go Outdated Show resolved Hide resolved
Name: volumeSnapshotContent.Spec.VolumeSnapshotRef.Name,
},
PersistentVolumeClaimRef: v1.LocalObjectReference{
Name: pvcName,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If pv.Spec.ClaimRef is nil above, this will be an empty string?

Copy link
Contributor Author

@leonardoce leonardoce May 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it will. And this is equivalent to an empty LocalObjectReference.
And since the VolumeSnapshotRef itself is marked as omitempty, it will be omitted.

But I realized now that this is not evident from the code. Let me express it more clearly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it better now?

pkg/sidecar-controller/groupsnapshot_helper.go Outdated Show resolved Hide resolved
pkg/sidecar-controller/groupsnapshot_helper.go Outdated Show resolved Hide resolved
This use the update API to set `persistentVolumeClaimRef` in
`VolumeGroupSnapshot` and `persistentVolumeName` in
`VolumeGroupSnapshotContent` to the corresponding objects.

This makes restoring volumes from a VolumeGroupSnapshot easier.

Related: kubernetes-csi#969
@xing-yang
Copy link
Collaborator

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 17, 2024
@xing-yang
Copy link
Collaborator

/hold cancel

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 17, 2024
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: leonardoce, sxd, xing-yang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 17, 2024
@k8s-ci-robot k8s-ci-robot merged commit 52b72d3 into kubernetes-csi:master May 17, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants