Remove VolumeSnapshot Name annotation from PVC during the CSI Restore operation

[shubham-pampattiwar]

What steps did you take and what happened:

Restore fails in the following two scenarios:

• Backup with CSI for an application and then later perform restic backup of the same application
• Backup with CSI and then Restore with CSI, if VS Label remains on PVC then if a restic backup is executed

What did you expect to happen:
Restore to be successful

The following information will help us better understand what's going on:

If you are using velero v1.7.0+:
Please use velero debug --backup <backupname> --restore <restorename> to generate the support bundle, and attach to this issue, more options please refer to velero debug --help

If you are using earlier versions:
Please provide the output of the following commands (Pasting long output into a GitHub gist or other pastebin is fine.)

• kubectl logs deployment/velero -n velero
• velero backup describe <backupname> or kubectl get backup/<backupname> -n velero -o yaml
• velero backup logs <backupname>
• velero restore describe <restorename> or kubectl get restore/<restorename> -n velero -o yaml
• velero restore logs <restorename>

Anything else you would like to add:

Environment:

• Velero version (use velero version):
• Velero features (use velero client config get features):
• Kubernetes version (use kubectl version):
• Kubernetes installer & version:
• Cloud provider or hardware configuration:
• OS (e.g. from /etc/os-release):

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.

• 👍 for "I would like to see this bug fixed as soon as possible"
• 👎 for "There are more important bugs to focus on right now"

[shubham-pampattiwar]

cc @weshayutin @sseago

[weshayutin]

ah.. very nice sir @shubham-pampattiwar

[Lyndon-Li]

@shubham-pampattiwar
Thanks for noticing and fixing this problem.

For the aforementioned cases, I understand why the second case has this problem, but I don't understand what happens in the first case:

Restore fails in the following two scenarios:

Backup with CSI for an application and then later perform restic backup of the same application
Backup with CSI and then Restore with CSI, if VS Label remains on PVC then if a restic backup is executed

As far as my understanding, since this label is not added to the PVC in users' workload(only to the PVC item in the backup tarball), the following restic backup should not see this label, so the restore from the restic backup should have no problem.

Any things I've missed?

[shubham-pampattiwar]

@Lyndon-Li for case 1 if you have the csi plugin enabled as well then wont the restore part of the csi plugin act on the volume because of the presence of the annotation ?

[blackpiglet]

During restoration, the object is retrieved from the backup-created metadata.
So the annotation in tarball also affects the result, is my understanding right?

[Lyndon-Li]

@shubham-pampattiwar @blackpiglet
I think Restic backup/restore and CSI backup/restore won't process the same PVC in one backup even though they are both installed. Anything I miss?
By this I mean, the Restic backup/restore and the CSI backup/restore are inter-exclusive, once a PVC is backed up/restored by one method, the other method won't process it. Otherwise, if any, it should be treated as a bug because users don't want to have two copies of data in the same backup.

[shubham-pampattiwar]

@blackpiglet Bingo, Exactly !