Correcting OpenShift on IBM Documentation Error #8077
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
I have to admit to some significant error and embarrassment regarding the documentation update about Openshift on IBM Cloud pull request vmware-tanzu#8069. I will correct my error before it gets any further. Just exposing /var/data/kubelet/pods is incorrect and host path /var/lib/kubelet/pods should remain unchanged. The errors with the defaults during csi snapshot data movement were: data path backup failed: Failed to run kopia backup: unable to get local block device entry: resolveSymlink: lstat /var/data/: no such file or directory I suspected this was the same as RancherOS and Nutanix. It is not. The original tested changes changed both /var/lib/kubelet/{pods,plugins} to /var/data/kubelet/{pods,plugins}. The published changes only result in the error: ``` status: completionTimestamp: '2024-08-02T17:12:29Z' message: >- data path backup failed: Failed to run kopia backup: unable to get local block device entry: resolveSymlink: lstat /var/data/kubelet/plugins: no such file or directory node: 10.240.0.5 phase: Failed progress: {} startTimestamp: '2024-08-02T17:12:11Z' ``` After making continued modifications to the daemonset the correct configuration was: ``` volumeMounts: - name: host-pods mountPath: /host_pods mountPropagation: HostToContainer - name: host-plugins mountPath: /var/data/kubelet/plugins mountPropagation: HostToContainer ``` ``` volumes: - name: host-pods hostPath: path: /var/lib/kubelet/pods type: '' - name: host-plugins hostPath: path: /var/data/kubelet/plugins type: '' ``` Only the changes to the plugin path were required. The plugin path changes were required to both the mount path and the host path. Regardless of whether /var/lib/kubelet/pods or /var/data/kubelet/pods host path, backups and restore succeeded provided the plugin path was modified. ``` volumeMounts: - name: host-pods mountPath: /host_pods mountPropagation: HostToContainer - name: host-plugins mountPath: /var/data/kubelet/plugins mountPropagation: HostToContainer ``` ``` volumes: - name: host-pods hostPath: path: /var/data/kubelet/pods type: '' - name: host-plugins hostPath: path: /var/data/kubelet/plugins type: '' ``` After getting on-host access was able to confirm. Pods are at /var/lib/kubelet/pods. ``` ls /var/lib/kubelet/pods 07c0be63-335d-4cfb-b39f-816bc2fb32cd 51f31b3e-4710-4ef0-8626-5f1a78a624b2 a4802fd3-3b62-45a4-8f21-974880b6f92a cccb35c9-b4f9-4ca9-a697-736ae64f09ad 0a5d4366-7fa1-4525-9e45-a43a362b8542 558b0643-0661-4d4a-b03e-aac60c6ad710 a4b106fb-5b7b-48e5-828a-ea7b41ba0e59 ce1290e1-4330-4df6-8166-14784bcce930 ``` On host the volumes are in /var/data/kubelet/plugins. ``` ls /var/data/kubelet/plugins/kubernetes.io/csi/openshift-storage.cephfs.csi.ceph.com/231e04896c4f528efb95d23a3c153db9fc4a7206b7320f74443f30de7228dba5/globalmount/velero/backups/backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c/ backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-csi-volumesnapshotclasses.json.gz backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-resource-list.json.gz backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-csi-volumesnapshotcontents.json.gz backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-results.gz backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-csi-volumesnapshots.json.gz backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-volumesnapshots.json.gz backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-itemoperations.json.gz backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c.tar.gz backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-logs.gz velero-backup.json backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-podvolumebackups.json.gz ``` With the volume config changed to expose /var/data/kubelet/plugins for the plugin hostPath, the DataUploads and DataDownloads succeed for both Filesystem and Block mode PVCs. ``` status: completionTimestamp: '2024-08-02T17:23:33Z' node: 10.240.0.5 path: >- /host_pods/7fcb9d56-7885-437c-acd3-67db6b1ee8ae/volumeDevices/kubernetes.io~csi/pvc-47b91f56-db8c-44bf-9ecc-737170561b4b phase: Completed progress: bytesDone: 5368709120 totalBytes: 5368709120 snapshotID: 8faae36b3592fee4efbfad024f26033e startTimestamp: '2024-08-02T17:21:22Z' ``` ``` status: completionTimestamp: '2024-08-02T18:42:19Z' node: 10.240.0.5 phase: Completed progress: bytesDone: 5368709120 totalBytes: 5368709120 startTimestamp: '2024-08-02T18:41:00Z' ``` My apologies for the error. Signed-off-by: Michael Fruchtman <[email protected]>
Signed-off-by: Michael Fruchtman <[email protected]>
|
/kind changelog-not-required |
github-actions
bot
added
the
kind/changelog-not-required
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #8077 +/- ##
=======================================
Coverage 58.80% 58.80%
=======================================
Files 358 358
Lines 30085 30085
=======================================
Hits 17692 17692
Misses 10945 10945
Partials 1448 1448 ☔ View full report in Codecov by Sentry. |
Lyndon-Li
approved these changes
Aug 5, 2024
mateusoliveira43
approved these changes
Aug 6, 2024
sseago
approved these changes
Aug 6, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Please add a summary of your change
Corrects the error of #8069
Sets the correct host path and mount path required for CSI data movement in Velero 1.13 and 1.14 for OpenShift on IBM Cloud.
Does your change fix a particular issue?
I have to admit to some significant error and embarrassment regarding the documentation update about OpenShift on IBM Cloud pull request #8069.
I will correct my error before it gets any further.
Just exposing /var/data/kubelet/pods is incorrect and host path /var/lib/kubelet/pods should remain unchanged.
The errors with the defaults during csi snapshot data movement were:
data path backup failed: Failed to run kopia backup: unable to get local
block device entry: resolveSymlink: lstat /var/data/: no such
file or directory
I suspected this was the same as RancherOS and Nutanix. It is not.
The original tested changes changed both /var/lib/kubelet/{pods,plugins} to /var/data/kubelet/{pods,plugins}.
The published changes only result in the error:
After making continued modifications to the daemonset the correct configuration was:
Only the changes to the plugin path were required. The plugin path changes were required to both the mount path and the host path.
Regardless of whether /var/lib/kubelet/pods or /var/data/kubelet/pods host path, backups and restore succeeded provided the plugin path was modified.
After getting on-host access was able to confirm. Pods are at /var/lib/kubelet/pods.
On host the volumes are in /var/data/kubelet/plugins.
With the volume config changed to expose /var/data/kubelet/plugins for the plugin hostPath, the DataUploads and DataDownloads succeed for both Filesystem and Block mode PVCs.
My apologies for the error.
Fixes #(issue)
#8069
Please indicate you've done the following:
/kind changelog-not-requiredas a comment on this pull request.site/content/docs/main.