Repository navigation
feat(cozystack): add vm-tenant-move skill - #28
Andrei Kvapil (kvaps) wants to merge 4 commits into
Conversation
Moving a VM between tenants has no documented procedure, and copying large disks through backups is slow. Add a skill that moves a VMInstance and its VMDisks without copying data: the PersistentVolume is re-bound to a PVC in the target namespace and CDI adopts it through the storage.populatedFor annotation, which works without disabling CDI webhooks or enabling claim adoption. The kube-ovn IP record is moved the way platform migration 29 does it, so the VM keeps its IP and MAC. Backup restore into another namespace is kept as the fallback. Assisted-by: LLM Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
|
Aleksei Sviridkin (lexfrei)
left a comment
There was a problem hiding this comment.
NOT LGTM. The transfer and recovery paths need changes.
| ## Phase 4 — Delete the VM and disks in the source tenant | ||
|
|
||
| ```bash | ||
| kubectl --context "$CTX" -n "$SRC" delete vminstance "$VM" |
There was a problem hiding this comment.
[P1] Reject persistent-EFI VMs before deleting the source. firmware.efiPersistent: true is supported by the vm-instance chart. KubeVirt keeps its NVRAM in a separate PVC owned by the VM, outside spec.disks. This deletion garbage-collects that PVC, whose PV was never switched to Retain. With a Delete reclaim policy, persistent NVRAM state is lost. Add a preflight refusal for this configuration unless its backend storage is also protected and transferred.
There was a problem hiding this comment.
Fixed in d66f499. I went with moving the NVRAM PVC instead of refusing: its PV gets Retain together with the disks and is re-bound in the target under the same persistent-state-for label, so KubeVirt picks it up. Not tested on a live cluster with efiPersistent yet.
|
|
||
| ```bash | ||
| kubectl --context "$CTX" apply -f "$WORK/dst-vmdisk-$d.yaml" | ||
| kubectl --context "$CTX" -n "$DST" wait dv "vm-disk-$d" --for=jsonpath='{.status.phase}'=Succeeded --timeout=5m |
There was a problem hiding this comment.
[P2] Handle deferred binding before this wait. The preceding instruction copies bindImmediately from the source, so a supported false value on a WaitForFirstConsumer class creates a blank DV with no consumer. The shipped CDI enables HonorWaitForFirstConsumer and does not start population without a selected node or immediate-binding annotation. No target VM exists yet, so the DV cannot reach Succeeded and this wait times out. Add a preparation path that does not require the deferred blank volume to complete, or reject this variant in preflight.
There was a problem hiding this comment.
Fixed in 972259a. The target VMDisk is now always prepared with bindImmediately: true, the moved PV is already provisioned anyway. Phase 9 sets it back to the source value.
| While the PVs are `Retain`, no phase loses data. Only deleting a PV, or deleting a PVC while its PV is `Delete`, is irreversible. | ||
|
|
||
| - **Before Phase 4**: restore the source `runStrategy` and PV policies, delete the prepared VMDisks in `$DST`. | ||
| - **After Phase 4**: run the same procedure in reverse with `SRC` and `DST` swapped. Back in the source namespace the VM gets its original firmware UUID (the formula is deterministic), so cloud-init does not run a second time. |
There was a problem hiding this comment.
[P2] Define rollback for the states before the target VM exists. If Phase 5 fails after Phase 4, the original VMInstance is already deleted and the target is not created until Phase 7. Swapping SRC/DST makes Phase 1 query a nonexistent VMInstance, so it cannot obtain DISKS. A failure halfway through the disk loop also leaves some PVs bound to the target and others still released against the source. Add recovery steps based on the saved source manifests and pv-map for these partial states. Reserve the reverse-procedure shortcut for a completed destination.
There was a problem hiding this comment.
Fixed in 1dc7926, added a recovery path into SRC from the saved manifests and pv-map for the states before Phase 7. The reverse procedure is kept only for a completed target. BTW Flux skips helm uninstall when a suspended HelmRelease is deleted, so prepared VMDisks are now resumed before deletion.
A VM with firmware.efiPersistent keeps its EFI NVRAM in a separate PVC owned by the VirtualMachine and outside spec.disks. Deleting the source VMInstance garbage-collected that PVC while its PV kept the class reclaim policy, so the NVRAM state was lost. Record the backend-storage PVC in the preflight, protect its PV with Retain together with the disks, bind it to a PVC carrying the persistent-state-for label in the target, and give that PVC the new VM as owner. Assisted-by: LLM Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Copying bindImmediately: false from the source left the blank target DataVolume without a consumer on a WaitForFirstConsumer class. CDI with HonorWaitForFirstConsumer never populates such a volume before a node is selected, so the preparation wait timed out. Always prepare the target VMDisk with immediate binding: the moved PV is already provisioned, so the flag no longer affects it. Restore the source value at the end of the procedure. Assisted-by: LLM Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Swapping SRC and DST only works once the target VMInstance exists: the reverse Phase 1 reads the disk list from it. A failure between Phase 4 and Phase 7, or half way through the disk loop, left PVs split between the source and the target with no documented way back. Add a recovery path built from the saved source manifests and the PV records: release whatever the target claimed, classify every PV, re-bind the released ones into the source namespace and rebuild the VMInstance there. Keep the reverse procedure for a completed target and give it its own work directory so the original PV records survive. Resume prepared HelmReleases before deleting their VMDisks, as Flux skips the uninstall of a suspended release. Assisted-by: LLM Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Adds
cozystack:vm-tenant-move. It moves a VM (VMInstance and its VMDisks) to another tenant namespace without copying disk data: the PV is re-bound to a new PVC in the target namespace, and the DataVolume is re-created over it.CDI adopts the PVC through
cdi.kubevirt.io/storage.populatedFor, so there is no need to touch the CDI webhooks or feature gates.allowClaimAdoptionon a PVC does nothing in CDI v1.66.1 (CDI reads it from the DataVolume only), that's why migration 29 had to disable the webhooks. IP and MAC are kept by moving theip.kubeovn.iorecord, the same way migration 29 does it.Checked against the code (cozystack main, CDI v1.66.1, KubeVirt v1.9.0), but not run on a live cluster yet, so keeping it as a draft.