Skip to content

feat(cozystack): add vm-tenant-move skill - #28

Open
Andrei Kvapil (kvaps) wants to merge 4 commits into
mainfrom
feat/vm-tenant-move
Open

Andrei Kvapil (kvaps) wants to merge 4 commits into
mainfrom
feat/vm-tenant-move

Conversation

@kvaps

Copy link
Copy Markdown
Member

Adds cozystack:vm-tenant-move. It moves a VM (VMInstance and its VMDisks) to another tenant namespace without copying disk data: the PV is re-bound to a new PVC in the target namespace, and the DataVolume is re-created over it.

CDI adopts the PVC through cdi.kubevirt.io/storage.populatedFor, so there is no need to touch the CDI webhooks or feature gates. allowClaimAdoption on a PVC does nothing in CDI v1.66.1 (CDI reads it from the DataVolume only), that's why migration 29 had to disable the webhooks. IP and MAC are kept by moving the ip.kubeovn.io record, the same way migration 29 does it.

Checked against the code (cozystack main, CDI v1.66.1, KubeVirt v1.9.0), but not run on a live cluster yet, so keeping it as a draft.

Moving a VM between tenants has no documented procedure, and copying
large disks through backups is slow. Add a skill that moves a
VMInstance and its VMDisks without copying data: the PersistentVolume
is re-bound to a PVC in the target namespace and CDI adopts it through
the storage.populatedFor annotation, which works without disabling
CDI webhooks or enabling claim adoption. The kube-ovn IP record is
moved the way platform migration 29 does it, so the VM keeps its IP
and MAC. Backup restore into another namespace is kept as the
fallback.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 68568c52-f004-4a63-be0a-4b41a45f8300
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@kvaps
Andrei Kvapil (kvaps) marked this pull request as ready for review November 20, 2026 09:59

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NOT LGTM. The transfer and recovery paths need changes.

## Phase 4 — Delete the VM and disks in the source tenant

```bash
kubectl --context "$CTX" -n "$SRC" delete vminstance "$VM"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Reject persistent-EFI VMs before deleting the source. firmware.efiPersistent: true is supported by the vm-instance chart. KubeVirt keeps its NVRAM in a separate PVC owned by the VM, outside spec.disks. This deletion garbage-collects that PVC, whose PV was never switched to Retain. With a Delete reclaim policy, persistent NVRAM state is lost. Add a preflight refusal for this configuration unless its backend storage is also protected and transferred.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in d66f499. I went with moving the NVRAM PVC instead of refusing: its PV gets Retain together with the disks and is re-bound in the target under the same persistent-state-for label, so KubeVirt picks it up. Not tested on a live cluster with efiPersistent yet.


```bash
kubectl --context "$CTX" apply -f "$WORK/dst-vmdisk-$d.yaml"
kubectl --context "$CTX" -n "$DST" wait dv "vm-disk-$d" --for=jsonpath='{.status.phase}'=Succeeded --timeout=5m

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Handle deferred binding before this wait. The preceding instruction copies bindImmediately from the source, so a supported false value on a WaitForFirstConsumer class creates a blank DV with no consumer. The shipped CDI enables HonorWaitForFirstConsumer and does not start population without a selected node or immediate-binding annotation. No target VM exists yet, so the DV cannot reach Succeeded and this wait times out. Add a preparation path that does not require the deferred blank volume to complete, or reject this variant in preflight.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 972259a. The target VMDisk is now always prepared with bindImmediately: true, the moved PV is already provisioned anyway. Phase 9 sets it back to the source value.

While the PVs are `Retain`, no phase loses data. Only deleting a PV, or deleting a PVC while its PV is `Delete`, is irreversible.

- **Before Phase 4**: restore the source `runStrategy` and PV policies, delete the prepared VMDisks in `$DST`.
- **After Phase 4**: run the same procedure in reverse with `SRC` and `DST` swapped. Back in the source namespace the VM gets its original firmware UUID (the formula is deterministic), so cloud-init does not run a second time.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Define rollback for the states before the target VM exists. If Phase 5 fails after Phase 4, the original VMInstance is already deleted and the target is not created until Phase 7. Swapping SRC/DST makes Phase 1 query a nonexistent VMInstance, so it cannot obtain DISKS. A failure halfway through the disk loop also leaves some PVs bound to the target and others still released against the source. Add recovery steps based on the saved source manifests and pv-map for these partial states. Reserve the reverse-procedure shortcut for a completed destination.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 1dc7926, added a recovery path into SRC from the saved manifests and pv-map for the states before Phase 7. The reverse procedure is kept only for a completed target. BTW Flux skips helm uninstall when a suspended HelmRelease is deleted, so prepared VMDisks are now resumed before deletion.

A VM with firmware.efiPersistent keeps its EFI NVRAM in a separate
PVC owned by the VirtualMachine and outside spec.disks. Deleting the
source VMInstance garbage-collected that PVC while its PV kept the
class reclaim policy, so the NVRAM state was lost.

Record the backend-storage PVC in the preflight, protect its PV with
Retain together with the disks, bind it to a PVC carrying the
persistent-state-for label in the target, and give that PVC the new
VM as owner.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Copying bindImmediately: false from the source left the blank target
DataVolume without a consumer on a WaitForFirstConsumer class. CDI
with HonorWaitForFirstConsumer never populates such a volume before a
node is selected, so the preparation wait timed out.

Always prepare the target VMDisk with immediate binding: the moved PV
is already provisioned, so the flag no longer affects it. Restore the
source value at the end of the procedure.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Swapping SRC and DST only works once the target VMInstance exists:
the reverse Phase 1 reads the disk list from it. A failure between
Phase 4 and Phase 7, or half way through the disk loop, left PVs split
between the source and the target with no documented way back.

Add a recovery path built from the saved source manifests and the PV
records: release whatever the target claimed, classify every PV,
re-bind the released ones into the source namespace and rebuild the
VMInstance there. Keep the reverse procedure for a completed target
and give it its own work directory so the original PV records
survive. Resume prepared HelmReleases before deleting their VMDisks,
as Flux skips the uninstall of a suspended release.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants