configure_hypervisor: add rbd helpers for quadlet containers

[insatomcat]

Context

SEAPATH containers (Podman quadlets) that use Ceph RBD for persistent storage
must today embed a long sequence of ExecStartPre= and ExecStopPost= steps
directly in the quadlet file. A typical unit needs to:

1. Detect and clean up dirty state from a previous interrupted run
   (unmount stale mountpoints, unmap all stale RBD device mappings).
2. Create the RBD image on first use (rbd create).
3. Map the block device (rbd map).
4. Create the mountpoint directory if absent.
5. Detect and format the device on first use (mkfs.ext4).
6. Mount the device.
7. On stop: unmount all stacked mounts and unmap all device mappings.

This boilerplate is identical across every container that uses RBD, only the
image name changes. Writing it by hand is error-prone (especially the
dirty-state cleanup) and produces quadlet files that are hard to read and
maintain.

What this commit adds

Two bash scripts installed to /usr/local/bin/ by the configure_hypervisor role.
Both take only the image name; pool (rbd) and mountpoint (/mnt/rbd/) are
fixed by convention.

seapath-rbd-mount []

Idempotent mount script.  Handles the full lifecycle:

- Dirty-state cleanup: loops umount until the mountpoint is fully clear
  (interrupted previous runs can stack multiple mounts at the same path),
  then unmaps every device node still mapped to the image.

- First-use image creation: calls rbd create with --image-feature layering
  and the caller-supplied size (default 1G) only if rbd info reports the
  image does not exist.

- Device mapping: rbd map creates both /dev/rbdN and the stable symlink
  /dev/rbd/rbd/<image>.

- First-use filesystem: blkid is used to detect whether a filesystem
  already exists.  If not (freshly created image), mkfs.ext4 -q formats it
  silently.  Subsequent runs skip this step.

- Mount.

seapath-rbd-unmount

Best-effort unmount/unmap script for ExecStopPost=.  Loops umount until the
mountpoint is fully clear (handles stacked mounts from repeated interrupted
starts), then force-unmaps (-o force) all device nodes still mapped to the
image.  Force-unmapping handles devices held by udevd, blkid, or other
kernel references after a container crash.  Exits 0 regardless.

Enumerating the device mappings

Both scripts need the list of kernel mappings for an image, because the stable
symlink /dev/rbd// points only to the most recent mapping: earlier
stale mappings from interrupted runs accumulate as /dev/rbdN and can only be
found by enumerating the device list.

That enumeration is done with rbd device list --format json filtered by jq, in
a mapped_devices() helper that is deliberately identical in both scripts:

rbd device list --format json | jq -r --arg pool ... --arg image ... '
    .[]
    | select(.pool == $pool and .namespace == ""
             and .image == $image and .snap == "-")
    | .device
'

Selecting on the structured output rather than on column positions matters for
correctness, not only for readability. All four identity fields are matched:

• pool and namespace, so a same-named image in another pool or namespace is
  never unmapped from under its user.

• snap == "-", which selects read-write mappings, the only kind these helpers
  ever create. A snapshot mapping of the same image was made by something
  else (an admin, a backup job) and must not be force-unmapped.

jq is already part of the expected package set on SEAPATH images and is already
used to parse Ceph JSON output elsewhere in the tree (deploy_cephfs
wait-for-mds.sh), so it introduces no new dependency. seapath-rbd-mount still
checks for it explicitly and fails with a clear message if it is missing,
rather than silently enumerating zero devices and skipping the cleanup.

Usage in a quadlet

The ExecStartPre / ExecStopPost block in the quadlet file is reduced to:

ExecStartPre=/usr/local/bin/seapath-rbd-mount mycontainer
ExecStopPost=/usr/local/bin/seapath-rbd-unmount mycontainer

Any app-specific subdirectories that the container needs inside the mountpoint
are the caller's responsibility (mkdir -p in a separate ExecStartPre= line),
they are inherently application-specific and out of scope for a generic helper.

Deployment

The two scripts are installed by a new rbd_helpers.yml task file included from
configure_hypervisor/tasks/main.yml, following the same pattern as the other
helpers in that role (ansible.builtin.copy, mode 0755).

The task is skipped on Yocto (seapath_distro != "Yocto"): the root filesystem
is read-only there, so the scripts must be shipped by the image itself.
Porting them to meta-seapath is required before quadlets relying on these
helpers can be used on Yocto.

[eroussy]

Overall scripts and philosophy looks good to me.

Again, I would still advise for python instead of bash for readability

[insatomcat]

I gave the Python rewrite a serious look before answering. I still lean toward bash, but I looked into the readability point: the part that genuinely deserved the criticism is gone now.

What changed

The most unreadable bit was the awk that enumerated the device mappings:

NR>1 && $(NF-1)=="-" && $(NF-2)==img {print $NF}

Positional indexing from the end, because the namespace column disappears when it is empty and awk collapses the whitespace. It is replaced by a named, documented helper, deliberately identical in both scripts:

mapped_devices() {
    rbd device list --format json 2> /dev/null | jq -r \
        --arg pool "$POOL" --arg image "$IMAGE" '
            .[]
            | select(.pool == $pool and .namespace == ""
                     and .image == $image and .snap == "-")
            | .device
        '
}

Call sites are now just done < <(mapped_devices), so the cleanup loops read on their own.

Selecting on structured output also fixed two real bugs, so thanks for pushing on this:

• the old awk did not filter on pool, so a same-named image in another pool was unmapped from under its user
• the unmount did not filter on snap either, so a snapshot mapping made by a backup job or an admin was force-unmapped

jq is not a new dependency: it is already in the expected package set for our images, and deploy_cephfs/files/wait-for-mds.sh already parses Ceph JSON output with it. seapath-rbd-mount still checks for it explicitly and fails with a clear message, rather than silently enumerating zero devices and skipping the cleanup.

Why I did not go to Python

I went through it operation by operation. Only 3 of the 9 operations gain a real API: rbd info and rbd create via librbd, and findmnt via os.path.ismount(). The other 6 stay subprocess calls, and one of them is structural: krbd is not exposed by the librbd Python bindings. Kernel mapping only happens through the rbd map CLI or by writing to /sys/bus/rbd/add. The evidence is in our own tree: vm_manager/helpers/rbd_manager.py has 47 methods on the rados/rbd bindings and not one map/unmap, which is not an oversight.

On top of that, set -euo pipefail gives error propagation for free. In Python it becomes subprocess.run(..., check=True) plus CalledProcessError handling at every step, so the Python version comes out longer than the bash one, not shorter.

Two practical points as well:

• these run in quadlet ExecStartPre=. bash starts in ~3 ms, while python3 plus import rbd pulls in librados and opens a cluster connection, which is hundreds of ms on the container start path.
• the commit already notes that Yocto needs a meta-seapath port. Adding python3-rbd (so librados) to a hardened minimal image is real footprint and CVE surface for 150 lines of glue. No role installs python3-rbd on the hypervisors today either, only vm_manager's README mentions it as a manual install.

For a hundred lines of sequential "glue" around 6 external commands I really think bash is the right call here.