Skip to content

Add support for using aarch64 images and macOS support - #5

Merged
abbra merged 5 commits into
freeipa:mainfrom
rjeffman:aarch64_macos_support
Nov 15, 2026
Merged

abbra merged 5 commits into
freeipa:mainfrom
rjeffman:aarch64_macos_support

Conversation

@rjeffman

@rjeffman rjeffman commented Nov 14, 2026 •

Copy link
Copy Markdown
Member

Runs freeipa-env's VM lane on aarch64 guests and on macOS (Apple Silicon) hosts, builds aarch64 channel images, and covers both with CI.

qemu backend: aarch64 guests, hvf on macOS, URL images (86ea604)

  • New vm.* keys: arch (default: host arch, arm64 normalized to aarch64, picks qemu-system-), accel (default auto: hvf on macOS, kvm with a writable /dev/kvm, else tcg; -cpu host under kvm/hvf, max under tcg), machine, cpu, firmware, dns.
  • aarch64 guests run -machine virt with UEFI firmware auto-discovered next to the qemu binary (brew layout) or in the usual distro paths; vm.firmware overrides. x86_64 argv is unchanged bit for bit (q35,accel=kvm -cpu host).
  • vm.image accepts http(s) URLs, downloaded once to ~/.cache/freeipa-ci/images/.- Containerdisk resolution is arch-aware: the member matching each VM's arch is what gets pulled (--platform with podman, --override-arch/--override-os with skopeo inside a container); atag without that member fails with a clear error. containers-storage: refs are no longer pulled at all.
  • macOS host support: pid lookup and the krun hub's VM-liveness check fall back to ps when /proc is absent, so terminate and vms work. vm.dns sets guest nameservers, bypassing the slirp forwarder (10.0.2.3) for networks whose host resolvers don't answer the guest.
  • qemu-smoke and nested-qemu-external now reference the multi-arch ghcr.io/freeipa/ci-freeipa-vm-images/fedora-45 — one preset serves both host architectures.
  • KrunBackend rejects the qemu-only keys (arch, accel, cpu, machine, firmware) like its other unsupported keys.

images: build aarch64 images with --distarch (d730c6c)

  • build.sh --distarch ARCH passes --platform linux/ARCH to every container-tool invocation (base/build images, SRPM rpmbuild container, full image); the full image is labeled org.freeipa-ci.arch=ARCH. Default stays empty (host platform).
  • publish.sh --arch lets a hand-published containerdisk carry an arm64 manifest; the tag scheme stays arch-less.

CI: unit matrix on arm, macOS unit job, aarch64 image build (5ec05ca)

  • unit matrix on ubuntu-24.04 + ubuntu-24.04-arm; tool-dependent tests skip via shutil.which, so one suite proves the control-node Python on both arches.
  • unit-macos runs the suite on macos-15 (pytest/PyYAML in a venv — the runner's Python is externally managed and Homebrew has no pyyaml formula).
  • arm-image builds the dist-45 channel image --distarch aarch64 on a native arm runner and checks architecture + label.
  • containerdisk-arm pulls the published fedora-45 tag on an arm runner through the arch-aware resolution and checks it is arm64 (with qemu-utils installed, since the resolved disk is validated with qemu-img).
  • No macOS/arm VM-smoke job (GitHub macOS runners can't use Hypervisor.framework; arm Linux runners have no nested virt) — the x86_64 qemu job stays the VM lane.

docs (12a35e2)

README: the new vm.* keys, arch-aware containerdisk resolution, and a "Running on macOS" section covering the VM lane (hvf + brew qemu/xorriso/podman) and the container lane (podman machine, --distarch aarch64).

images: --srpm build lane on a macOS podman machine (3d76d97)

Two applehv/virtiofs constraints fixed: build workdir must live under $HOME (the podman machine shares only the home dir), and the rpmbuild topdir must be container-local (virtiofs refuses cpio's mode-0200 temp files); RPMs are copied out afterwards. Linux keeps the mounted topdir. The request flow (SRPM → RPMs → nested qemu preset) completes locally on macOS.

@rjeffman
rjeffman force-pushed the aarch64_macos_support branch 2 times, most recently from 656a88a to 87e2240 Compare November 15, 2026 00:58
vm.arch (default: the host arch, arm64 normalized to aarch64) picks
qemu-system-<arch>. vm.accel (default: auto) selects hvf on macOS, kvm with
a writable /dev/kvm, else tcg; -cpu is host under kvm/hvf and max under tcg.
aarch64 guests run on the -machine virt with a UEFI firmware auto-discovered
next to the qemu binary (brew layout) or in the usual distro paths
(vm.firmware overrides); x86_64 keeps today's q35,accel=kvm -cpu host argv
bit for bit.

The port follows the backend's hook structure: _qemu_system_args() composes
the -machine/-accel/-cpu and firmware args _start() splices in, and
_needs_kvm() makes the /dev/kvm requirement accel-conditional. KrunBackend
overrides _needs_kvm() to True (libkrun is KVM-only) and rejects the
qemu-only keys (arch, accel, cpu, machine, firmware) in vm.vms entries like
its other unsupported keys.

vm.image can now be an http(s) URL: it is downloaded once to
~/.cache/freeipa-ci/images/<basename> (no checksum verification), in
_cfgs_with_images() next to the containerdisk resolution. That resolution
is arch-aware: the member of the arch each VM boots is what gets pulled
(--platform with podman, --override-arch/--override-os with skopeo inside
a container), and a tag without that member fails with a clear error
instead of handing the backend a disk of the wrong architecture. A
containers-storage: ref is not pulled at all anymore: the image is already
in the local store, and podman 6 refuses the transport anyway.

The pid lookup falls back to ps when /proc is absent (macOS): terminate
kills the VM and freeipa-env vms lists it, and the krun hub's VM-liveness
check falls back to ps the same way. vm.dns sets guest nameservers,
bypassing the slirp DNS forwarder (10.0.2.3) for networks whose host
resolvers do not answer the guest.

The smoke presets (qemu-smoke, nested-qemu-external) reference the
multi-arch containerdisk ghcr.io/freeipa/ci-freeipa-vm-images/fedora-45
and set vm.dns: one preset serves both host architectures, the aarch64
member coming with the multi-arch tags of ci-freeipa-vm-images. Unit
tests in test_vm_qemu_port.py; test_containerdisk.py drives a throwaway
local registry to prove the per-arch member selection and the
missing-member refusal. The port tests pin the accel they assert (hvf,
kvm) instead of trusting the host's auto pick: a KVM-less CI runner
auto-selects tcg. The krun hub test binds its AF_UNIX sockets under a
short temp dir, out of pytest's numbered per-test dirs, which overflow
darwin's 104-byte sun_path. On this machine (Apple Silicon), up of
qemu-smoke under hvf pulls the aarch64 member, boots it and installs the
distro FreeIPA packages.

Signed-off-by: Rafael Guterres Jeffman <rjeffman@redhat.com>
@rjeffman
rjeffman force-pushed the aarch64_macos_support branch from 87e2240 to 3d76d97 Compare November 15, 2026 02:28
@rjeffman
rjeffman marked this pull request as ready for review November 15, 2026 03:26
--distarch ARCH passes --platform linux/ARCH to every container-tool
invocation that builds or runs: the base and build images, the SRPM rpmbuild
container (the RPM arch follows the platform) and the full image. The full
image is labeled org.freeipa-ci.arch=ARCH. The default stays empty (build
for the host platform): the old x86_64 default was parsed but never used.

vm/publish.sh gets --arch ARCH, so a hand-published containerdisk can carry
an arm64 manifest. The tag scheme stays arch-less: aarch64 builds on a
shared host use a distinct --tag.

Verified on an arm64 podman machine: the native build and
--distarch aarch64 both produce linux/arm64 with the label set.

Signed-off-by: Rafael Guterres Jeffman <rjeffman@redhat.com>
The unit job runs on ubuntu-24.04 and ubuntu-24.04-arm; tool-dependent
tests skip via shutil.which, so the same suite proves the control-node
Python on both architectures. unit-macos installs pytest and PyYAML into
a venv (the runner's Python is externally managed, and Homebrew has no
pyyaml formula) and runs the suite on macos-15 (the qemu backend's ps
fallback, no /proc, no /dev/kvm). arm-image builds the dist-45 channel
image with --distarch aarch64 on a native arm runner and checks the
architecture and the org.freeipa-ci.arch label. containerdisk-arm pulls
the published fedora-45 tag on an arm runner through the arch-aware
resolution and checks the pulled image is arm64, with qemu-utils
installed because the resolved disk is validated with qemu-img: the
multi-arch consumption without a VM, which is all an arm runner can do.

No macOS or arm VM-smoke job: GitHub macOS runners cannot use
Hypervisor.framework and arm Linux runners have no nested virt; the x86_64
qemu job stays the VM-lane CI.

Signed-off-by: Rafael Guterres Jeffman <rjeffman@redhat.com>
The new qemu backend keys (arch, accel, machine, cpu, firmware, dns, URL
images) and the macOS host packages. The containerdisk section documents
the arch-aware resolution: the published tags are multi-arch manifests, one
preset ref serves both host architectures, and a tag without the member of
the VM's arch fails with a clear error. A Running on macOS section covers
both lanes: the VM lane under Apple hvf with qemu-smoke (brew's qemu,
xorriso and podman; the multi-arch tag yields its aarch64 member), and the
container lane in the podman machine (8192 MiB, workdir under $HOME
because the machine only shares the home directory, image baked with
--distarch aarch64).

Signed-off-by: Rafael Guterres Jeffman <rjeffman@redhat.com>
Two applehv/virtiofs constraints, both hit while running a local
request on Apple Silicon:

- The build work dir must live under $HOME: the podman machine shares
  only the home directory with its VM, so the /tmp topdir could not be
  mounted into the build container (statfs ... no such file or
  directory).

- The rpmbuild topdir must live inside the container: RPM's cpio
  creates its unpack temp files with mode 0200, which virtiofs refuses
  outright (open(O_CREAT, 0200) -> EACCES), so `rpmbuild --rebuild`
  into a mounted topdir died on the first source. On Darwin the build
  now runs against a container-local /root/rpmbuild and copies the
  RPMs out to the mounted dir afterwards; Linux keeps the mounted
  topdir.

The `request` flow (SRPM -> RPMs -> nested qemu preset) then completes
locally on macOS: the built aarch64 RPMs install in the smoke VM and
test_install_master passes.

Signed-off-by: Rafael Guterres Jeffman <rjeffman@redhat.com>
@rjeffman
rjeffman force-pushed the aarch64_macos_support branch from 3d76d97 to 2b49d0d Compare November 15, 2026 03:28

@abbra abbra left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks you for adding macOS support!

@abbra
abbra merged commit 267e213 into freeipa:main Nov 15, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants