ci: add postmarketOS to integration testing by craftyguy · Pull Request #4297 · systemd/mkosi

craftyguy · 2026-05-11T23:24:36Z

apk-tools isn't packaged in Ubuntu, so the scope of testing is limited quite a bit to only a handful of distros/tools trees that have apk available. This also means we're not able to test building images for other distros in a pmOS tools tree since the base distro is... 🥁... Ubuntu and it's unable to build a pmOS tools tree because it doesn't have apk-tools.

craftyguy · 2026-05-11T23:38:21Z

This opensuse tools tree failure seems unrelated to my changes, it's also failing when building images for other distros:

[32](https://github.com/systemd/mkosi/actions/runs/25703288918/job/75468214832?pr=4297#step:10:4133)
+ /usr/lib/systemd/systemd-keyutil extract-public --private-key /work/home/runner/work/mkosi/mkosi/mkosi.key
+ /usr/lib/systemd/systemd-measure sign --osrel=/work/var/tmp/mkosi-workspace-ovxxn95g/root/usr/lib/os-release --cmdline=/work/var/tmp/mkosi-workspace-ovxxn95g/cmdline --uname=/tmp/tmp.uname0_1a3qma --pcrpkey=/tmp/tmp.pcrpkey7wvwhp5e --linux=/work/var/tmp/mkosi-workspace-ovxxn95g/root/usr/lib/modules/6.18.29-0-virt/vmlinuz --initrd=/tmp/tmp.initrdxwyk1xfg --sbat=/tmp/tmp.sbatetn4_75z --bank=sha256 --private-key=/work/home/runner/work/mkosi/mkosi/mkosi.key
Could not calculate PolicyPCR digest: Operation not supported

behrmann · 2026-05-12T08:36:26Z

Thanks for this!

Yeah, opensuse is borked at the moment and nobody had time to investigate yet.

daandemeyer · 2026-05-19T13:35:32Z

@craftyguy We can mess around a bit, you can add a step to the mkosi action to download a static apk (check the hash ofc) and together with #4313 we'll be able to build from ubuntu

daandemeyer · 2026-05-21T17:30:52Z

If you rebase we now install apk in the github action

behrmann · 2026-06-13T14:17:02Z

@craftyguy could you rebase this? I think it would be great if we could land this before any larger work on the integration tests, like #4357, happens.

craftyguy · 2026-06-16T04:29:14Z

@behrmann rebased, and got pmOS-related tests passing now

martinpitt · 2026-06-16T04:40:35Z

This fails opensuse's test_extension[sysext] on both debian and archi, I haven't seen that breakage yet. That needs a closer look, smells like and actual regression? this run could be an opensuse mirror flake, worth retrying (I can't trigger retries myself)

Update: The two failures above are the same reason, suse repo download timeouts/failures. ✔️

martinpitt

Thanks @craftyguy ! I have some questions, and the commit series could use some squashing and reordering, but by and large this looks great!

martinpitt · 2026-06-16T06:47:37Z

          - debian
          - fedora
          - opensuse
+          - postmarketos


Is pOS actually meant to be a dev platform where you would run things like mkosi to build e.g. ubuntu? If not, testing with a pOS tools tree feels like a waste (we already run into serious github runner capacity issues). If so, then fine of course.

If this gets dropped, the excludes from below can be reduced.

In theory, yes, it's a general purpose distro that happens to run on phones (among many other things). In practice, I have no idea who is doing dev work on other distros using mkosi on pmOS. We do have users running other distros in containers with podman, docker, and VMs.

I enabled these because it's what was done for other distros, but I'd be OK with removing them if it makes your CI load lighter and re-adding them IIF people start reporting issues e.g. building ubuntu on pmOS with mkosi.

wdyt?

I'm on the fence, and as I haven't yet earned the right to make large directional decisions for mkosi: @behrmann @daandemeyer WDYT?

Generally, I think, that a lot of the matrix of our build tests covers test cases people will probably never do, since e.g. I assume there's little overlap between Arch and CentOS target audiences. I kinda don't want to make pmOS a second class citizen, but it's also clear that this grows quadratically, which isn't great either.

That being said. Even if we only test pmOS as a target from various distros and don't build other distros from pmOS, I really would always want to see that pmOS builds from a pmOS tools tree.

Even if we only test pmOS as a target from various distros and don't build other distros from pmOS, I really would always want to see that pmOS builds from a pmOS tools tree.

Sounds good to me, and we can always adjust later if necessary. So IIUC, only enable:

build pmOS on pmOS

build pmOS on everything

And I would not enable:

build everything on pmOS

I kinda don't want to make pmOS a second class citizen

I (a pmOS dev) definitely don't feel like a second class citizen here, so don't worry :P

@behrmann ok I think I got this implemented. Now pmOS is built in other distro tools trees (that have apk available) and no distros are built with a pmOS tools tree (other than pmOS). I had to add to the "include:" list to get pmOS building with pmOS tools tree.

busybox's losetup, mount, and umount are missing many options supported by the "full" versions so the "full" versions are added sfdisk is packaged separately from util-linux on Alpine Linux

sulogin is packaged separately on Alpine Linux, and this other pkg is not pulled in automatically. Fixes emergency shell in the initrd device-mapper is not pulled in automatically as a dependency, and needed for lvm support

On Alpine/pmOS, the EFI stub is split into a separate systemd-efistub package rather than being bundled with systemd-boot, so it needs to be added to the tools tree so mkosi can find it when building UKI/ESP images.

martinpitt · 2026-06-16T17:52:13Z

I had another look. If you revert the moving of apk install and put it back into actions.yml in /usr, I have no further objections. (Pending the "tools test or not" question from the maintainers above). Thanks!

/usr/local is cleaned up by ci.yml immediately after actions.yaml adds apk into that dir, which leads to job that try to use apk to fail with "apk not found"

behrmann · 2026-06-17T07:35:09Z

Modulo the concerns about the size of the build matrix this LGTM.

martinpitt

Excellent! I'm looking into these ( ➡️ #4363) two ( ➡️ #4362 ) failures. They are unrelated to this PR, but annoying.

behrmann

Lovely, thank you! One last question

behrmann · 2026-06-18T07:55:28Z

-        sudo curl -fsSL -o /usr/local/bin/apk https://gitlab.alpinelinux.org/api/v4/projects/5/packages/generic/v3.0.6/x86_64/apk.static
-        echo 'f1489e05bace7d7dd0a687fcd38d50b585ac660af4231668b123649bef3718c4  /usr/local/bin/apk' | sha256sum --check
-        sudo chmod +x /usr/local/bin/apk
+        sudo curl -fsSL -o /usr/bin/apk https://gitlab.alpinelinux.org/api/v4/projects/5/packages/generic/v3.0.6/x86_64/apk.static


You didn't introduce this, but I wonder whether we have a good way of good way of keeping this up to date.

The best idea I have is to use the gitlab API to get the latest release of apk-tools, get the checksum for it, then download/verify the apk.static binary for x86_64. the API is json, so it would be a handful of curl | jq lines.

If that seems like an OK approach, I can take a shot at implementing it

That would be lovely, but that's for a later PR.

ok I gave it a go over here: #4364

craftyguy force-pushed the pmos-can-into-ci branch 9 times, most recently from af935f7 to 39fd928 Compare June 16, 2026 04:07

martinpitt reviewed Jun 16, 2026

View reviewed changes

craftyguy added 5 commits June 16, 2026 09:12

mkosi-tools: add apk-tools to tools trees for Arch and OpenSuSE

0774bc2

mkosi-tools: add missing packages to postmarketOS tools tree

c3575c0

busybox's losetup, mount, and umount are missing many options supported by the "full" versions so the "full" versions are added sfdisk is packaged separately from util-linux on Alpine Linux

mkosi.conf: add grub to postmarketOS

5640ace

mkosi-vm: add systemd-efistub to postmarketOS config

9980f31

On Alpine/pmOS, the EFI stub is split into a separate systemd-efistub package rather than being bundled with systemd-boot, so it needs to be added to the tools tree so mkosi can find it when building UKI/ESP images.

craftyguy force-pushed the pmos-can-into-ci branch from 39fd928 to 36a1074 Compare June 16, 2026 16:53

martinpitt reviewed Jun 16, 2026

View reviewed changes

Comment thread .github/workflows/ci.yml Outdated

craftyguy force-pushed the pmos-can-into-ci branch 2 times, most recently from 8a360f2 to 2e662f3 Compare June 16, 2026 19:22

action: install apk to /usr/bin

d4c6761

/usr/local is cleaned up by ci.yml immediately after actions.yaml adds apk into that dir, which leads to job that try to use apk to fail with "apk not found"

craftyguy force-pushed the pmos-can-into-ci branch from 2e662f3 to abc2c67 Compare June 16, 2026 20:41

ci: enable postmarketOS in integration testing

0dc5ecb

craftyguy force-pushed the pmos-can-into-ci branch from abc2c67 to 0dc5ecb Compare June 17, 2026 21:43

martinpitt approved these changes Jun 18, 2026

View reviewed changes

behrmann approved these changes Jun 18, 2026

View reviewed changes

behrmann merged commit 50c8feb into systemd:main Jun 18, 2026
78 of 85 checks passed

Uh oh!

Conversation

craftyguy commented May 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

craftyguy commented May 11, 2026

Uh oh!

behrmann commented May 12, 2026

Uh oh!

daandemeyer commented May 19, 2026

Uh oh!

daandemeyer commented May 21, 2026

Uh oh!

behrmann commented Jun 13, 2026

Uh oh!

craftyguy commented Jun 16, 2026

Uh oh!

martinpitt commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

martinpitt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

martinpitt commented Jun 16, 2026

Uh oh!

behrmann commented Jun 17, 2026

Uh oh!

martinpitt left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

behrmann left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

4 participants

craftyguy commented May 11, 2026 •

edited

Loading

martinpitt commented Jun 16, 2026 •

edited

Loading

martinpitt left a comment •

edited

Loading