Skip to content

Commit

Permalink
nixos/xen: refactor dom0 configuration (NixOS#324911)
Browse files Browse the repository at this point in the history
  • Loading branch information
emilazy authored Sep 18, 2024
2 parents d0b8a0b + 9e5f77a commit 5320e21
Show file tree
Hide file tree
Showing 3 changed files with 978 additions and 385 deletions.
10 changes: 10 additions & 0 deletions nixos/doc/manual/release-notes/rl-2411.section.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,16 @@
If you experience any issues, please report them.
The original Perl script can still be used for now by setting `system.switch.enableNg` to `false`.

- The [Xen Hypervisor](https://xenproject.org) is once again available as a virtualisation option under [`virtualisation.xen`](#opt-virtualisation.xen.enable).
- This release includes Xen [4.17.5](https://wiki.xenproject.org/wiki/Xen_Project_4.17_Release_Notes), [4.18.3](https://wiki.xenproject.org/wiki/Xen_Project_4.18_Release_Notes) and [4.19.0](https://wiki.xenproject.org/wiki/Xen_Project_4.19_Release_Notes), as well as support for booting the hypervisor on EFI systems.
::: {.warning}
Booting into Xen through a legacy BIOS bootloader or with the legacy script-based Stage 1 initrd have been **deprecated**. Only EFI booting and the new systemd-based Stage 1 initrd are supported.
:::
- There are two flavours of Xen available by default: `xen`, which includes all built-in components, and `xen-slim`, which replaces the built-in components with their Nixpkgs equivalents.
- The `qemu-xen-traditional` component has been deprecated by upstream Xen, and is no longer available in any of the Xen packages.
- The OCaml-based Xen Store can now be configured using [`virtualisation.xen.store.settings`](#opt-virtualisation.xen.store.settings).
- The `virtualisation.xen.bridge` options have been deprecated in this release cycle. Users who need network bridges are encouraged to set up their own networking configurations.

## New Modules {#sec-release-24.11-new-modules}

- [TaskChampion Sync-Server](https://github.com/GothenburgBitFactory/taskchampion-sync-server), a [Taskwariror 3](https://taskwarrior.org/docs/upgrade-3/) sync server, replacing Taskwarrior 2's sync server named [`taskserver`](https://github.com/GothenburgBitFactory/taskserver).
Expand Down
165 changes: 165 additions & 0 deletions nixos/modules/virtualisation/xen-boot-builder.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
# This script is called by ./xen-dom0.nix to create the Xen boot entries.
# shellcheck shell=bash

# Handle input argument and exit if the flag is invalid. See virtualisation.xen.efi.bootBuilderVerbosity below.
[[ $# -ne 1 ]] && echo -e "\e[1;31merror:\e[0m xenBootBuilder must be called with exactly one verbosity argument. See the \e[1;34mvirtualisation.xen.efi.bootBuilderVerbosity\e[0m option." && exit 1
case "$1" in
"quiet") true ;;
"default" | "info") echo -n "Installing Xen Hypervisor boot entries..." ;;
"debug") echo -e "\e[1;34mxenBootBuilder:\e[0m called with the '$1' flag" ;;
*)
echo -e "\e[1;31merror:\e[0m xenBootBuilder was called with an invalid argument. See the \e[1;34mvirtualisation.xen.efi.bootBuilderVerbosity\e[0m option."
exit 2
;;
esac

# Get the current Xen generations and store them in an array. This will be used
# for displaying the diff later, if xenBootBuilder was called with `info`.
# We also delete the current Xen entries here, as they'll be rebuilt later if
# the corresponding NixOS generation still exists.
mapfile -t preGenerations < <(find "$efiMountPoint"/loader/entries -type f -name 'xen-*.conf' | sort -V | sed 's_/loader/entries/nixos_/loader/entries/xen_g;s_^.*/xen_xen_g;s_.conf$__g')
if [ "$1" = "debug" ]; then
if ((${#preGenerations[@]} == 0)); then
echo -e "\e[1;34mxenBootBuilder:\e[0m no previous Xen entries."
else
echo -e "\e[1;34mxenBootBuilder:\e[0m deleting the following stale xen entries:" && for debugGen in "${preGenerations[@]}"; do echo " - $debugGen"; done
fi
fi

# Cleanup all Xen entries.
rm -f "$efiMountPoint"/{loader/entries/xen-*.conf,efi/nixos/xen-*.efi}

# Main array for storing which generations exist in $efiMountPoint after
# systemd-boot-builder.py builds the main entries.
mapfile -t gens < <(find "$efiMountPoint"/loader/entries -type f -name 'nixos-*.conf' | sort -V)
[ "$1" = "debug" ] && echo -e "\e[1;34mxenBootBuilder:\e[0m found the following NixOS boot entries:" && for debugGen in "${gens[@]}"; do echo " - $debugGen"; done

# This is the main loop that installs the Xen entries.
for gen in "${gens[@]}"; do

# We discover the path to Bootspec through the init attribute in the entries,
# as it is equivalent to $toplevel/init.
bootspecFile="$(sed -nr 's/^options init=(.*)\/init.*$/\1/p' "$gen")/boot.json"
[ "$1" = "debug" ] && echo -e "\e[1;34mxenBootBuilder:\e[0m processing bootspec file $bootspecFile"

# We do nothing if the Bootspec for the current $gen does not contain the Xen
# extension, which is added as a configuration attribute below.
if grep -sq '"org.xenproject.bootspec.v1"' "$bootspecFile"; then
[ "$1" = "debug" ] && echo -e " \e[1;32msuccess:\e[0m found Xen entries in $gen."

# TODO: Support DeviceTree booting. Xen has some special handling for DeviceTree
# attributes, which will need to be translated in a boot script similar to this
# one. Having a DeviceTree entry is rare, and it is not always required for a
# successful boot, so we don't fail here, only warn with `debug`.
if grep -sq '"devicetree"' "$bootspecFile"; then
echo -e "\n\e[1;33mwarning:\e[0m $gen has a \e[1;34morg.nixos.systemd-boot.devicetree\e[0m Bootspec entry. Xen currently does not support DeviceTree, so this value will be ignored in the Xen boot entries, which may cause them to \e[1;31mfail to boot\e[0m."
else
[ "$1" = "debug" ] && echo -e "\e[1;34mxenBootBuilder:\e[0m no DeviceTree entries found in $gen."
fi

# Prepare required attributes for `xen.cfg/xen.conf`. It inherits the name of
# the corresponding nixos generation, substituting `nixos` with `xen`:
# `xen-$profile-generation-$number-specialisation-$specialisation.{cfg,conf}`
xenGen=$(echo "$gen" | sed 's_/loader/entries/nixos_/loader/entries/xen_g;s_^.*/xen_xen_g;s_.conf$__g')
bootParams=$(jq -re '."org.xenproject.bootspec.v1".xenParams | join(" ")' "$bootspecFile")
kernel=$(jq -re '."org.nixos.bootspec.v1".kernel | sub("^/nix/store/"; "") | sub("/bzImage"; "-bzImage.efi")' "$bootspecFile")
kernelParams=$(jq -re '."org.nixos.bootspec.v1".kernelParams | join(" ")' "$bootspecFile")
initrd=$(jq -re '."org.nixos.bootspec.v1".initrd | sub("^/nix/store/"; "") | sub("/initrd"; "-initrd.efi")' "$bootspecFile")
init=$(jq -re '."org.nixos.bootspec.v1".init' "$bootspecFile")
title=$(sed -nr 's/^title (.*)$/\1/p' "$gen")
version=$(sed -nr 's/^version (.*)$/\1/p' "$gen")
machineID=$(sed -nr 's/^machine-id (.*)$/\1/p' "$gen")
sortKey=$(sed -nr 's/^sort-key (.*)$/\1/p' "$gen")

# Write `xen.cfg` to a temporary location prior to UKI creation.
tmpCfg=$(mktemp)
[ "$1" = "debug" ] && echo -ne "\e[1;34mxenBootBuilder:\e[0m writing $xenGen.cfg to temporary file..."
cat > "$tmpCfg" << EOF
[global]
default=xen
[xen]
options=$bootParams
kernel=$kernel init=$init $kernelParams
ramdisk=$initrd
EOF
[ "$1" = "debug" ] && echo -e "done."

# Create Xen UKI for $generation. Most of this is lifted from
# https://xenbits.xenproject.org/docs/unstable/misc/efi.html.
[ "$1" = "debug" ] && echo -e "\e[1;34mxenBootBuilder:\e[0m making Xen UKI..."
xenEfi=$(jq -re '."org.xenproject.bootspec.v1".xen' "$bootspecFile")
padding=$(objdump --header --section=".pad" "$xenEfi" | awk '/\.pad/ { printf("0x%016x\n", strtonum("0x"$3) + strtonum("0x"$4))};')
[ "$1" = "debug" ] && echo " - padding: $padding"
objcopy \
--add-section .config="$tmpCfg" \
--change-section-vma .config="$padding" \
"$xenEfi" \
"$efiMountPoint"/EFI/nixos/"$xenGen".efi
[ "$1" = "debug" ] && echo -e " - \e[1;32msuccessfully built\e[0m $xenGen.efi"
rm -f "$tmpCfg"

# Write `xen.conf`.
[ "$1" = "debug" ] && echo -ne "\e[1;34mxenBootBuilder:\e[0m writing $xenGen.conf to EFI System Partition..."
cat > "$efiMountPoint"/loader/entries/"$xenGen".conf << EOF
title $title (with Xen Hypervisor)
version $version
efi /EFI/nixos/$xenGen.efi
machine-id $machineID
sort-key $sortKey
EOF
[ "$1" = "debug" ] && echo -e "done."

# Sometimes, garbage collection weirdness causes a generation to still exist in
# the loader entries, but its Bootspec file was deleted. We consider such a
# generation to be invalid, but we don't write extra code to handle this
# situation, as supressing grep's error messages above is quite enough, and the
# error message below is still technically correct, as no Xen can be found in
# something that does not exist.
else
[ "$1" = "debug" ] && echo -e " \e[1;33mwarning:\e[0m \e[1;31mno Xen found\e[0m in $gen."
fi
done

# Counterpart to the preGenerations array above. We use it to diff the
# generations created/deleted when callled with the `info` argument.
mapfile -t postGenerations < <(find "$efiMountPoint"/loader/entries -type f -name 'xen-*.conf' | sort -V | sed 's_/loader/entries/nixos_/loader/entries/xen_g;s_^.*/xen_xen_g;s_.conf$__g')

# In the event the script does nothing, guide the user to debug, as it'll only
# ever run when Xen is enabled, and it makes no sense to enable Xen and not have
# any hypervisor boot entries.
if ((${#postGenerations[@]} == 0)); then
case "$1" in
"default" | "info") echo "none found." && echo -e "If you believe this is an error, set the \e[1;34mvirtualisation.xen.efi.bootBuilderVerbosity\e[0m option to \e[1;34m\"debug\"\e[0m and rebuild to print debug logs." ;;
"debug") echo -e "\e[1;34mxenBootBuilder:\e[0m wrote \e[1;31mno generations\e[0m. Most likely, there were no generations with a valid \e[1;34morg.xenproject.bootspec.v1\e[0m entry." ;;
esac

# If the script is successful, change the default boot, say "done.", write a
# diff, or print the total files written, depending on the argument this script
# was called with. We use some dumb dependencies here, like `diff` or `bat` for
# colourisation, but they're only included with the `info` argument.
#
# It's also fine to change the default here, as this runs after the
# `systemd-boot-builder.py` script, which overwrites the file, and this script
# does not run after an user disables the Xen module.
else
sed --in-place 's/^default nixos-/default xen-/g' "$efiMountPoint"/loader/loader.conf
case "$1" in
"default" | "info") echo "done." ;;
"debug") echo -e "\e[1;34mxenBootBuilder:\e[0m \e[1;32msuccessfully wrote\e[0m the following generations:" && for debugGen in "${postGenerations[@]}"; do echo " - $debugGen"; done ;;
esac
if [ "$1" = "info" ]; then
if [[ ${#preGenerations[@]} == "${#postGenerations[@]}" ]]; then
echo -e "\e[1;33mNo Change:\e[0m Xen Hypervisor boot entries were refreshed, but their contents are identical."
else
echo -e "\e[1;32mSuccess:\e[0m Changed the following boot entries:"
# We briefly unset errexit and pipefail here, as GNU diff has no option to not fail when files differ.
set +o errexit
set +o pipefail
diff <(echo "${preGenerations[*]}" | tr ' ' '\n') <(echo "${postGenerations[*]}" | tr ' ' '\n') -U 0 | grep --invert-match --extended-regexp '^(@@|---|\+\+\+).*' | sed '1{/^-$/d}' | bat --language diff --theme ansi --paging=never --plain
true
set -o errexit
set -o pipefail
fi
fi
fi
Loading

0 comments on commit 5320e21

Please sign in to comment.