feat: dnf module

modules/dnf/README.md

@@ -0,0 +1,29 @@
+# `dnf`
+
+The [`dnf`](https://docs.fedoraproject.org/en-US/quick-docs/dnf/) module offers pseudo-declarative package and repository management using `dnf`.
+
+The module first downloads the repository files from URLs declared under `repos:` into `/etc/yum.repos.d/`. The magic string `%OS_VERSION%` is substituted with the current VERSION_ID (major Fedora version), which can be used, for example, for pulling correct versions of repositories from [Fedora's Copr](https://copr.fedorainfracloud.org/).
+
+You can also add repository files directly into your git repository if URLs are not provided. For example:
+```yml
+repos:
+   - my-repository.repo # copies in .repo file from files/dnf/my-repository.repo to /etc/yum.repos.d/
+```
+If you use a repo that requires adding custom keys (eg. Brave Browser), you can import the keys by declaring the key URLs under `keys:`. The magic string acts the same as it does in `repos`.
+
+Then the module installs the packages declared under `install:` using `dnf install`, it removes the packages declared under `remove:` using `dnf remove`. If there are packages declared under both `install:` and `remove:` a hybrid command `dnf remove <packages> --install <packages>` is used, which should allow you to switch required packages for other ones.
+
+Installing RPM packages directly from a `http(s)` url that points to the RPM file is also supported, you can just put the URLs under `install:` and they'll be installed along with the other packages. The magic string `%OS_VERSION%` is substituted with the current VERSION_ID (major Fedora version) like with the `repos:` property.
+
+If an RPM is not available in a repository or as an URL, you can also install it directly from a file in your git repository. For example:
+```yml
+install:
+   - weird-package.rpm # tries to install files/dnf/weird-package.rpm
+```
+The module can also replace base RPM packages with packages from COPR repo. Under `replace:`, the module finds every pair of keys `- from-repo:` and `packages:`. (Multiple pairs are supported.) The module downloads the COPR repository file declared by `- from-repo:` into `/etc/yum.repos.d/`, and from that repository replaces packages declared under `packages:` using the command `dnf replace`. The COPR repository file is then deleted. The magic string `%OS_VERSION%` is substituted with the current VERSION_ID (major Fedora version) as already said above. At the moment, only COPR repo is supported.
+
+:::note
+[Removed packages are still present in the underlying ostree repository](https://coreos.github.io/rpm-ostree/administrator-handbook/#removing-a-base-package), what `remove` does is kind of like hiding them from the system, it doesn't free up storage space.
+:::
+
+Additionally, the `dnf` module supports a fix for packages that install into `/opt/`. Installation for packages that install into folder names declared under `optfix:` are fixed using some symlinks. Directory path in `/opt/` for those packages should be provided in recipe, like in Example Configuration.

modules/dnf/dnf.sh

@@ -0,0 +1,189 @@
+#!/usr/bin/env bash
+
+# Tell build process to exit if there are any errors.
+set -euo pipefail
+
+# Pull in repos
+get_json_array REPOS 'try .["repos"][]' "$1"
+if [[ ${#REPOS[@]} -gt 0 ]]; then
+    echo "Adding repositories"
+    for REPO in "${REPOS[@]}"; do
+        REPO="${REPO//%OS_VERSION%/${OS_VERSION}}"
+        REPO="${REPO//[$'\t\r\n ']}"
+
+        # If it's the COPR repo, then download the repo normally
+        # If it's not, then download the repo with URL in it's filename, to avoid duplicate repo name issue
+        if [[ "${REPO}" =~ ^https?:\/\/.* ]] && [[ "${REPO}" == "https://copr.fedorainfracloud.org/coprs/"* ]]; then
+          echo "Downloading repo file ${REPO}"
+          curl -fLs --create-dirs -O "${REPO}" --output-dir "/etc/yum.repos.d/"
+          echo "Downloaded repo file ${REPO}"
+        elif [[ "${REPO}" =~ ^https?:\/\/.* ]] && [[ "${REPO}" != "https://copr.fedorainfracloud.org/coprs/"* ]]; then
+          CLEAN_REPO_NAME=$(echo "${REPO}" | sed -E 's|^https?://([^?]+)(\?.*)?$|\1|')
+          CLEAN_REPO_NAME="${CLEAN_REPO_NAME//\//.}"
+
+          echo "Downloading repo file ${REPO}"
+          curl -fLs --create-dirs "${REPO}" -o "/etc/yum.repos.d/${CLEAN_REPO_NAME}"
+          echo "Downloaded repo file ${REPO}"
+        elif [[ ! "${REPO}" =~ ^https?:\/\/.* ]] && [[ "${REPO}" == *".repo" ]] && [[ -f "${CONFIG_DIRECTORY}/rpm-ostree/${REPO}" ]]; then
+          cp "${CONFIG_DIRECTORY}/rpm-ostree/${REPO}" "/etc/yum.repos.d/${REPO##*/}"
+        fi  
+    done
+fi
+
+get_json_array KEYS 'try .["keys"][]' "$1" 
+if [[ ${#KEYS[@]} -gt 0 ]]; then
+    echo "Adding keys"
+    for KEY in "${KEYS[@]}"; do
+        KEY="${KEY//%OS_VERSION%/${OS_VERSION}}"
+        rpm --import "${KEY//[$'\t\r\n ']}"
+    done
+fi
+
+# Create symlinks to fix packages that create directories in /opt
+get_json_array OPTFIX 'try .["optfix"][]' "$1"
+if [[ ${#OPTFIX[@]} -gt 0 ]]; then
+    echo "Creating symlinks to fix packages that install to /opt"
+    # Create symlink for /opt to /var/opt since it is not created in the image yet
+    mkdir -p "/var/opt"
+    ln -s "/var/opt"  "/opt"
+    # Create symlinks for each directory specified in recipe.yml
+    for OPTPKG in "${OPTFIX[@]}"; do
+        OPTPKG="${OPTPKG%\"}"
+        OPTPKG="${OPTPKG#\"}"
+        mkdir -p "/usr/lib/opt/${OPTPKG}"
+        ln -s "../../usr/lib/opt/${OPTPKG}" "/var/opt/${OPTPKG}"
+        echo "Created symlinks for ${OPTPKG}"
+    done
+fi
+
+get_json_array INSTALL_PKGS 'try .["install"][]' "$1"
+get_json_array REMOVE_PKGS 'try .["remove"][]' "$1"
+
+CLASSIC_INSTALL=false
+HTTPS_INSTALL=false
+LOCAL_INSTALL=false
+
+# Install and remove RPM packages
+# Sort classic, URL & local packages
+if [[ ${#INSTALL_PKGS[@]} -gt 0 ]]; then
+  for i in "${!INSTALL_PKGS[@]}"; do
+      PKG="${INSTALL_PKGS[$i]}"
+      if [[ "${PKG}" =~ ^https?:\/\/.* ]]; then
+        INSTALL_PKGS[$i]="${PKG//%OS_VERSION%/${OS_VERSION}}"
+        HTTPS_INSTALL=true
+        HTTPS_PKGS+=("${INSTALL_PKGS[$i]}")
+      elif [[ ! "${PKG}" =~ ^https?:\/\/.* ]] && [[ -f "${CONFIG_DIRECTORY}/rpm-ostree/${PKG}" ]]; then
+        LOCAL_INSTALL=true
+        LOCAL_PKGS+=("${CONFIG_DIRECTORY}/rpm-ostree/${PKG}")
+      else
+        CLASSIC_INSTALL=true
+        CLASSIC_PKGS+=("${PKG}")
+      fi
+  done
+fi
+
+echo_rpm_install() {
+    if ${CLASSIC_INSTALL} && ! ${HTTPS_INSTALL} && ! ${LOCAL_INSTALL}; then
+      echo "Installing: ${CLASSIC_PKGS[*]}"
+    elif ! ${CLASSIC_INSTALL} && ${HTTPS_INSTALL} && ! ${LOCAL_INSTALL}; then
+      echo "Installing package(s) directly from URL: ${HTTPS_PKGS[*]}"
+    elif ! ${CLASSIC_INSTALL} && ! ${HTTPS_INSTALL} && ${LOCAL_INSTALL}; then
+      echo "Installing local package(s): ${LOCAL_PKGS[*]}"
+    elif ${CLASSIC_INSTALL} && ${HTTPS_INSTALL} && ! ${LOCAL_INSTALL}; then
+      echo "Installing: ${CLASSIC_PKGS[*]}"
+      echo "Installing package(s) directly from URL: ${HTTPS_PKGS[*]}"
+    elif ${CLASSIC_INSTALL} && ! ${HTTPS_INSTALL} && ${LOCAL_INSTALL}; then
+      echo "Installing: ${CLASSIC_PKGS[*]}"
+      echo "Installing local package(s): ${LOCAL_PKGS[*]}"
+    elif ! ${CLASSIC_INSTALL} && ${HTTPS_INSTALL} && ${LOCAL_INSTALL}; then
+      echo "Installing package(s) directly from URL: ${HTTPS_PKGS[*]}"    
+      echo "Installing local package(s): ${LOCAL_PKGS[*]}"
+    elif ${CLASSIC_INSTALL} && ${HTTPS_INSTALL} && ${LOCAL_INSTALL}; then
+      echo "Installing: ${CLASSIC_PKGS[*]}"
+      echo "Installing package(s) directly from URL: ${HTTPS_PKGS[*]}"
+      echo "Installing local package(s): ${LOCAL_PKGS[*]}"
+    fi
+}
+
+if [[ ${#INSTALL_PKGS[@]} -gt 0 && ${#REMOVE_PKGS[@]} -gt 0 ]]; then
+    echo "Installing & Removing RPMs"
+    echo_rpm_install
+    echo "Removing: ${REMOVE_PKGS[*]}"
+    # Doing both actions in one command allows for replacing required packages with alternatives
+    # When --install= flag is used, URLs & local packages are not supported
+    if ${CLASSIC_INSTALL} && ! ${HTTPS_INSTALL} && ! ${LOCAL_INSTALL}; then
+      rpm-ostree override remove "${REMOVE_PKGS[@]}" $(printf -- "--install=%s " "${CLASSIC_PKGS[@]}")
+    elif ${CLASSIC_INSTALL} && ${HTTPS_INSTALL} && ! ${LOCAL_INSTALL}; then
+      rpm-ostree override remove "${REMOVE_PKGS[@]}" $(printf -- "--install=%s " "${CLASSIC_PKGS[@]}")
+      rpm-ostree install "${HTTPS_PKGS[@]}"
+    elif ${CLASSIC_INSTALL} && ! ${HTTPS_INSTALL} && ${LOCAL_INSTALL}; then
+      rpm-ostree override remove "${REMOVE_PKGS[@]}" $(printf -- "--install=%s " "${CLASSIC_PKGS[@]}")    
+      rpm-ostree install "${LOCAL_PKGS[@]}"
+    elif ${CLASSIC_INSTALL} && ${HTTPS_INSTALL} && ${LOCAL_INSTALL}; then
+      rpm-ostree override remove "${REMOVE_PKGS[@]}" $(printf -- "--install=%s " "${CLASSIC_PKGS[@]}")
+      rpm-ostree install "${HTTPS_PKGS[@]}" "${LOCAL_PKGS[@]}"
+    elif ! ${CLASSIC_INSTALL} && ! ${HTTPS_INSTALL} && ${LOCAL_INSTALL}; then
+      rpm-ostree override remove "${REMOVE_PKGS[@]}"
+      rpm-ostree install "${LOCAL_PKGS[@]}"
+    elif ! ${CLASSIC_INSTALL} && ${HTTPS_INSTALL} && ! ${LOCAL_INSTALL}; then
+      rpm-ostree override remove "${REMOVE_PKGS[@]}"
+      rpm-ostree install "${HTTPS_PKGS[@]}"
+    elif ! ${CLASSIC_INSTALL} && ${HTTPS_INSTALL} && ${LOCAL_INSTALL}; then
+      rpm-ostree override remove "${REMOVE_PKGS[@]}"
+      rpm-ostree install "${HTTPS_PKGS[@]}" "${LOCAL_PKGS[@]}"
+    fi  
+elif [[ ${#INSTALL_PKGS[@]} -gt 0 ]]; then
+    echo "Installing RPMs"
+    echo_rpm_install
+    rpm-ostree install "${INSTALL_PKGS[@]}"
+elif [[ ${#REMOVE_PKGS[@]} -gt 0 ]]; then
+    echo "Removing RPMs"
+    echo "Removing: ${REMOVE_PKGS[*]}"
+    rpm-ostree override remove "${REMOVE_PKGS[@]}"
+fi
+
+get_json_array REPLACE 'try .["replace"][]' "$1"
+
+# Override-replace RPM packages
+if [[ ${#REPLACE[@]} -gt 0 ]]; then
+    for REPLACEMENT in "${REPLACE[@]}"; do
+
+        # Get repository
+        REPO=$(echo "${REPLACEMENT}" | jq -r 'try .["from-repo"]')
+        REPO="${REPO//%OS_VERSION%/${OS_VERSION}}"
+
+        # Ensure repository is provided
+        if [[ "${REPO}" == "null" ]]; then
+            echo "Error: Key 'from-repo' was declared, but repository URL was not provided."
+            exit 1
+        fi
+
+        # Get info from repository URL
+        MAINTAINER=$(awk -F'/' '{print $5}' <<< "${REPO}")
+        REPO_NAME=$(awk -F'/' '{print $6}' <<< "${REPO}")
+        FILE_NAME=$(awk -F'/' '{print $9}' <<< "${REPO}" | sed 's/\?.*//') # Remove params after '?'
+
+        # Get packages to replace
+        get_json_array PACKAGES 'try .["packages"][]' "${REPLACEMENT}"
+        REPLACE_STR="$(echo "${PACKAGES[*]}" | tr -d '\n')"
+
+        # Ensure packages are provided
+        if [[ ${#PACKAGES[@]} == 0 ]]; then
+            echo "Error: No packages were provided for repository '${REPO_NAME}'."
+            exit 1
+        fi
+
+        echo "Replacing packages from COPR repository: '${REPO_NAME}' owned by '${MAINTAINER}'"
+        echo "Replacing: ${REPLACE_STR}"
+
+        REPO_URL="${REPO//[$'\t\r\n ']}"
+
+        echo "Downloading repo file ${REPO_URL}"
+        curl -fLs --create-dirs -O "${REPO_URL}" --output-dir "/etc/yum.repos.d/"
+        echo "Downloaded repo file ${REPO_URL}"
+
+        rpm-ostree override replace --experimental --from "repo=copr:copr.fedorainfracloud.org:${MAINTAINER}:${REPO_NAME}" ${REPLACE_STR}
+        rm "/etc/yum.repos.d/${FILE_NAME}"
+
+    done
+fi

modules/dnf/dnf.tsp

@@ -0,0 +1,33 @@
+import "@typespec/json-schema";
+using TypeSpec.JsonSchema;
+
+@jsonSchema("/modules/dnf.json")
+model RpmOstreeModule {
+    /** The dnf module offers pseudo-declarative package and repository management using dnf.
+     * https://blue-build.org/reference/modules/dnf/
+     */
+    type: "dnf";
+
+    /** List of links to .repo files to download into /etc/yum.repos.d/. */
+    repos?: Array<string>;
+
+    /** List of links to key files to import for installing from custom repositories. */
+    keys?: Array<string>;
+
+    /** List of folder names under /opt/ to enable for installing into. */
+    optfix?: Array<string>;
+
+    /** List of RPM packages to install. */
+    install?: Array<string>;
+
+    /** List of RPM packages to remove. */
+    remove?: Array<string>;
+
+    /** List of configurations for `rpm-ostree override replace`ing packages. */
+    replace?: Array<{
+        /** URL to the source COPR repo for the new packages. */
+        "from-repo": string,
+        /** List of packages to replace using packages from the defined repo. */
+        packages: Array<string>,
+    }>;
+}

modules/dnf/module.yml

@@ -0,0 +1,25 @@
+name: dnf
+shortdesc: The dnf module offers pseudo-declarative package and repository management using dnf.
+example: |
+  type: dnf
+  repos:
+    - https://copr.fedorainfracloud.org/coprs/atim/starship/repo/fedora-%OS_VERSION%/atim-starship-fedora-%OS_VERSION%.repo # when including COPR repos, use the %OS_VERSION% magic string
+    - https://brave-browser-rpm-release.s3.brave.com/brave-browser.repo
+  keys:
+    - https://brave-browser-rpm-release.s3.brave.com/brave-core.asc
+  optfix:
+    - Tabby # needed because tabby installs into /opt/Tabby
+    - brave.com
+  install:
+    - starship
+    - brave-browser
+    - https://github.com/Eugeny/tabby/releases/download/v1.0.209/tabby-1.0.209-linux-x64.rpm
+  remove:
+    - firefox
+    - firefox-langpacks
+  replace:
+    - from-repo: https://copr.fedorainfracloud.org/coprs/trixieua/mutter-patched/repo/fedora-%OS_VERSION%/trixieua-mutter-patched-fedora-%OS_VERSION%.repo
+      packages:
+        - mutter
+        - mutter-common
+        - gdm