|
| 1 | +================== |
| 2 | +Config Annotations |
| 3 | +================== |
| 4 | + |
| 5 | +:Author: Andrea Righi |
| 6 | + |
| 7 | +Overview |
| 8 | +======== |
| 9 | + |
| 10 | +Each Ubuntu kernel needs to maintain its own .config for each supported |
| 11 | +architecture and each flavour. |
| 12 | + |
| 13 | +Every time a new patch is applied or a kernel is rebased on top of a new |
| 14 | +one, we need to update the .config's accordingly (config options can be |
| 15 | +added, removed and also renamed). |
| 16 | + |
| 17 | +So, we need to make sure that some critical config options are always |
| 18 | +matching the desired value in order to have a functional kernel. |
| 19 | + |
| 20 | +State of the art |
| 21 | +================ |
| 22 | + |
| 23 | +At the moment configs are maintained as a set of Kconfig chunks (inside |
| 24 | +`debian.<kernel>/config/`): a global one, plus per-arch / per-flavour |
| 25 | +chunks. |
| 26 | + |
| 27 | +In addition to that, we need to maintain also a file called |
| 28 | +'annotations'; the purpose of this file is to make sure that some |
| 29 | +critical config options are not silently removed or changed when the |
| 30 | +real .config is re-generated (for example after a rebase or after |
| 31 | +applying a new set of patches). |
| 32 | + |
| 33 | +The main problem with this approach is that, often, we have duplicate |
| 34 | +information that is stored both in the Kconfig chunks *and* in the |
| 35 | +annotations files and, at the same time, the whole .config's information |
| 36 | +is distributed between Kconfig chunks and annotations, making it hard to |
| 37 | +maintain, review and manage in general. |
| 38 | + |
| 39 | +Proposed solution |
| 40 | +================= |
| 41 | + |
| 42 | +The proposed solution is to store all the config information into the |
| 43 | +"annotations" format and get rid of the config chunks (basically the |
| 44 | +real .config's can be produced "compiling" annotations). |
| 45 | + |
| 46 | +Implementation |
| 47 | +============== |
| 48 | + |
| 49 | +To help the management of the annotations an helper script is provided |
| 50 | +(`debian/scripts/misc/annotations`): |
| 51 | + |
| 52 | +``` |
| 53 | +usage: annotations [-h] [--version] [--file FILE] [--arch ARCH] [--flavour FLAVOUR] [--config CONFIG] |
| 54 | + (--query | --export | --import FILE | --update FILE | --check FILE) |
| 55 | + |
| 56 | +Manage Ubuntu kernel .config and annotations |
| 57 | + |
| 58 | +options: |
| 59 | + -h, --help show this help message and exit |
| 60 | + --version, -v show program's version number and exit |
| 61 | + --file FILE, -f FILE Pass annotations or .config file to be parsed |
| 62 | + --arch ARCH, -a ARCH Select architecture |
| 63 | + --flavour FLAVOUR, -l FLAVOUR |
| 64 | + Select flavour (default is "generic") |
| 65 | + --config CONFIG, -c CONFIG |
| 66 | + Select a specific config option |
| 67 | + |
| 68 | +Action: |
| 69 | + --query, -q Query annotations |
| 70 | + --export, -e Convert annotations to .config format |
| 71 | + --import FILE, -i FILE |
| 72 | + Import a full .config for a specific arch and flavour into annotations |
| 73 | + --update FILE, -u FILE |
| 74 | + Import a partial .config into annotations (only resync configs specified in FILE) |
| 75 | + --check FILE, -k FILE |
| 76 | + Validate kernel .config with annotations |
| 77 | +``` |
| 78 | + |
| 79 | +This script allows to query config settings (per arch/flavour/config), |
| 80 | +export them into the Kconfig format (generating the real .config files) |
| 81 | +and check if the final .config matches the rules defined in the |
| 82 | +annotations. |
| 83 | + |
| 84 | +Examples (annotations is defined as an alias to `debian/scripts/annotations`): |
| 85 | + |
| 86 | + - Show settings for `CONFIG_DEBUG_INFO_BTF` for master kernel across all the |
| 87 | + supported architectures and flavours: |
| 88 | + |
| 89 | +``` |
| 90 | +$ annotations --query --config CONFIG_DEBUG_INFO_BTF |
| 91 | +{ |
| 92 | + "policy": { |
| 93 | + "amd64": "y", |
| 94 | + "arm64": "y", |
| 95 | + "armhf": "n", |
| 96 | + "ppc64el": "y", |
| 97 | + "riscv64": "y", |
| 98 | + "s390x": "y" |
| 99 | + }, |
| 100 | + "note": "'Needs newer pahole for armhf'" |
| 101 | +} |
| 102 | +``` |
| 103 | + |
| 104 | + - Dump kernel .config for arm64 and flavour generic-64k: |
| 105 | + |
| 106 | +``` |
| 107 | +$ annotations --arch arm64 --flavour generic-64k --export |
| 108 | +CONFIG_DEBUG_FS=y |
| 109 | +CONFIG_DEBUG_KERNEL=y |
| 110 | +CONFIG_COMPAT=y |
| 111 | +... |
| 112 | +``` |
| 113 | + |
| 114 | + - Update annotations file with a new kernel .config for amd64 flavour |
| 115 | + generic: |
| 116 | + |
| 117 | +``` |
| 118 | +$ annotations --arch amd64 --flavour generic --import build/.config |
| 119 | +``` |
| 120 | + |
| 121 | +Moreover, an additional kernelconfig commands are provided |
| 122 | +(via debian/rules targets): |
| 123 | + - `migrateconfigs`: automatically merge all the previous configs into |
| 124 | + annotations (local changes still need to be committed) |
| 125 | + |
| 126 | +Annotations headers |
| 127 | +=================== |
| 128 | + |
| 129 | +The main annotations file should contain a header to define the architectures |
| 130 | +and flavours that are supported. |
| 131 | + |
| 132 | +Here is the format of the header for the generic kernel: |
| 133 | +``` |
| 134 | +# Menu: HEADER |
| 135 | +# FORMAT: 4 |
| 136 | +# ARCH: amd64 arm64 armhf ppc64el riscv64 s390x |
| 137 | +# FLAVOUR: amd64-generic arm64-generic arm64-generic-64k armhf-generic armhf-generic-lpae ppc64el-generic riscv64-generic s390x-generic |
| 138 | + |
| 139 | +``` |
| 140 | + |
| 141 | +Example header of a derivative (linux-aws): |
| 142 | +``` |
| 143 | +# Menu: HEADER |
| 144 | +# FORMAT: 4 |
| 145 | +# ARCH: amd64 arm64 |
| 146 | +# FLAVOUR: amd64-aws arm64-aws |
| 147 | +# FLAVOUR_DEP: {'amd64-aws': 'amd64-generic', 'arm64-aws': 'arm64-generic'} |
| 148 | + |
| 149 | +include "../../debian.master/config/annotations" |
| 150 | + |
| 151 | +# Below you can define only the specific linux-aws configs that differ from linux generic |
| 152 | + |
| 153 | +``` |
| 154 | + |
| 155 | +Pros and Cons |
| 156 | +============= |
| 157 | + |
| 158 | + Pros: |
| 159 | + - avoid duplicate information in .config's and annotations |
| 160 | + - allow to easily define groups of config settings (for a specific |
| 161 | + environment or feature, such as annotations.clouds, annotations.ubuntu, |
| 162 | + annotations.snapd, etc.) |
| 163 | + - config options are more accessible, easy to change and review |
| 164 | + - we can easily document how config options are managed (and external |
| 165 | + contributors won't be discouraged anymore when they need to to change a |
| 166 | + config option) |
| 167 | + |
| 168 | + Cons: |
| 169 | + - potential regressions: the new tool/scripts can have potential bugs, |
| 170 | + so we could experience regressions due to some missed config changes |
| 171 | + - kernel team need to understand the new process (even if everything |
| 172 | + is transparent, kernel cranking process is the same, there might be |
| 173 | + corner cases that need to be addressed and resolved manually) |
| 174 | + |
| 175 | +TODO |
| 176 | +==== |
| 177 | + |
| 178 | + - Migrate all flavour and arch definitions into annotations (rather |
| 179 | + than having this information defined in multiple places inside |
| 180 | + debian/scripts); right now this information is "partially" migrated, |
| 181 | + meaning that we need to define arches and flavours in the headers |
| 182 | + section of annotations (so that the annotations tool can figure out |
| 183 | + the list of supported arches and flavours), but arches and flavours |
| 184 | + are still defined elsewhere, ideally we would like to have arches and |
| 185 | + flavours defined only in one place: annotations. |
0 commit comments