Add package requirements #24
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Currently, upgrades with custom repositories hosted on private https servers are not functional. The reason is that the server certificates are unavailable in the upgrade container as they are not copied from the source system; therefore, the upgrade fails when trying to access the custom repository. This change fixes the issue by copying all files from the ``/etc/pki`` folder of the source system, while respecting any new files installed by packages into the container. If a conflict arises a file installed by a package into a container is preferred. Any broken symlinks are skipped and ignored.
It happened several times that a link we used in reports became invalid over time, so it could not provide expected help to customers. The fix of such links require update of the code, which does not have to be possible on existing releases. Instead of that, we rather decided to use shortnened URLs when possible and when it makes sense to reduce the risk. The benefit is that we are able to change the target destination of such https://red.ht/... URLs so we can fix such issues without the need to update the code (and rpms). New shortened URLs: * https://red.ht/upgrading-rhel7-to-rhel8-main-official-doc * https://red.ht/upgrading-rhel8-to-rhel9-main-official-doc * https://red.ht/ipu-customisation-repos-known-issues * https://red.ht/leapp-dialogs * https://red.ht/file-systems-and-storage-removed-btrfs-rhel-8 * https://red.ht/rhel-8-migrate-postgresql-server * https://red.ht/rhel-8-python * https://red.ht/rhel-8-configuring-routing-protocols * https://red.ht/nm-settings-ifcfg-rh * https://red.ht/rhel-9-intel-microarchitectures * https://red.ht/rhel-9-blog-openssh-scp-deprecation Additional changes: * checktargetrepos: replace "TBA" by valid URL for 8 -> 9 docs * dropped unneeded pylint skips
Refactor the handling of kernel-related information away from using distributed ad-hoc logic based only on kernel release in IPUConfiguration. Instead, introduce the KernelInfo message providing rich information about the booted kernel. These changes also affect the information about the target kernel which previously only included target kernel's nevra that was misleadingly marked as 'version'. The new target kernel info message also contains paths to frequently used files such as the kernel image path and initramfs location. All old functionality has been kept in place, but deprecated. Deprecated: * `leapp.libraries.common.config.version.is_rhel_realtime` (check type in msg `KernelInfo`, field `type`) * `leapp.models.InstalledTargetKernelVersion` (replaced by `InstalledTargetKernelInfo`) Jira ref: OAMG-9452
…ownload drop As the leapp upgrade data files are nowadays part of the install rpm, there is no need to download them anymore. Also, we plan to drop the service providing the data files online in future. For that reason, update all texts and related error messages so people are not instructed to visit obsoleted article and do not try to apply invalid (obsoleted) data files anymore. Right now, we are keeping the functionality for the data files download, but the fetch functino is already updated and prepared to stop trying to download the files if not present. As we have the functionality already present, I think we should keep a possibility of the download for additional custom data files (not provided by us) in custom actors, but for the official data files we will require them in future to be present locally only. NOTE: regarding another planned changes soon, skipping update of unit-tests Co-authored-by: Michal Hečko <[email protected]>
Leapp introduced new output apis (and removed some other). See oamg/leapp#840 for details. The calls to `generate_report_file()` in `commands/upgrade/utils.py` are swapped to workaround a bug in the framework where the json report generation modifies messages passed to it. Jira: OAMG-9663
## Packaging - Requires leapp-framework 5.0 ## Upgrade handling ### Fixes - Add el8toel9 actor to handle directory -> symlink with ruby IRB. (oamg#1076) - Do not try to update GRUB core on IBM Z systems (oamg#1117) - Fix failing upgrades with devtmpfs file systems specified in FSTAB (oamg#1090) - Fix the calculation of the required free space on each partitions/volume for the upgrade transactions (oamg#1097) - Fix the generation of the report about hybrid images (oamg#1064) - Handle correctly the installed certificates to allow upgrades with custom repositories using HTTPs with enabled SSL verification (oamg#1106) - Minor improvements and fixes of various reports (oamg#1066, oamg#1067, oamg#1085) - Update error messages about leapp data files to inform user how to obtain valid data files (oamg#1121) - Update links in various reports (oamg#1062, oamg#1086) - Update the repomap data to cover changed repoids in RHUI Azure (oamg#1087) - [IPU 7 -> 8] Fix false positive report about invalid symlinks on RHEL 7 (oamg#1052) - [IPU 8 -> 9] Inhibit the upgrade when unsupported x86-64 microarchitecture is detected (oamg#1059) ### Enhancements - Include updated leapp data files in the RPM (oamg#1046, oamg#1092, oamg#1119) - Update the set of supported upgrade paths (oamg#1077): - RHEL with SAP HANA 7.9 -> 8.6, 8.8 (default: 8.6) - RHEL with SAP HANA 8.8 -> 9.2 - Introduce new upgrade paths: - RHEL 7.9 -> 8.9 (default) - RHEL 8.9 -> 9.3 - Correctly update grub2 when /boot resides on multiple devices aggregated in RAID (oamg#1093, oamg#1115) - Enable upgrades for machines using RHUI on AlibabaCloud (oamg#1088) - Introduce possibility to add kernel drivers to initramfs (oamg#1081) - Redesign handling of information about kernel (booted and target) in preparation for new changes in RHEL 9 (oamg#1107) - Redesign source system overlay to use disk images backed by sparse files to optimize disk space consumption (oamg#1097, oamg#1103) - Requires leapp-framework 5.0 (oamg#1061, oamg#1116) - Use new leapp CLI API which provides better report summary output (oamg#1061, oamg#1116) - [IPU 8 -> 9] Detect and report use of deprecated Xorg drivers (oamg#1078) - [IPU 8 -> 9] Introduce IPU for systems with FIPS enabled (oamg#1053) ## Additional changes interesting for devels - Deprecated `GrubInfo.orig_device_name` field in the `GrubInfo` model (replaced by `GrubInfo.orig_devices`) (oamg#1093) - Deprecated `InstalledTargetKernelVersion` model (replaced by `InstalledTargetKernelInfo`) (oamg#1107) - Deprecated `leapp.libraries.common.config.version.is_rhel_realtime` (check the type in msg `KernelInfo`, field `type`) (oamg#1107) - Deprecated `leapp.libraries.common.grub.get_grub_device()` (replaced by `leapp.libraries.common.grub.get_grub_devices()`) (oamg#1093) - Introduced new devel envar LEAPP_DEVEL_KEEP_DISK_IMGS=1 to skip the removal of the created disk images for OVL. That's sometimes handy for the debugging. (oamg#1097)
Previously only used for alt-ruby31, further testing has shown that multiple other versions also have the same issues and need to be included until the problem is resolved on alt-ruby end.
We want to make it possible to upgrade the system with rollout repositories present and enabled, but at the moment the current Leapp upgrade structure makes that difficult to achieve. Some restructuring will be required - for now, we just tell the user to disable the rollout repos to be safe.
This reverts commit f44ec58.
We don't really need to tell the user to disable rollout repos. We can just ignore them, as any other third-party repos.
CLOS-2148: skip over enabled rollout repositories with a warning instead of including them
(cherry picked from commit 0f31a66)
(cherry picked from commit 3b0e3da)
…ices on 8->9+ Beginning with RHEL8 there are two possible configurations (monolithic and modular) for the libvirt and virt* (virtqemud, virtlockd, virtproxyd, ...) services. The generic strategy implemented in the transitionsystemdservicesstates actor cannot handle such setup and leaves the upgraded system with a an incorrect combination of these configurations. Therefore these services are ignored by the actor and their state transition will be done by the services (and other related units) themselves. Jira: RHEL-22597 (cherry picked from commit 5518ae0)
I've already realized I missed couple of breakpoints over time inside
the initramfs and cannot remember expected cmdline params for each
of them. So let's try to make it more consistent. Keeping the original
cmdline arguments as some people are used to them already, but
introducing new ones.
New introduced breakpoints:
* rd.upgrade.break=leapp-initram
* breaks right after getting to leapp dracut modules
* orig
* rd.break=upgrade
* rd.upgrade.break=upgrade
* rd.upgrade.break=leapp-pre-upgrade
* breaks just right before running leapp in initramfs
* rd.upgrade.break=leapp-post-upgrade
* running just right after leapp upgrade (dnf transaction done),
before the start of post-upgrade phases
* orig:
* rd.break=leapp-upgrade
* rd.upgrade.break=leapp-upgrade
* rd.upgrade.break=leapp-finish
* the last breakpoint, after logs are saved, before the upcoming
reboot (leaving the upgrade environment)
* orig:
* rd.break=leapp-logs
Note: we could possibly drop old ones already, but as this is
going to be one of last changes for IPU 7 -> 8, I decided to leave
such a decision for future. Also, it's possible that we will need
to redesign this part of the upgrade process, so that's another reason
why to just add something, but do not drop anything.
(cherry picked from commit 64c34b5)
## Packaging - Requires xfsprogs and e2fsprogs (oamg#1154) - Bump leapp-repository-dependencies to 10 (oamg#1154) ## Upgrade handling ### Fixes - Detect changes in openssl default configuration file and restore it to the default to the target default during the upgrade to reduce risk of potential issues (oamg#1131) - Do not try to download data files anymore when missing as the service is obsoleted since the data is part of installed packages (oamg#1120) - Drop the invalid `tuv` target channel (oamg#1130) - Fix handling of symlinks under /etc/pki when managing certificates (oamg#1135, oamg#1160, oamg#1166) - Fix semanage import issue (oamg#1164) - Fix the issue of going out of bounds in the isccfg parser (oamg#1124) - Fix traceback when saving the rhsm facts results and the /etc/rhsm/facts directory doesn’t exist yet (oamg#1132) - Handle the upgrade better when a proxy is configured in YUM/DNF configutations (oamg#1143) - Load all rpm repository substitutions that dnf knows about, not just releasever since repofiles may use the other substitutions too (oamg#1134) - Minor updates of generated reports (oamg#1151) - Print nice error msg when device and driver deprecation data is malformed (oamg#1168) - Report information about required manual steps after the upgrade when openssl-ibmca is installed (oamg#1131) - Update error messages and reports when installed upgrade data files are malformed or missing (oamg#1120) - [IPU 7 -> 8] Fix the upgrade of the RH Satellite server when tomcat is installed (oamg#1150) - [IPU 8 -> 9] Fix the upgrade from RHEL 8.9+ when the release is locked by subscription-manager (oamg#1136, oamg#1138) ### Enhancements - Update upgrade paths: (oamg#1146, oamg#1147, oamg#1175) - RHEL 7.9 -> 8.10, 8.8 (default: 8.10) - RHEL with SAPAHA 7.9 -> 8.10, 8.8 (default: 8.8) - RHEL w/o SAP HANA 8.8 -> 9.2 - RHEL w/o SAP HANA 8.10 -> 9.4 - Added possibility to define DNF configuration for the target system (oamg#1143) - Code cleaning: drop redundant and invalid NFS checks (oamg#1127) - Default to NO_RHSM mode when subscription-manager is not found (oamg#1133) - Detect customized configuration of dynamic linker (oamg#1118) - Detect possible unexpected RPM GPG keys has been installed during RPM transaction (oamg#1101) - Drop obsoleted upgrade paths that relates to releases: 8.6, 8.9, 9.0, 9.3 (oamg#1175) - Ignore Leapp related PES events (oamg#1153) - Introduce generic transition of systemd services states during the IPU (oamg#1060, oamg#1174) - Introduce possibility to upgrade with local repositories (oamg#1099) - Introduced some changes getting us closer to possibility of IPU for Centos (Stream) systems (oamg#1140) - Report the upgrade customisations and modifications of the upgrade tooling (oamg#1148) - Simplify handling of upgrades on systems using RHUI, reducing the maintenance burden for cloud providers (oamg#1057) - Update the leapp upgrade data files - bump data stream to "3.0" (oamg#1163, oamg#1165, oamg#1170) - [IPU 8 -> 9] Enable upgrades RHEL 8 -> 9 using RHUI on Alibaba cloud (oamg#1137, oamg#1165, oamg#1172) - Unify breakpoints inside the upgrade initramfs for the easier troubleshooting (oamg#1157) ## Additional changes interesting for devels - Introduced new functions returning a list of packages related to upgrade - see the rpms library (oamg#1156) - Make detection of installed signed packages distribution agnostic - covers RHEL & CentOS (oamg#876) - Model InstalledRedHatSignedRPM is deprecated, replaced by DistributionSignedRPM (oamg#876) (cherry picked from commit 6421225)
There will be another update of data duing the next week(s), so keeping expected provided_data_stream set on 2.0. However the data are already marked to provide 2.0 & 3.0. (cherry picked from commit 0e51aeb)
* Added RHEL 9 repos for Alibaba RHUI with mapping for IPU 8 -> 9 * Actors require "3.0" in the list of provided_data_streams * All data files updated to provide onls "3.0" data stream * Add NL at the end of the device_driver_deprecation_data.json file to be POSIX compatible as expected. (cherry picked from commit 8552bbf)
Includes fixed idm-tomcatjss related events for upgrades IPU 8 -> 9. Jira: RHEL-21779 (cherry picked from commit b75dc49)
``` The following repos have been added: - Repo(pesid='rhel8-CRB', major_version='8', repoid='rhui-codeready-builder-for-rhel-8-aarch64-rhui-rpms', repo_type='rpm', channel='ga', arch='aarch64', rhui='alibaba') - Repo(pesid='rhel8-rhui-custom-client-at-alibaba', major_version='8', repoid='rhui-custom-rhui_client_at_alibaba-rhel-8', repo_type='rpm', channel='ga', arch='aarch64', rhui='alibaba') - Repo(pesid='rhel8-BaseOS', major_version='8', repoid='rhui-rhel-8-for-aarch64-baseos-rhui-rpms', repo_type='rpm', channel='ga', arch='aarch64', rhui='alibaba') - Repo(pesid='rhel8-Supplementary', major_version='8', repoid='rhui-rhel-8-for-aarch64-supplementary-rhui-rpms', repo_type='rpm', channel='ga', arch='aarch64', rhui='alibaba') - Repo(pesid='rhel8-AppStream', major_version='8', repoid='rhui-rhel-8-for-aarch64-appstream-rhui-rpms', repo_type='rpm', channel='ga', arch='aarch64', rhui='alibaba') ``` Related PR: oamg#1137 (cherry picked from commit c63ccbb)
Update to version 0.20.0
Add ELevate key SIG to all distros.
- gpg-pubkey-73e3b907-6581b071 gpg(PostgreSQL RPM Repository <[email protected]>)
Add one of PostgreSQL's keys to the list of obsoleted ones for 9
Add ELS key SIG to all distros.
…malinux-ng # Conflicts: # README.md # commands/upgrade/util.py # packaging/leapp-repository.spec # repos/system_upgrade/common/actors/checkenabledvendorrepos/actor.py # repos/system_upgrade/common/actors/checketcreleasever/libraries/checketcreleasever.py # repos/system_upgrade/common/actors/checkgrubcore/actor.py # repos/system_upgrade/common/actors/commonleappdracutmodules/files/dracut/85sys-upgrade-redhat/do-upgrade.sh # repos/system_upgrade/common/actors/commonleappdracutmodules/files/dracut/90sys-upgrade/initrd-system-upgrade-generator # repos/system_upgrade/common/actors/efibootorderfix/finalization/actor.py # repos/system_upgrade/common/actors/filterrpmtransactionevents/actor.py # repos/system_upgrade/common/actors/kernel/checkinstalledkernels/libraries/checkinstalledkernels.py # repos/system_upgrade/common/actors/peseventsscanner/actor.py # repos/system_upgrade/common/actors/peseventsscanner/libraries/peseventsscanner.py # repos/system_upgrade/common/actors/peseventsscanner/tests/unit_test_peseventsscanner.py # repos/system_upgrade/common/actors/redhatsignedrpmscanner/actor.py # repos/system_upgrade/common/actors/removeobsoletegpgkeys/libraries/removeobsoleterpmgpgkeys.py # repos/system_upgrade/common/actors/repositoriesmapping/libraries/repositoriesmapping.py # repos/system_upgrade/common/actors/scancustomrepofile/tests/test_scancustomrepofile.py # repos/system_upgrade/common/actors/scanvendorrepofiles/actor.py # repos/system_upgrade/common/actors/scanvendorrepofiles/libraries/scanvendorrepofiles.py # repos/system_upgrade/common/actors/setuptargetrepos/libraries/setuptargetrepos.py # repos/system_upgrade/common/actors/targetuserspacecreator/libraries/userspacegen.py # repos/system_upgrade/common/actors/vendorreposignaturescanner/actor.py # repos/system_upgrade/common/actors/vendorrepositoriesmapping/libraries/vendorrepositoriesmapping.py # repos/system_upgrade/common/libraries/config/version.py # repos/system_upgrade/common/libraries/dnfconfig.py # repos/system_upgrade/common/libraries/dnfplugin.py # repos/system_upgrade/common/libraries/overlaygen.py # repos/system_upgrade/common/libraries/repomaputils.py # repos/system_upgrade/common/libraries/rpms.py # repos/system_upgrade/el7toel8/actors/checkleftoverpackages/actor.py # repos/system_upgrade/el7toel8/actors/networkmanagerupdateconnections/tools/nm-update-client-ids.py # repos/system_upgrade/el7toel8/actors/opensshpermitrootlogincheck/actor.py # repos/system_upgrade/el7toel8/actors/opensshpermitrootlogincheck/libraries/opensshpermitrootlogincheck.py # repos/system_upgrade/el7toel8/actors/satellite_upgrade_facts/actor.py # repos/system_upgrade/el7toel8/actors/satellite_upgrade_facts/tests/unit_test_satellite_upgrade_facts.py
These files were originally added by RHEL. Neither Alma nor CloudLinux uses them. device_driver_deprecation_data.json pes-events.json repomap.json Files were removed in order not to confuse people.
Mute broken upstream tests and fix whatever I can do fast. Result: # REPOSITORIES="common,el7toel8" make test_no_lint == 1871 passed, 53 skipped in 39.92 seconds ==
Importing modules from another repository is restricted with latest leapp.
Rebase leapp-repository on top of almalinux latest version
Thank you for contributing to the Leapp project!Please note that every PR needs to comply with the Leapp Guidelines and must pass all tests in order to be mergable.
To launch regression testing public members of oamg organization can leave the following comment:
Please open ticket in case you experience technical problem with the CI. (RH internal only) Note: In case there are problems with tests not being triggered automatically on new PR/commit or pending for a long time, please consider rerunning the CI by commenting leapp-ci build (might require several comments). If the problem persists, contact leapp-infra. |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
There was a problem hiding this comment.
ShellCheck found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add package requirements to update dependecies automatically.