DPDK: install path fixes for meson and Ubuntu 24.04 #3598

mcgov · 2025-01-13T22:57:13Z

Fixing some bugs which were noted after testing on Ubuntu 24.04:

'meson' is occasionally packaged as 'meson' but it is being standardized as 'python3-meson' in future releases. This PR fixes the installation path for Meson to account for an prefer the packaged version if it's available and the correct version.
linux-modules-extra-azure requires a restart, dependencies occasionally do. So add a 'needs_restart' parameter to the dependency installer to allow rebooting before continuing to install dependencies.

lisa/operating_system.py

microsoft/testsuites/dpdk/dpdktestpmd.py

lisa/tools/meson.py

lisa/operating_system.py

LiliDeng · 2025-01-23T07:40:02Z

I still saw issues when run test against canonical ubuntu-24_04-lts server 24.04.202411030

2025-01-23T03:45:57.2367877Z 2025-01-23 03:45:57.233[5488][DEBUG] lisa.env[generated_0].node[0].cmd[2463] cmd: ['pip3', 'install', '-q', 'pyelftools'], cwd: None, shell: False, sudo: False, nohup: False, posix: True, remote: True, encoding: utf-8
2025-01-23T03:45:57.5747063Z 2025-01-23 03:45:57.573[5492][DEBUG] lisa.env[generated_0].node[0].cmd[2463].stdout error: externally-managed-environment
2025-01-23T03:45:57.5752932Z 2025-01-23 03:45:57.573[5492][DEBUG] lisa.env[generated_0].node[0].cmd[2463].stdout � This environment is externally managed
2025-01-23T03:45:57.5757649Z 2025-01-23 03:45:57.573[5492][DEBUG] lisa.env[generated_0].node[0].cmd[2463].stdout \u2570\u2500> To install Python packages system-wide, try apt install
2025-01-23T03:45:57.5762505Z 2025-01-23 03:45:57.573[5492][DEBUG] lisa.env[generated_0].node[0].cmd[2463].stdout     python3-xyz, where xyz is the package you are trying to
2025-01-23T03:45:57.5765328Z 2025-01-23 03:45:57.573[5492][DEBUG] lisa.env[generated_0].node[0].cmd[2463].stdout     install.
2025-01-23T03:45:57.5771737Z 2025-01-23 03:45:57.573[5492][DEBUG] lisa.env[generated_0].node[0].cmd[2463].stdout     If you wish to install a non-Debian-packaged Python package,
2025-01-23T03:45:57.5778365Z 2025-01-23 03:45:57.573[5492][DEBUG] lisa.env[generated_0].node[0].cmd[2463].stdout     create a virtual environment using python3 -m venv path/to/venv.
2025-01-23T03:45:57.5783775Z 2025-01-23 03:45:57.573[5492][DEBUG] lisa.env[generated_0].node[0].cmd[2463].stdout     Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
2025-01-23T03:45:57.5788002Z 2025-01-23 03:45:57.573[5492][DEBUG] lisa.env[generated_0].node[0].cmd[2463].stdout     sure you have python3-full installed.
2025-01-23T03:45:57.5794100Z 2025-01-23 03:45:57.573[5492][DEBUG] lisa.env[generated_0].node[0].cmd[2463].stdout     If you wish to install a non-Debian packaged Python application,
2025-01-23T03:45:57.5799329Z 2025-01-23 03:45:57.573[5492][DEBUG] lisa.env[generated_0].node[0].cmd[2463].stdout     it may be easiest to use pipx install xyz, which will manage a
2025-01-23T03:45:57.5804530Z 2025-01-23 03:45:57.573[5492][DEBUG] lisa.env[generated_0].node[0].cmd[2463].stdout     virtual environment for you. Make sure you have pipx installed.
2025-01-23T03:45:57.5810086Z 2025-01-23 03:45:57.573[5492][DEBUG] lisa.env[generated_0].node[0].cmd[2463].stdout     See /usr/share/doc/python3.12/README.venv for more information.
2025-01-23T03:45:57.5822285Z 2025-01-23 03:45:57.573[5492][DEBUG] lisa.env[generated_0].node[0].cmd[2463].stdout note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
2025-01-23T03:45:57.5825567Z 2025-01-23 03:45:57.573[5492][DEBUG] lisa.env[generated_0].node[0].cmd[2463].stdout hint: See PEP 668 for the detailed specification.
2025-01-23T03:45:57.6262525Z 2025-01-23 03:45:57.625[5488][DEBUG] lisa.env[generated_0].node[0].cmd[2463] execution time: 0.389 sec, exit code: 1

lisa/operating_system.py

squirrelsc · 2025-01-24T04:25:18Z

lisa/operating_system.py

+        install_result = self._node.execute(
+            command, shell=True, sudo=True, timeout=timeout
+        )
+        assert_that(install_result.exit_code).described_as(


install_result.assert_exist_code implemented the similar error message, please reuse it.

squirrelsc · 2025-01-24T04:28:33Z

lisa/operating_system.py

+        remove_packages = " ".join(packages)
+        command += f" rm {remove_packages}"
+        self.wait_running_process("zypper")
+        install_result = self._node.execute(


install_result -> uninstall_result

squirrelsc · 2025-01-24T04:31:37Z

lisa/tools/meson.py

@@ -15,13 +15,18 @@


 class Meson(Tool):
+    _minimum_version = VersionInfo(major=0, minor=52, patch=0)


use lisa.util.parse_version.

squirrelsc · 2025-01-24T04:33:07Z

lisa/tools/meson.py

+            if posix_os.package_exists(pkg, minimum_version=self._minimum_version):
+                package_installed = pkg
+                break
+            elif posix_os.is_package_in_repo(pkg):


elif to if is the same, but it means the two if blocks are actually independent.

Undoing a change I added which enabled a _minimum_version check on _check_exists. This is a bad system because you'll end up needing to do things like this: if not posix_os._check_exists(pkg, minimum_version=...): # Now what? Was it installed or not? # Was it installed and just the wrong version? The ultimate fix should be is_package_in_repo(pkg, minimum_version) Since we should be checking if an available package is the right version _before_ installing it blindly. This all seems out-of-scope for fixing a bug installing meson in _just_ the DPDK suite. So, I'm removing it. We can move the fixed install pattern to the operating_system.py level later if anyone else needs it.

mcgov · 2025-01-29T00:12:56Z

I still saw issues when run test against canonical ubuntu-24_04-lts server 24.04.202411030
@LiliDeng Correct! Fixed this with the latest two commits. Apologies.

mcgov added 3 commits January 13, 2025 14:39

Meson: try both 'meson' and 'python3-meson'

75180f2

DPDK: Add needs_reboot option to installer

08df703

Suse: implement _uninstall_packages

edba78f

squirrelsc reviewed Jan 14, 2025

View reviewed changes

lisa/operating_system.py Outdated Show resolved Hide resolved

Suse: i guess just 0

d5eca26

squirrelsc reviewed Jan 15, 2025

View reviewed changes

microsoft/testsuites/dpdk/dpdktestpmd.py Show resolved Hide resolved

squirrelsc reviewed Jan 15, 2025

View reviewed changes

lisa/tools/meson.py Outdated Show resolved Hide resolved

squirrelsc reviewed Jan 15, 2025

View reviewed changes

lisa/tools/meson.py Show resolved Hide resolved

squirrelsc reviewed Jan 15, 2025

View reviewed changes

lisa/operating_system.py Outdated Show resolved Hide resolved

mcgov added 4 commits January 16, 2025 13:47

Meson: restructure and comment installation options

a4fd67c

package_exists: add minimum_version

06a685d

rdma-core: check for existence before removal

6520c65

Dpdk: move pyelftools to single installation point

9a02ce2

squirrelsc reviewed Jan 17, 2025

View reviewed changes

lisa/operating_system.py Outdated Show resolved Hide resolved

Minimum version add debug log if no match

1fad9f0

mcgov force-pushed the mcgov/pyelf branch from 6a98033 to 1fad9f0 Compare January 21, 2025 17:20

OS SUSE uninstall: use assert_that

13ed81a

mcgov marked this pull request as ready for review January 21, 2025 18:52

mcgov requested a review from LiliDeng as a code owner January 21, 2025 18:52

OS packages, fix printing of List[str] of packages

2f1faab

squirrelsc reviewed Jan 24, 2025

View reviewed changes

lisa/operating_system.py Outdated Show resolved Hide resolved

squirrelsc reviewed Jan 24, 2025

View reviewed changes

mcgov added 4 commits January 27, 2025 07:45

make package_exists version check simpler

3371aca

DPDK: prefer newer pyelftools package

1b4c29b

Use helper function instead of instantiating the class directly

2979507

DPDK: linux-modules-extra-azure reboot

815b612

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DPDK: install path fixes for meson and Ubuntu 24.04 #3598

DPDK: install path fixes for meson and Ubuntu 24.04 #3598

mcgov commented Jan 13, 2025 •

edited

Loading

LiliDeng commented Jan 23, 2025 •

edited

Loading

squirrelsc Jan 24, 2025

squirrelsc Jan 24, 2025

squirrelsc Jan 24, 2025

squirrelsc Jan 24, 2025

mcgov commented Jan 29, 2025

		@@ -15,13 +15,18 @@


		class Meson(Tool):
		_minimum_version = VersionInfo(major=0, minor=52, patch=0)

DPDK: install path fixes for meson and Ubuntu 24.04 #3598

Are you sure you want to change the base?

DPDK: install path fixes for meson and Ubuntu 24.04 #3598

Conversation

mcgov commented Jan 13, 2025 • edited Loading

LiliDeng commented Jan 23, 2025 • edited Loading

squirrelsc Jan 24, 2025

Choose a reason for hiding this comment

squirrelsc Jan 24, 2025

Choose a reason for hiding this comment

squirrelsc Jan 24, 2025

Choose a reason for hiding this comment

squirrelsc Jan 24, 2025

Choose a reason for hiding this comment

mcgov commented Jan 29, 2025

mcgov commented Jan 13, 2025 •

edited

Loading

LiliDeng commented Jan 23, 2025 •

edited

Loading