SONIC Source Code Compilation

[sarthakg0191]

I am compiling the SONIC source code in my Ubuntu 22.04 system by issuing the following commands
make ini,
make configure PLATFORM=vs,
make SONIC_BUILD_JOBS=4 all)
just to see whether I am able to compile or not but it is giving error in compilation. Can anybody tell me the right steps to compile so that it will be compiled successfully?
As I am exploring about SONIC so I don't know much about it so I will be grateful if anyone is ready to help me.

[arista-hpandya]

Before you wish to setup the repo make sure you have the correct hardware requirements as mentioned in the README.md:

Multiple cores to increase build speed
Plenty of RAM (less than 8 GiB is likely to cause issues)
300G of free disk space
KVM Virtualization Support.

Once you have that the steps to build the repo are as follows:
sudo modprobe overlay
git checkout master
git submodule update --init --force --recursive
make init
make configure PLATFORM=vs

Now you can run make list and you will get a number of debian (.deb) and wheel (.whl) targets. Based on the section you want to work on, choose the relevant target. For example when working with sonic-utilities:

make NOSTRETCH=1 NOBUSTER=1 NOBULLSEYE=1 KEEP_SLAVE_ON=yes target/python-wheels/bookworm/sonic_utilities-1.2-py3-none-any.whl

This will get you within the slave docker, then you can run:
cd src/sonic-utilities/
python3 setup.py bdist_wheel

That is it, now you can modify the code and run unit tests. Most tests in SONiC are written using the pytest framework and have the setup.py file. You can use python3 setup.py test to run them.

[sandwichdoge]

I would also like to know if there's a confirmed distro that works for building the image from source. I've been trying building for broadcom and vs, on a physical machine, on a VM, in Ubuntu 22 LTS, Ubuntu 23. Got different errors every time, such as building all with -j4 would fail some targets and I had to build that single target only.
However the Azure pipeline is passing so I can only guess it's inconsistency between build machines.

[arista-hpandya]

Here are the OS and firmware versions of my VM on which I have successfully built sonic:

Operating System: AlmaLinux 9.4 (Seafoam Ocelot)      
CPE OS Name: cpe:/o:almalinux:almalinux:9::baseos
Kernel: Linux 5.14.0-427.13.1.el9_4.x86_64
Architecture: x86-64
Firmware Version: 1.16.1-1.el9

[sandwichdoge]

Hello again, I tried AlmaLinux 9.4 but still no luck, got this error:

So I went back to Ubuntu 20.04, fresh installation.
Built with one job (to avoid potential race condition issues):

git clone --recurse-submodules https://github.com/sonic-net/sonic-buildimage.git -b 202305
sudo modprobe overlay

make init
make configure PLATFORM=vs
make ENABLE_TRANSLIB_WRITE=y SONIC_BUILD_JOBS=1 target/sonic-vs.img.gz

I got this error at the last step (sonic-vs.img.gz.log):

+ sudo https_proxy= LANG=C chroot ./fsroot-vs pip3 install --no-build-isolation pygobject
Collecting pygobject
  Downloading pygobject-3.50.0.tar.gz (1.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 2.1 MB/s eta 0:00:00
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
ERROR: Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/cli/base_command.py", line 105, in _run_wrapper
    status = _inner_run()
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/cli/base_command.py", line 96, in _inner_run
    return self.run(options, args)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/cli/req_command.py", line 67, in wrapper
    return func(self, options, args)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/commands/install.py", line 379, in run
    requirement_set = resolver.resolve(
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve
    result = self._result = resolver.resolve(
  File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/resolvelib/resolvers.py", line 397, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria
    if not criterion.candidates:
  File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/resolvelib/structs.py", line 156, in __bool__
    return bool(self._sequence)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 174, in __bool__
    return any(self)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 162, in <genexpr>
    return (c for c in iterator if id(c) not in self._incompatible_ids)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 53, in _iter_built
    candidate = func()
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/resolution/resolvelib/factory.py", line 186, in _make_candidate_from_link
    base: Optional[BaseCandidate] = self._make_base_candidate_from_link(
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/resolution/resolvelib/factory.py", line 232, in _make_base_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 303, in __init__
    super().__init__(
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 158, in __init__
    self.dist = self._prepare()
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 235, in _prepare
    dist = self._prepare_distribution()
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 314, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/operations/prepare.py", line 527, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/operations/prepare.py", line 642, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/operations/prepare.py", line 72, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/distributions/sdist.py", line 69, in prepare_distribution_metadata
    self.req.prepare_metadata()
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/req/req_install.py", line 575, in prepare_metadata
    self.metadata_directory = generate_metadata(
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/operations/build/metadata.py", line 35, in generate_metadata
    distinfo_dir = backend.prepare_metadata_for_build_wheel(metadata_dir)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/utils/misc.py", line 727, in prepare_metadata_for_build_wheel
    return super().prepare_metadata_for_build_wheel(
  File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/pyproject_hooks/_impl.py", line 186, in prepare_metadata_for_build_wheel
    return self._call_hook('prepare_metadata_for_build_wheel', {
  File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/pyproject_hooks/_impl.py", line 321, in _call_hook
    raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pyproject_hooks._impl.BackendUnavailable: Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend
    obj = import_module(mod_path)
  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'mesonpy'

+ clean_sys
+ sudo chroot ./fsroot-vs umount /sys/fs/cgroup/blkio /sys/fs/cgroup/cpu /sys/fs/cgroup/cpu,cpuacct /sys/fs/cgroup/cpuacct /sys/fs/cgroup/cpuset /sys/fs/cgroup/devices /sys/fs/cgroup/freezer /sys/fs/cgroup/hugetlb /sys/fs/cgroup/memory /sys/fs/cgroup/misc /sys/fs/cgroup/net_cls /sys/fs/cgroup/net_cls,net_prio /sys/fs/cgroup/net_prio /sys/fs/cgroup/perf_event /sys/fs/cgroup/pids /sys/fs/cgroup/rdma /sys/fs/cgroup/systemd /sys/fs/cgroup /sys
umount: /sys/fs/cgroup/cpu: no mount point specified.
umount: /sys/fs/cgroup/cpu,cpuacct: no mount point specified.
umount: /sys/fs/cgroup/cpuacct: no mount point specified.
umount: /sys/fs/cgroup/net_cls: no mount point specified.
umount: /sys/fs/cgroup/net_cls,net_prio: no mount point specified.
umount: /sys/fs/cgroup/net_prio: no mount point specified.
umount: /sys/fs/cgroup/systemd: no mount point specified.
+ true
+ true
+ sudo LANG=C chroot ./fsroot-vs umount /proc
+ true

How is the Azure CICD pipeline passing? Is it doing something different than the build manual?

[philo-micas]

@sandwichdoge We also encountered this issue while compiling the 202305 branch. Have you solved it? Can you share it? thank you!

[sandwichdoge]

@philo-micas I gave up on 202305, then successfully built on branch 202405, this commit specifically 0017365. I also had to spin up a new build machine because apparently make clean doesn't clean up all the artifacts in the build machine and that gave me some random errors.

[yaoting-chang]

Met the same issue on branch 202311. Please help this issue.

+ sudo https_proxy= LANG=C chroot ./fsroot-broadcom pip3 install --no-build-isolation pygobject
Collecting pygobject
  Downloading pygobject-3.50.0.tar.gz (1.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 486.4 kB/s eta 0:00:00
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
ERROR: Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/cli/base_command.py", line 105, in _run_wrapper
    status = _inner_run()
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/cli/base_command.py", line 96, in _inner_run
    return self.run(options, args)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/cli/req_command.py", line 67, in wrapper
    return func(self, options, args)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/commands/install.py", line 379, in run
    requirement_set = resolver.resolve(
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve
    result = self._result = resolver.resolve(
  File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/resolvelib/resolvers.py", line 397, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria
    if not criterion.candidates:
  File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/resolvelib/structs.py", line 156, in __bool__
    return bool(self._sequence)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 174, in __bool__
    return any(self)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 162, in <genexpr>
    return (c for c in iterator if id(c) not in self._incompatible_ids)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 53, in _iter_built
    candidate = func()
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/resolution/resolvelib/factory.py", line 187, in _make_candidate_from_link
    base: Optional[BaseCandidate] = self._make_base_candidate_from_link(
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/resolution/resolvelib/factory.py", line 233, in _make_base_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 304, in __init__
    super().__init__(
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 159, in __init__
    self.dist = self._prepare()
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 236, in _prepare
    dist = self._prepare_distribution()
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 315, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/operations/prepare.py", line 527, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/operations/prepare.py", line 642, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/operations/prepare.py", line 72, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/distributions/sdist.py", line 69, in prepare_distribution_metadata
    self.req.prepare_metadata()
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/req/req_install.py", line 575, in prepare_metadata
    self.metadata_directory = generate_metadata(
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/operations/build/metadata.py", line 35, in generate_metadata
    distinfo_dir = backend.prepare_metadata_for_build_wheel(metadata_dir)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/utils/misc.py", line 722, in prepare_metadata_for_build_wheel
    return super().prepare_metadata_for_build_wheel(
  File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/pyproject_hooks/_impl.py", line 186, in prepare_metadata_for_build_wheel
    return self._call_hook('prepare_metadata_for_build_wheel', {
  File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/pyproject_hooks/_impl.py", line 321, in _call_hook
    raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pyproject_hooks._impl.BackendUnavailable: Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend
    obj = import_module(mod_path)
  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'mesonpy'

+ clean_sys
+ sudo chroot ./fsroot-broadcom umount /sys/fs/cgroup/blkio /sys/fs/cgroup/cpu /sys/fs/cgroup/cpu,cpuacct /sys/fs/cgroup/cpuacct /sys/fs/cgroup/cpuset /sys/fs/cgroup/devices /sys/fs/cgroup/freezer /sys/fs/cgroup/hugetlb /sys/fs/cgroup/memory /sys/fs/cgroup/misc /sys/fs/cgroup/net_cls /sys/fs/cgroup/net_cls,net_prio /sys/fs/cgroup/net_prio /sys/fs/cgroup/perf_event /sys/fs/cgroup/pids /sys/fs/cgroup/rdma /sys/fs/cgroup/systemd /sys/fs/cgroup /sys
umount: /sys/fs/cgroup/cpu: no mount point specified.
umount: /sys/fs/cgroup/cpu,cpuacct: no mount point specified.
umount: /sys/fs/cgroup/cpuacct: no mount point specified.
umount: /sys/fs/cgroup/net_cls: no mount point specified.
umount: /sys/fs/cgroup/net_cls,net_prio: no mount point specified.
umount: /sys/fs/cgroup/net_prio: no mount point specified.
umount: /sys/fs/cgroup/systemd: no mount point specified.
+ true
+ true
+ sudo LANG=C chroot ./fsroot-broadcom umount /proc
+ true
[  FAIL LOG END  ] [ target/sonic-broadcom.bin ]
make: *** [slave.mk:1369: target/sonic-broadcom.bin] Error 1
make[1]: *** [Makefile.work:622: target/sonic-broadcom.bin] Error 2
make[1]: Leaving directory '/media/eric/test/dd-s64c8v2/sonic-buildimage'
make: *** [Makefile:41: target/sonic-broadcom.bin] Error 2

[Seitanas]

Fixable by manually altering files/build_templates/sonic_debian_extension.j2:
#20794 (comment)