Could not build wheels for faster-fifo, which is required to install pyproject.toml-based projects

[muhammad-faizan-122]

Hi, I am facing following issue while installing faster-fifo library.
python version: 3.9.16
usage: pip install faster-fifo following this faster-fifo repo.
OS: Ubuntu 22.04.2 LTS

Building wheels for collected packages: faster-fifo
  Building wheel for faster-fifo (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for faster-fifo (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [30 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-39
      creating build/lib.linux-x86_64-cpython-39/faster_fifo_reduction
      copying faster_fifo_reduction/__init__.py -> build/lib.linux-x86_64-cpython-39/faster_fifo_reduction
      creating build/lib.linux-x86_64-cpython-39/cpp_faster_fifo
      copying cpp_faster_fifo/__init__.py -> build/lib.linux-x86_64-cpython-39/cpp_faster_fifo
      creating build/lib.linux-x86_64-cpython-39/cpp_faster_fifo/tests
      copying cpp_faster_fifo/tests/test_faster_fifo.py -> build/lib.linux-x86_64-cpython-39/cpp_faster_fifo/tests
      copying cpp_faster_fifo/tests/comparison_tests.py -> build/lib.linux-x86_64-cpython-39/cpp_faster_fifo/tests
      copying cpp_faster_fifo/tests/__init__.py -> build/lib.linux-x86_64-cpython-39/cpp_faster_fifo/tests
      running build_ext
      building 'faster_fifo' extension
      creating build/temp.linux-x86_64-cpython-39
      creating build/temp.linux-x86_64-cpython-39/cpp_faster_fifo
      creating build/temp.linux-x86_64-cpython-39/cpp_faster_fifo/cpp_lib
      gcc -pthread -B /home/user/miniconda3/envs/my-env/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /home/user/miniconda3/envs/my-env/include -I/home/user/miniconda3/envs/my-env/include -fPIC -O2 -isystem /home/user/miniconda3/envs/my-env/include -fPIC -I. -Icpp_faster_fifo/cpp_lib -I/home/user/miniconda3/envs/my-env/include/python3.9 -c cpp_faster_fifo/cpp_lib/faster_fifo.cpp -o build/temp.linux-x86_64-cpython-39/cpp_faster_fifo/cpp_lib/faster_fifo.o -std=c++11
      In file included from /usr/include/stdint.h:27,
                       from /usr/include/c++/11/cstdint:41,
                       from /usr/include/c++/11/ratio:40,
                       from /usr/include/c++/11/chrono:39,
                       from /usr/include/c++/11/mutex:39,
                       from cpp_faster_fifo/cpp_lib/faster_fifo.cpp:1:
      /usr/include/x86_64-linux-gnu/bits/types.h:141:10: fatal error: bits/typesizes.h: No such file or directory
        141 | #include <bits/typesizes.h>     /* Defines __*_T_TYPE macros.  */
            |          ^~~~~~~~~~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for faster-fifo
Failed to build faster-fifo
ERROR: Could not build wheels for faster-fifo, which is required to install pyproject.toml-based projects

If anyone could offer some guidance on how to get faster-fifo installed, I would greatly appreciate it. Thank you!

[alex-petrenko]

Hi @muhammad-faizan-122
Thank you for reporting this!

Looks like some files for the GNU C-library are missing. I found a similar error here: https://stackoverflow.com/questions/33559884/fatal-error-bits-types-h-file-not-found

Can you try installing these packages to see if this fixes the issue? Or their alternative versions available in 22.04?

Normally it's sufficient to just install build-essentials and the compiler, but maybe something recently changed in Ubuntu 22.

[muhammad-faizan-122]

HI @alex-petrenko,
Thanks for quick response. I have installed build-essentials for my Ubuntu version (22.04.2). Then I tried to install faster-fifo library facing following issue.

Collecting faster-fifo
  Using cached faster-fifo-1.4.5.tar.gz (87 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: setuptools>=45.2.0 in /home/user/miniconda3/envs/my-env/lib/python3.9/site-packages (from faster-fifo) (67.7.2)
Requirement already satisfied: cython>=0.29 in /home/user/miniconda3/envs/my-env/lib/python3.9/site-packages (from faster-fifo) (0.29.34)
Building wheels for collected packages: faster-fifo
  Building wheel for faster-fifo (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for faster-fifo (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [20 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-39
      creating build/lib.linux-x86_64-cpython-39/faster_fifo_reduction
      copying faster_fifo_reduction/__init__.py -> build/lib.linux-x86_64-cpython-39/faster_fifo_reduction
      creating build/lib.linux-x86_64-cpython-39/cpp_faster_fifo
      copying cpp_faster_fifo/__init__.py -> build/lib.linux-x86_64-cpython-39/cpp_faster_fifo
      creating build/lib.linux-x86_64-cpython-39/cpp_faster_fifo/tests
      copying cpp_faster_fifo/tests/test_faster_fifo.py -> build/lib.linux-x86_64-cpython-39/cpp_faster_fifo/tests
      copying cpp_faster_fifo/tests/comparison_tests.py -> build/lib.linux-x86_64-cpython-39/cpp_faster_fifo/tests
      copying cpp_faster_fifo/tests/__init__.py -> build/lib.linux-x86_64-cpython-39/cpp_faster_fifo/tests
      running build_ext
      building 'faster_fifo' extension
      creating build/temp.linux-x86_64-cpython-39
      creating build/temp.linux-x86_64-cpython-39/cpp_faster_fifo
      creating build/temp.linux-x86_64-cpython-39/cpp_faster_fifo/cpp_lib
      gcc -pthread -B /home/user/miniconda3/envs/my-env/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /home/user/miniconda3/envs/my-env/include -I/home/user/miniconda3/envs/my-env/include -fPIC -O2 -isystem /home/user/miniconda3/envs/my-env/include -fPIC -I. -Icpp_faster_fifo/cpp_lib -I/home/user/miniconda3/envs/my-env/include/python3.9 -c cpp_faster_fifo/cpp_lib/faster_fifo.cpp -o build/temp.linux-x86_64-cpython-39/cpp_faster_fifo/cpp_lib/faster_fifo.o -std=c++11
      error: command 'gcc' failed: No such file or directory
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for faster-fifo
Failed to build faster-fifo
ERROR: Could not build wheels for faster-fifo, which is required to install pyproject.toml-based projects

Can you please help me out to resolve this?

Hi @muhammad-faizan-122 Thank you for reporting this!

Looks like some files for the GNU C-library are missing. I found a similar error here: https://stackoverflow.com/questions/33559884/fatal-error-bits-types-h-file-not-found

Can you try installing these packages to see if this fixes the issue? Or their alternative versions available in 22.04?

Normally it's sufficient to just install build-essentials and the compiler, but maybe something recently changed in Ubuntu 22.

[alex-petrenko]

error: command 'gcc' failed: No such file or directory indicates that you don't have a compiler in your system. Must be a very new system! :)

Try:
sudo apt install --reinstall build-essential gcc g++

[muhammad-faizan-122]

error: command 'gcc' failed: No such file or directory indicates that you don't have a compiler in your system. Must be a very new system! :)

Try: sudo apt install --reinstall build-essential gcc g++

@alex-petrenko,

/usr/include/time.h:29:10: fatal error: stddef.h: No such file or directory
         29 | #include <stddef.h>
            |          ^~~~~~~~~~
      compilation terminated.
      error: Command "x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I. -Icpp_faster_fifo/cpp_lib -I/usr/include/python3.10 -c faster_fifo.cpp -o build/temp.linux-x86_64-3.10/faster_fifo.o -std=c++11" failed with exit status 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for faster-fifo
Failed to build faster-fifo
ERROR: Could not build wheels for faster-fifo, which is required to install pyproject.toml-based projects

[alex-petrenko]

@muhammad-faizan-122 You shouldn't be seeing problems like this.
Can you provide any information about your machine, i.e. what operating system it is, what you're using it for? Is it a VM, a Docker container, or a bare metal machine?

stddef.h is a very basic C-header that seems to be missing on your machine. Are you able to compile any C/C++ programs at all?

I found this old forum post about it https://ubuntuforums.org/showthread.php?t=674047

You need the libc6-dev package for stddef.h.
You can get that and other useful build packages with 'sudo apt-get install build-essential'.
In general you can look for the package that provides a header file with apt-file-
Code:
sudo apt-get install apt-file
sudo apt-file update
apt-file search stddef.h
You can also install the packages required to build a particular package from source by using 'apt-get build-dep pkg'.

Can you try installing libc6-dev ?

[naddeoa]

@alex-petrenko Have you guys ever considered vending out prebuilt wheels? I'm planning on using faster-fifo in my companies open source library (https://github.com/whylabs/whylogs) and I'm a bit worried about making our consumers have build-essential/gcc installed. It's ok most of the time but it can become a nuisance when you want to do things like use our library on Sagemaker (or other container environments) where the default images may not have gcc installed. That ends up forcing people to create their own container to install build-essential when they otherwise would have just used an off-the shelf container.

Thanks for the work btw. This repo is what enabled us to create a python based container that does CPU bound operations behind a rest interface. The built-in mp.Queue is waaaay slower.

[alex-petrenko]

@naddeoa on one hand requiring build-essentials is not such a big deal, on the other hand I understand your concern. It is nice to have clean dependencies with no requirements.

I have lots of commitments now and not in the position to actively work on most of my repos. If something like this can be made as a PR/contribution, I'd love to review it and integrate! This would be a very welcome contribution.

I've had bad experiences with prebuilt wheels in the past, so I'm trying to avoid them when I can (building something that will work reliably across many Linux distros is actually quite tricky).
But if you have the know-how please let me know!

Perhaps as a temporary solution for your customers, you can build and host the wheel yourself, i.e. in a repo fork?
Then it's as easy as pip install https://<your_link>/faster-fifo.whl

I am also happy to host such a wheel for the latest version but I cannot promise that I'll keep updating it for future releases.

[mantianjiang]

python version: 3.9
OS：Ubuntu 24.04 LTS
I have also encountered similar problems. Currently, there are two solutions:

1. Ubuntu returns to stable version 20
2. Restricted version:
   pip install faster-fifo==1.4.7
   pip install sample-factory==2.1.1