armhf: Debian 10: ERROR: Failed building wheel for cmake

[leamas]

As heading says: On Debian 10/buster trying to install cmake using pypi I run into problems:

Building wheel for cmake (PEP 517): started
  Building wheel for cmake (PEP 517): finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 /root/.local/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /tmp/tmp027pwjbz

       cwd: /tmp/pip-install-_r4xwown/cmake_82599890a8a04506a314588eb24cfeef

  Complete output (9 lines):
    File "/tmp/pip-build-env-jqdgcx1z/overlay/lib/python3.7/site-packages/skbuild/setuptools_wrap.py", line 564, in setup
      cmkr = cmaker.CMaker(cmake_executable)
    File "/tmp/pip-build-env-jqdgcx1z/overlay/lib/python3.7/site-packages/skbuild/cmaker.py", line 95, in __init__
      self.cmake_version = get_cmake_version(self.cmake_executable)
    File "/tmp/pip-build-env-jqdgcx1z/overlay/lib/python3.7/site-packages/skbuild/cmaker.py", line 82, in get_cmake_version
      "Problem with the CMake installation, aborting build. CMake executable is %s" % cmake_executable)
  Traceback (most recent call last):

  Problem with the CMake installation, aborting build. CMake executable is cmake
  ----------------------------------------
  ERROR: Failed building wheel for cmake

Failed to build cmake

ERROR: Could not build wheels for cmake which use PEP 517 and cannot be installed directly

This is part of a larger build. Complete logs at https://cloud.drone.io/leamas/shipdriver_pi/188/2/2

[mayeut]

cmake-python-distributions require cmake to build. From the logs, I'm guessing there's no previous installation of cmake.
see also #33 which might be relevant.

[leamas]

Sorry for long delay...

Nope, not that simple. cmake is present, although an old version. Also, there is a seemingly undeclared build dependency on scikit-build package which needs to be installed manually, otherwise the build breaks. Current error message:

 ERROR: Command errored out with exit status 1:
205s
1900   command: /usr/bin/python3 /root/.local/lib/python3.5/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /tmp/tmphpi4j6ti
205s
1901       cwd: /tmp/pip-install-ka0g08dv/cmake_36849a7ce17247d8a0ce2ae47eb8c1e5
205s
1902  Complete output (9 lines):
205s
1903    File "/tmp/pip-build-env-_s0r4lso/overlay/lib/python3.5/site-packages/skbuild/setuptools_wrap.py", line 564, in setup
205s
1904      cmkr = cmaker.CMaker(cmake_executable)
205s
1905    File "/tmp/pip-build-env-_s0r4lso/overlay/lib/python3.5/site-packages/skbuild/cmaker.py", line 95, in __init__
205s
1906      self.cmake_version = get_cmake_version(self.cmake_executable)
205s
1907    File "/tmp/pip-build-env-_s0r4lso/overlay/lib/python3.5/site-packages/skbuild/cmaker.py", line 82, in get_cmake_version
205s
1908      "Problem with the CMake installation, aborting build. CMake executable is %s" % cmake_executable)

Complete build logs here

[rhastie]

@leamas @mayeut - I am seeing a very similar issue when running pip3 install cmake on RaspberryPi (armhf)/Raspbian... I've tried several workarounds (eg. install wheel first and then using --no-use-pep517 or --no-build-isolation as pip options but no luck)...

I've tested same Dockerfile on x86_64 and ARM64 both with Ubuntu 18.04 and it runs perfectly...

Any help would be appreciated as compiling latest Cmake from source is very slow on a Pi

[stage1-build  4/17] RUN pip3 install cmake:
#7 11.00 Collecting cmake
#7 16.19   Downloading cmake-3.21.3.tar.gz (31 kB)
#7 16.30   Installing build dependencies: started
#7 47.36   Installing build dependencies: finished with status 'done'
#7 47.37   Getting requirements to build wheel: started
#7 50.13   Getting requirements to build wheel: finished with status 'done'
#7 50.15   Preparing wheel metadata (pyproject.toml): started
#7 52.88   Preparing wheel metadata (pyproject.toml): finished with status 'done'
#7 53.01 Building wheels for collected packages: cmake
#7 53.02   Building wheel for cmake (pyproject.toml): started
#7 55.46   Building wheel for cmake (pyproject.toml): finished with status 'error'
#7 55.47   ERROR: Command errored out with exit status 1:
#7 55.47    command: /usr/bin/python3 /usr/local/lib/python3.6/dist-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /tmp/tmpd5qy1dc5
#7 55.47        cwd: /tmp/pip-install-r6zsx9ma/cmake_030a301060a64e33bb067fa88584edc3
#7 55.47   Complete output (9 lines):
#7 55.47     File "/tmp/pip-build-env-8hqcoe66/overlay/lib/python3.6/site-packages/skbuild/setuptools_wrap.py", line 564, in setup
#7 55.47       cmkr = cmaker.CMaker(cmake_executable)
#7 55.47     File "/tmp/pip-build-env-8hqcoe66/overlay/lib/python3.6/site-packages/skbuild/cmaker.py", line 95, in __init__
#7 55.47       self.cmake_version = get_cmake_version(self.cmake_executable)
#7 55.47     File "/tmp/pip-build-env-8hqcoe66/overlay/lib/python3.6/site-packages/skbuild/cmaker.py", line 82, in get_cmake_version
#7 55.47       "Problem with the CMake installation, aborting build. CMake executable is %s" % cmake_executable)
#7 55.47   Traceback (most recent call last):
#7 55.47
#7 55.47   Problem with the CMake installation, aborting build. CMake executable is cmake
#7 55.47   ----------------------------------------
#7 55.47   ERROR: Failed building wheel for cmake
#7 55.47 Failed to build cmake
#7 55.47 ERROR: Could not build wheels for cmake, which is required to install pyproject.toml-based projects
------
Dockerfile:24
--------------------
  22 |     ## Install latest CMake version
  23 |     RUN python3 -m pip install --upgrade pip setuptools wheel
  24 | >>> RUN pip3 install cmake
  25 |     ## Install latest Conan version
  26 |     RUN pip3 install conan
--------------------
error: failed to solve: rpc error: code = Unknown desc = process "/bin/sh -c pip3 install cmake" did not complete successfully: exit code: 1
make: *** [Makefile:21: buildx] Error 1

[leamas]

@rhastie:

Looks like the same bug.
For now, we work around by using pre-compiled binary packages, the build script has:

# Temporary fix until 3.19+  is available as a pypi package
# 3.19 is needed: https://gitlab.kitware.com/cmake/cmake/-/issues/20568
url='https://dl.cloudsmith.io/public/alec-leamas/opencpn-plugins-stable/deb/debian'
wget $url/pool/${OCPN_TARGET/-*/}/main/c/cm/cmake-data_3.19.3-0.1_all.deb
wget $url/pool/${OCPN_TARGET/-*/}/main/c/cm/cmake_3.19.3-0.1_armhf.deb
sudo apt install ./cmake_3.19.3-0.1_armhf.deb ./cmake-data_3.19.3-0.1_all.deb

$OCPN_TARGET expands to "buster"

[leamas]

@rhastie:
The more I look into this, I wonder if the pypi package is the correct solution. The main problem seems to be that the armhf binary isn't included in the wheel and the installation thus defaults to building from source. But, as you say, building cmake on a Rpi 3 is painfully slow. This is of course true also when building the pypi package from source.

@mayeut:
So the question really boils down to if the cmake community intends to add an armhf a. k. a. armv7l binary to the wheel.

If an armhf binary is added all is fine. But otherwise we probably need to explore other ways like distributing binary .deb packages or make a request to update cmake in Debian Buster backports.

[henryiii]

I believe PEP 599 added manylinux2014_armv7l (and manylinux2014_ppc64) tags, but manylinux does not build them (probably due to a lack of native support for any CI systems). https://github.com/pypa/manylinux

It's possible (though very slow) to emulate architectures on GitHub Actions, so I think it possibly could be done, but it would be slow. We'd need support from manylinux and cibuildwheel for armv7l to support it.

[ericLemanissier]

Same thing happening here on raspberry os, since cmake 3.22.1 is on pypi:
https://github.com/bincrafters/system-packages-checks/runs/4496926552?check_suite_focus=true#step:3:386

+ python3 -m pip install cmake
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting cmake
  Downloading cmake-3.22.1.tar.gz (30 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Building wheels for collected packages: cmake
  Building wheel for cmake (pyproject.toml): started
  Building wheel for cmake (pyproject.toml): finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 /usr/local/lib/python3.7/dist-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /tmp/tmphz12lnuh
       cwd: /tmp/pip-install-bmfk_sgw/cmake_655a3adb41a942308978185a07623cb8
  Complete output (9 lines):
    File "/tmp/pip-build-env-xxr67zw1/overlay/lib/python3.7/site-packages/skbuild/setuptools_wrap.py", line 564, in setup
      cmkr = cmaker.CMaker(cmake_executable)
    File "/tmp/pip-build-env-xxr67zw1/overlay/lib/python3.7/site-packages/skbuild/cmaker.py", line 95, in __init__
      self.cmake_version = get_cmake_version(self.cmake_executable)
    File "/tmp/pip-build-env-xxr67zw1/overlay/lib/python3.7/site-packages/skbuild/cmaker.py", line 82, in get_cmake_version
      "Problem with the CMake installation, aborting build. CMake executable is %s" % cmake_executable)
  Traceback (most recent call last):
  
  Problem with the CMake installation, aborting build. CMake executable is cmake
  ----------------------------------------
  ERROR: Failed building wheel for cmake

The build passed 1 hour earlier, when it got cmake 3.22.0

[henryiii]

Do you know why this might have passed with 3.22.0 but not with 3.22.1?

[henryiii]

I don't see anything here: 3.22.0...3.22.1 that seems likely to break?

[ericLemanissier]

+ python3 -m pip install cmake
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting cmake
  Downloading https://www.piwheels.org/simple/cmake/cmake-3.22.0-cp37-cp37m-linux_armv6l.whl (15.0 MB)
Installing collected packages: cmake
Successfully installed cmake-3.22.0

Because there was an existing piwheels for 3.22.0

[henryiii]

Ahh, piwheels. I think their CI just takes a little time after a release. Tell pip to require binaries and that will fix the small window after release issue. One of the three (https://www.piwheels.org/project/cmake/) is done, others are working.

[ericLemanissier]

indeed: https://www.piwheels.org/project/cmake/

[henryiii]

PIP_ONLY_BINARY=cmake should fix the problem.