charmcraft pack fail while packing cos-proxy-operator

[Abuelodelanada]

Bug Description

charmcraft version 3.2.2 fail packing cos-proxy-operator

charmcraft version 2.7.3 pack cos-proxy-operator charm successfully.

Execution log: charmcraft-20250116-193512.859515.log

To Reproduce

1. git clone https://github.com/canonical/cos-proxy-operator
2. charmcraft clean
3. charmcraft pack --verbose --debug

After that you'll see:

::    ::             building '_cffi_backend' extension                                                                                      
::    ::             creating build/temp.linux-x86_64-cpython-38/src/c                                                                       
::    ::             x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DFFI_BUILDING=1 -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/root/parts/charm/build/staging-venv/include -I/usr/include/python3.8 -c src/c/_cffi_backend.c -o build/temp.linux-x86_64-cpython-38/src/c/_cffi_backend.o                                                                                                                      
::    ::             src/c/_cffi_backend.c:15:10: fatal error: ffi.h: No such file or directory                                              
::    ::                15 | #include <ffi.h>                                                                                                
::    ::                   |          ^~~~~~~                                                                                                
::    ::             compilation terminated.                                                                                                 
::    ::             error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1                                                  
::    ::             [end of output]                                                                                                         
::    ::                                                                                                                                     
::    ::         note: This error originates from a subprocess, and is likely not a problem with pip.                                        
::    ::         ERROR: Failed building wheel for cffi                                                                                       
::    ::       Failed to build cffi                                                                                                          
::    ::       ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (cffi)                                
::    ::                                                                                                                                     
::    ::       [notice] A new release of pip is available: 24.1.1 -> 24.3.1                                                                  
::    ::       [notice] To update, run: python3 -m pip install --upgrade pip                                                                 
::    ::       [end of output]                                                                                                               
::    ::                                                                                                                                     
::    ::   note: This error originates from a subprocess, and is likely not a problem with pip.                                              
::    :: error: subprocess-exited-with-error                                                                                                 
::    ::                                                                                                                                     
::    :: × pip subprocess to install build dependencies did not run successfully.                                                            
::    :: │ exit code: 1                                                                                                                      
::    :: ╰─> See above for output.                                                                                                           
::    ::                                                                                                                                     
::    :: note: This error originates from a subprocess, and is likely not a problem with pip.                                                
::    ::                                                                                                                                     
::    :: [notice] A new release of pip is available: 24.1.1 -> 24.3.1                                                                        
::    :: [notice] To update, run: python3 -m pip install --upgrade pip                                                                       
:: Traceback (most recent call last):                                                                                                        
::   File "/snap/charmcraft/5303/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 483, in <module>                            
::     main()                                                                                                                                
::   File "/snap/charmcraft/5303/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 478, in main                                
::     builder.build_charm()                                                                                                                 
::   File "/snap/charmcraft/5303/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 93, in build_charm                          
::     self.handle_dependencies()                                                                                                            
::   File "/snap/charmcraft/5303/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 354, in handle_dependencies                 
::     self._install_dependencies(staging_venv_dir)                                                                                          
::   File "/snap/charmcraft/5303/lib/python3.10/site-packages/charmcraft/instrum.py", line 152, in _f                                        
::     return func(*args, **kwargs)                                                                                                          
::   File "/snap/charmcraft/5303/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 284, in _install_dependencies               
::     _process_run(                                                                                                                         
::   File "/snap/charmcraft/5303/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 414, in _process_run                        
::     raise RuntimeError(f"Subprocess command {cmd} execution failed with retcode {retcode}")                                               
:: RuntimeError: Subprocess command ['/root/parts/charm/build/staging-venv/bin/pip', 'install', '--no-binary=:all:', '--requirement=requirements.txt'] execution failed with retcode 1                                                                                                     
Failed to run the build script for part 'charm'.                                                                                             
Launching shell on build environment...                                                                                                      
charmcraft-cos-proxy-on-amd64-for-amd64-62 ../project# 

In that shell you can verify that libffi-dev is not installed and the file /usr/include/x86_64-linux-gnu/ffi.h does not exist:

charmcraft-cos-proxy-on-amd64-for-amd64-62 ../project# ls -l /usr/include/x86_64-linux-gnu/
total 52
-rw-r--r-- 1 root root 4351 Apr 14  2020 a.out.h
drwxr-xr-x 2 root root 4096 Nov 25 14:16 asm
drwxr-xr-x 3 root root 4096 Nov 25 14:16 bits
drwxr-xr-x 3 root root 4096 Nov 25 14:16 c++
-rw-r--r-- 1 root root 3830 Dec  2 00:26 expat_config.h
drwxr-xr-x 2 root root 4096 Nov 25 14:16 finclude
-rw-r--r-- 1 root root 3584 Apr 14  2020 fpu_control.h
drwxr-xr-x 2 root root 4096 Nov 25 14:16 gnu
-rw-r--r-- 1 root root 4916 Apr 14  2020 ieee754.h
drwxr-xr-x 2 root root 4096 Jan 15 10:22 python3.8
drwxr-xr-x 2 root root 4096 Nov 25 14:16 sys


charmcraft-cos-proxy-on-amd64-for-amd64-62 ../project# dpkg -l | grep libffi-dev
charmcraft-cos-proxy-on-amd64-for-amd64-62 ../project# 

But libffi-dev it is installable:

charmcraft-cos-proxy-on-amd64-for-amd64-62 ../project# apt install libffi-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  libffi-dev
0 upgraded, 1 newly installed, 0 to remove and 96 not upgraded.
Need to get 57.0 kB of archives.
After this operation, 312 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 libffi-dev amd64 3.3-4 [57.0 kB]
Fetched 57.0 kB in 1s (60.0 kB/s)     
Selecting previously unselected package libffi-dev:amd64.
(Reading database ... 16342 files and directories currently installed.)
Preparing to unpack .../libffi-dev_3.3-4_amd64.deb ...
Unpacking libffi-dev:amd64 (3.3-4) ...
Setting up libffi-dev:amd64 (3.3-4) ...

After that installation the file /usr/include/x86_64-linux-gnu/ffi.h is there:

charmcraft-cos-proxy-on-amd64-for-amd64-62 ../project# ls -l /usr/include/x86_64-linux-gnu/ffi.h
-rw-r--r-- 1 root root 13934 Mar 23  2020 /usr/include/x86_64-linux-gnu/ffi.h

Environment

• charmcraft: 3.2.2
• ubuntu: 24.04.1 LTS

charmcraft.yaml

type: charm
bases:
  - build-on:
    - name: "ubuntu"
      channel: "20.04"
    run-on:
    - name: "ubuntu"
      channel: "20.04"
  - build-on:
    - name: "ubuntu"
      channel: "22.04"
    run-on:
    - name: "ubuntu"
      channel: "22.04"
parts:
  charm:
    build-packages:
    - "git"
    # rustc and cargo required to build pydantic
    - rustc
    - cargo
    charm-binary-python-packages:
      # Install PyYAML from binary and avoid building it from sources. This way, we can use PyYAML with C-optimized lib.
      # With the C-optimized lib, serialization in ops is 20x faster.
      - PyYAML
      - cryptography
  nrpe-exporter:
    plugin: dump
    source: .
    build-packages:
      - curl
    override-pull: |
      if [ $CRAFT_ARCH_BUILD_FOR == "amd64" ]; then
        URI=https://github.com/canonical/nrpe_exporter/releases/latest/download/nrpe_exporter-amd64
        curl -L $URI -o nrpe_exporter-amd64
      elif [ $CRAFT_ARCH_BUILD_FOR == "arm64" ] || [ $CRAFT_ARCH_BUILD_FOR == "aarch64" ]; then
        URI=https://github.com/canonical/nrpe_exporter/releases/latest/download/nrpe_exporter-arm64
        curl -L $URI -o nrpe_exporter-aarch64
      fi
  vector:
    plugin: dump
    source: .
    build-packages:
      - curl
      - tar
    override-pull: |
      VERSION=0.44.0
      if [ $CRAFT_ARCH_BUILD_FOR == "amd64" ]; then
        ARCH=x86_64-unknown-linux-musl
      elif [ $CRAFT_ARCH_BUILD_FOR == "arm64" ]; then
        ARCH=aarch64-unknown-linux-musl
      fi

      VECTOR_URI="https://packages.timber.io/vector/${VERSION}/vector-${VERSION}-${ARCH}.tar.gz"
      curl -sSfL ${VECTOR_URI} | tar xzf - -C . --strip-components=3 ./vector-${ARCH}/bin/vector

Relevant log output

Execution log attached.

[syncronize-issues-to-jira]

Thank you for reporting your feedback to us!

The internal ticket has been created: https://warthogs.atlassian.net/browse/CRAFT-3918.

This message was autogenerated

[lengau]

Hi, thanks for the report! This is solved in Charmcraft 3.3, which is currently in the candidate channel and will be promoted to stable on Monday.