Reduce size of Docker image

[gemenerik]

The built Docker image is quite large, leading to slower CI processes due to frequent pulls. In fact, it is so big that GitHub's caching features are largely unsupported. Some features of the image are rarely used (esp-idf). To improve efficiency and sustainability, consider implementing multi-stage builds, separate builder images for different projects, using a smaller base image, or cleaning up unnecessary dependencies. This would enhance build times and reduce resource usage.

[gemenerik]

An alternative option is to run more of the CI locally

[gemenerik]

One approach is to stop using the Docker builder images for our GitHub Actions.

One example of what this could look like is bitcraze/crazyflie-lib-python#494.

One significant advantage of this transition is the elimination of the need to pull the large (currently 5.41 GB for bitcraze/builder, 977.51 MB for bitcraze/web-builder) Docker image for every CI step. In cflib, we download 6.39 GB of Docker image per CI run. For crazyflie-firmware ( with 24 pulls of the builder) this amounts to 129.84 GB per CI run. Let's hope GitHub does some automatic caching for us. This change should lead to faster and more sustainable CI runs. By leveraging GitHub's default environments, we gain flexibility in dependency management, which may help us adapt more easily to changing requirements.

However, this shift raises important considerations. If we choose to continue using monolithic Docker images for local development, we will need to ensure that these images are tested (e.g. in our CI pipeline) as well. While these Docker images provide a consistent environment for users to build all projects, maintaining them has become challenging. Compatibility issues can arise, especially when certain components or compilers require different dependencies, complicating the build process.