Latest Python 3.11 Alpine seems to have symlinks broken

[akmalharith]

In some packages that depend on python3.11-alpine, build totally breaks, ie: aws/aws-cli#8698

To reproduce this issue, build something on top of 3.11 Alpine:

FROM python:3.11-alpine AS python

# Install gcloud etc

RUN gcloud components install -q \
  kubectl \
  alpha \
  beta \
  gke-gcloud-auth-plugin

Error message:

 > [stage-2  5/17] RUN gcloud components install -q   kubectl   alpha   beta   gke-gcloud-auth-plugin:
0.265 Error relocating /usr/local/bin/../lib/libpython3.11.so.1.0: pwritev2: symbol not found
0.265 Error relocating /usr/local/bin/../lib/libpython3.11.so.1.0: preadv2: symbol not found
0.270 Error relocating /usr/local/bin/../lib/libpython3.11.so.1.0: pwritev2: symbol not found
0.271 Error relocating /usr/local/bin/../lib/libpython3.11.so.1.0: preadv2: symbol not found
0.273 Error relocating /usr/local/bin/../lib/libpython3.11.so.1.0: pwritev2: symbol not found
0.273 Error relocating /usr/local/bin/../lib/libpython3.11.so.1.0: preadv2: symbol not found

At the moment I have a workaround since our build pipelines are failing, to pin to Alpine 3.19

FROM python:3.11.9-alpine3.19

[LaurentGoderre]

Very similar to docker/compose#6292 (comment)

[tb102122]

@LaurentGoderre so what are you suggesting as solution?

[tianon]

That kind of sounds like there's two versions of Python in the image (perhaps distro Python via apk and the Python provided by the python image?)

Can we get a simplified/minimal reproducer?

[tb102122]

@tianon I think you have it already #927 (comment) or do you need something else?

[yosifkit]

# Install gcloud etc

That's where the problem likely lies. If python3 is installed in those layers, then it will very likely conflict in weird ways like the error given. Since you will have python3 (3.12, /usr/bin/python3) from apk packages and the source installed python3 (3.11, /usr/local/bin/python3) from the image.

(Pinning to Alpine 3.19 would mean the apk installed python is the same major Python version, 3.11, so might be less likely to conflict, but still not optimal)

[LaurentGoderre]

I tried to get a reproducer by installing gcloud proved very difficult for this version of Python

[tropicoo]

FROM python:3.12-alpine as builder

RUN python -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"

RUN apk update && apk upgrade && apk add --no-cache --virtual \
    build-base \
    libstdc++ \
    musl-dev \
    gcc \
    python3-dev \
    git \
    && pip install --upgrade pip wheel setuptools

Error:

20.14 (23/23) Installing build-base (20240727.102005)
20.14 Executing busybox-1.36.1-r29.trigger
20.15 OK: 263 MiB in 58 packages
20.24 Error relocating /usr/local/bin/../lib/libpython3.12.so.1.0: pwritev2: symbol not found
20.24 Error relocating /usr/local/bin/../lib/libpython3.12.so.1.0: preadv2: symbol not found

[tianon]

If you are FROM python and using the package manager to install Python-related packages, you will end up with two versions of Python in your image (one from our builds, one from the distribution). That's almost certainly never what you're actually looking for and you'll want to choose one or the other (either by going FROM alpine:xxx and installing their Python or by dropping the Python-installing deps from your package installs and using pip and friends to install dependencies manually instead).

[nrundle]

This just got me but it turned out it was a misconfiguration on my part so hopefully this info can help someone else. The python docker image tags that end in -alpine can change the alpine version. This is not an issue unless you are configuring and installing other alpine packages from an incompatible version.

In my case, I configured the alpine repositories to use our company Artifactory proxy for alpine 3.19 even though I was using the 3.12-alpine docker tag. This was working fine up until that tag changed to use alpine 3.20 instead. I wasn't installing python or python packages from the package manager but I did install build-base which exhibited this same behavior when trying to compile python packages installed from pip.

Usually I only use the non-pinned tags while initially developing things and then pin them down once I have a pipeline up and running.