-
Notifications
You must be signed in to change notification settings - Fork 229
feat: use cross-compilation to build images #1833
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mayeut
wants to merge
1
commit into
pypa:main
Choose a base branch
from
mayeut:cross-compilation
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -5,8 +5,12 @@ ARG PLATFORM=x86_64 | |
ARG DEVTOOLSET_ROOTPATH=/opt/rh/gcc-toolset-14/root | ||
ARG LD_LIBRARY_PATH_ARG=${DEVTOOLSET_ROOTPATH}/usr/lib64:${DEVTOOLSET_ROOTPATH}/usr/lib:${DEVTOOLSET_ROOTPATH}/usr/lib64/dyninst:${DEVTOOLSET_ROOTPATH}/usr/lib/dyninst | ||
ARG PREPEND_PATH=/usr/local/bin:${DEVTOOLSET_ROOTPATH}/usr/bin: | ||
ARG MANYLINUX_BUILDARCH=${BUILDARCH} | ||
ARG MANYLINUX_DISABLE_CLANG=0 | ||
ARG MANYLINUX_DISABLE_CLANG_FOR_CPYTHON=0 | ||
|
||
FROM $BASEIMAGE AS runtime_base | ||
|
||
FROM $BASEIMAGE AS runtime_base_packages | ||
ARG POLICY | ||
ARG PLATFORM | ||
ARG DEVTOOLSET_ROOTPATH | ||
|
@@ -42,29 +46,49 @@ RUN manylinux-entrypoint /build_scripts/install-runtime-packages.sh && rm -rf /b | |
|
||
COPY build_scripts/build_utils.sh /build_scripts/ | ||
|
||
|
||
# prepare cross-compilation support | ||
FROM --platform=linux/${MANYLINUX_BUILDARCH} ghcr.io/mayeut/static-clang:20.1.8.0 AS static_clang_bin | ||
FROM runtime_base_packages AS static_clang_prepare | ||
ARG MANYLINUX_DISABLE_CLANG | ||
COPY build_scripts/install-clang-static.sh /build_scripts/ | ||
RUN --mount=type=bind,target=/clang,from=static_clang_bin \ | ||
mkdir -p /tmp/cross-compiler && \ | ||
cp -rf /clang/opt/clang/* /tmp/cross-compiler/ && \ | ||
/build_scripts/install-clang-static.sh /tmp/cross-compiler | ||
FROM scratch AS static_clang | ||
COPY --from=static_clang_prepare /tmp/cross-compiler / | ||
|
||
|
||
FROM runtime_base_packages AS runtime_base | ||
COPY build_scripts/install-autoconf.sh /build_scripts/ | ||
RUN export AUTOCONF_ROOT=autoconf-2.72 && \ | ||
RUN --mount=type=bind,from=static_clang,target=/tmp/cross-compiler,ro \ | ||
export AUTOCONF_ROOT=autoconf-2.72 && \ | ||
export AUTOCONF_HASH=afb181a76e1ee72832f6581c0eddf8df032b83e2e0239ef79ebedc4467d92d6e && \ | ||
export AUTOCONF_DOWNLOAD_URL=http://ftp.gnu.org/gnu/autoconf && \ | ||
manylinux-entrypoint /build_scripts/install-autoconf.sh | ||
/tmp/cross-compiler/entrypoint /build_scripts/install-autoconf.sh | ||
|
||
COPY build_scripts/install-automake.sh /build_scripts/ | ||
RUN export AUTOMAKE_ROOT=automake-1.18.1 && \ | ||
RUN --mount=type=bind,from=static_clang,target=/tmp/cross-compiler,ro \ | ||
export AUTOMAKE_ROOT=automake-1.18.1 && \ | ||
export AUTOMAKE_HASH=63e585246d0fc8772dffdee0724f2f988146d1a3f1c756a3dc5cfbefa3c01915 && \ | ||
export AUTOMAKE_DOWNLOAD_URL=http://ftp.gnu.org/gnu/automake && \ | ||
manylinux-entrypoint /build_scripts/install-automake.sh | ||
/tmp/cross-compiler/entrypoint /build_scripts/install-automake.sh | ||
|
||
COPY build_scripts/install-libtool.sh /build_scripts/ | ||
RUN export LIBTOOL_ROOT=libtool-2.5.4 && \ | ||
RUN --mount=type=bind,from=static_clang,target=/tmp/cross-compiler,ro \ | ||
export LIBTOOL_ROOT=libtool-2.5.4 && \ | ||
export LIBTOOL_HASH=da8ebb2ce4dcf46b90098daf962cffa68f4b4f62ea60f798d0ef12929ede6adf && \ | ||
export LIBTOOL_DOWNLOAD_URL=http://ftp.gnu.org/gnu/libtool && \ | ||
manylinux-entrypoint /build_scripts/install-libtool.sh | ||
/tmp/cross-compiler/entrypoint /build_scripts/install-libtool.sh | ||
|
||
COPY build_scripts/install-libxcrypt.sh /build_scripts/ | ||
RUN export LIBXCRYPT_VERSION=4.4.38 && \ | ||
RUN --mount=type=bind,from=static_clang,target=/tmp/cross-compiler,ro \ | ||
export LIBXCRYPT_VERSION=4.4.38 && \ | ||
export LIBXCRYPT_HASH=80304b9c306ea799327f01d9a7549bdb28317789182631f1b54f4511b4206dd6 && \ | ||
export LIBXCRYPT_DOWNLOAD_URL=https://github.com/besser82/libxcrypt/releases/download && \ | ||
manylinux-entrypoint /build_scripts/install-libxcrypt.sh | ||
/tmp/cross-compiler/entrypoint /build_scripts/install-libxcrypt.sh | ||
|
||
|
||
FROM runtime_base AS build_base | ||
COPY build_scripts/install-build-packages.sh /build_scripts/ | ||
|
@@ -73,81 +97,111 @@ RUN manylinux-entrypoint /build_scripts/install-build-packages.sh | |
|
||
FROM build_base AS build_git | ||
COPY build_scripts/build-curl.sh /build_scripts/ | ||
RUN export CURL_ROOT=curl-8.15.0 && \ | ||
RUN --mount=type=bind,from=static_clang,target=/tmp/cross-compiler,ro \ | ||
export CURL_ROOT=curl-8.15.0 && \ | ||
export CURL_HASH=d85cfc79dc505ff800cb1d321a320183035011fa08cb301356425d86be8fc53c && \ | ||
export CURL_DOWNLOAD_URL=https://curl.haxx.se/download && \ | ||
manylinux-entrypoint /build_scripts/build-curl.sh | ||
/tmp/cross-compiler/entrypoint /build_scripts/build-curl.sh | ||
COPY build_scripts/build-git.sh /build_scripts/ | ||
RUN export GIT_ROOT=git-2.51.0 && \ | ||
RUN --mount=type=bind,from=static_clang,target=/tmp/cross-compiler,ro \ | ||
export GIT_ROOT=git-2.51.0 && \ | ||
export GIT_HASH=3d531799d2cf2cac8e294ec6e3229e07bfca60dc6c783fe69e7712738bef7283 && \ | ||
export GIT_DOWNLOAD_URL=https://www.kernel.org/pub/software/scm/git && \ | ||
manylinux-entrypoint /build_scripts/build-git.sh | ||
/tmp/cross-compiler/entrypoint /build_scripts/build-git.sh | ||
|
||
|
||
FROM build_base AS build_sqlite3 | ||
COPY build_scripts/build-sqlite3.sh /build_scripts/ | ||
RUN export SQLITE_AUTOCONF_ROOT=sqlite-autoconf-3500400 && \ | ||
RUN --mount=type=bind,from=static_clang,target=/tmp/cross-compiler,ro \ | ||
export SQLITE_AUTOCONF_ROOT=sqlite-autoconf-3500400 && \ | ||
export SQLITE_AUTOCONF_HASH=a3db587a1b92ee5ddac2f66b3edb41b26f9c867275782d46c3a088977d6a5b18 && \ | ||
export SQLITE_AUTOCONF_DOWNLOAD_URL=https://www.sqlite.org/2025 && \ | ||
manylinux-entrypoint /build_scripts/build-sqlite3.sh | ||
/tmp/cross-compiler/entrypoint /build_scripts/build-sqlite3.sh | ||
|
||
|
||
FROM build_base AS build_tcl_tk | ||
COPY build_scripts/build-tcltk.sh /build_scripts/ | ||
RUN export TCL_ROOT=tcl8.6.17 && \ | ||
RUN --mount=type=bind,from=static_clang,target=/tmp/cross-compiler,ro \ | ||
export TCL_ROOT=tcl8.6.17 && \ | ||
export TCL_HASH=a3903371efcce8a405c5c245d029e9f6850258a60fa3761c4d58995610949b31 && \ | ||
export TCL_DOWNLOAD_URL=https://prdownloads.sourceforge.net/tcl && \ | ||
export TK_ROOT=tk8.6.17 && \ | ||
export TK_HASH=e4982df6f969c08bf9dd858a6891059b4a3f50dc6c87c10abadbbe2fc4838946 && \ | ||
manylinux-entrypoint /build_scripts/build-tcltk.sh | ||
/tmp/cross-compiler/entrypoint /build_scripts/build-tcltk.sh | ||
|
||
|
||
FROM build_base AS build_mpdecimal | ||
COPY build_scripts/build-mpdecimal.sh /build_scripts/ | ||
RUN export MPDECIMAL_ROOT=mpdecimal-4.0.0 && \ | ||
RUN --mount=type=bind,from=static_clang,target=/tmp/cross-compiler,ro \ | ||
export MPDECIMAL_ROOT=mpdecimal-4.0.0 && \ | ||
export MPDECIMAL_HASH=942445c3245b22730fd41a67a7c5c231d11cb1b9936b9c0f76334fb7d0b4468c && \ | ||
export MPDECIMAL_DOWNLOAD_URL=https://www.bytereef.org/software/mpdecimal/releases && \ | ||
manylinux-entrypoint /build_scripts/build-mpdecimal.sh | ||
/tmp/cross-compiler/entrypoint /build_scripts/build-mpdecimal.sh | ||
|
||
|
||
FROM --platform=${BUILDPLATFORM} ghcr.io/sigstore/cosign/cosign:v2.5.0 AS cosign-bin | ||
|
||
|
||
FROM build_base AS build_cpython | ||
COPY --from=build_tcl_tk /manylinux-buildfs / | ||
COPY --from=build_mpdecimal /manylinux-buildfs / | ||
COPY --from=build_sqlite3 /manylinux-buildfs / | ||
RUN if command -v apk >/dev/null 2>&1; then ldconfig /; else ldconfig; fi | ||
COPY build_scripts/build-openssl.sh /build_scripts/ | ||
RUN export OPENSSL_ROOT=openssl-3.5.2 && \ | ||
RUN --mount=type=bind,from=static_clang,target=/tmp/cross-compiler,ro \ | ||
export OPENSSL_ROOT=openssl-3.5.2 && \ | ||
export OPENSSL_HASH=c53a47e5e441c930c3928cf7bf6fb00e5d129b630e0aa873b08258656e7345ec && \ | ||
export OPENSSL_DOWNLOAD_URL=https://github.com/openssl/openssl/releases/download/${OPENSSL_ROOT} && \ | ||
manylinux-entrypoint /build_scripts/build-openssl.sh | ||
/tmp/cross-compiler/entrypoint /build_scripts/build-openssl.sh | ||
COPY --from=cosign-bin /ko-app/cosign /usr/local/bin/cosign | ||
COPY build_scripts/build-cpython.sh /build_scripts/ | ||
|
||
|
||
FROM build_cpython AS build_cpython38 | ||
RUN manylinux-entrypoint /build_scripts/build-cpython.sh [email protected] https://github.com/login/oauth 3.8.20 | ||
ARG MANYLINUX_DISABLE_CLANG_FOR_CPYTHON | ||
RUN --mount=type=bind,from=static_clang,target=/tmp/cross-compiler,ro \ | ||
/tmp/cross-compiler/entrypoint /build_scripts/build-cpython.sh [email protected] https://github.com/login/oauth 3.8.20 | ||
|
||
FROM build_cpython AS build_cpython39 | ||
RUN manylinux-entrypoint /build_scripts/build-cpython.sh [email protected] https://github.com/login/oauth 3.9.23 | ||
ARG MANYLINUX_DISABLE_CLANG_FOR_CPYTHON | ||
RUN --mount=type=bind,from=static_clang,target=/tmp/cross-compiler,ro \ | ||
/tmp/cross-compiler/entrypoint /build_scripts/build-cpython.sh [email protected] https://github.com/login/oauth 3.9.23 | ||
|
||
FROM build_cpython AS build_cpython310 | ||
RUN manylinux-entrypoint /build_scripts/build-cpython.sh [email protected] https://accounts.google.com 3.10.18 | ||
ARG MANYLINUX_DISABLE_CLANG_FOR_CPYTHON | ||
RUN --mount=type=bind,from=static_clang,target=/tmp/cross-compiler,ro \ | ||
/tmp/cross-compiler/entrypoint /build_scripts/build-cpython.sh [email protected] https://accounts.google.com 3.10.18 | ||
|
||
FROM build_cpython AS build_cpython311 | ||
RUN manylinux-entrypoint /build_scripts/build-cpython.sh [email protected] https://accounts.google.com 3.11.13 | ||
ARG MANYLINUX_DISABLE_CLANG_FOR_CPYTHON | ||
RUN --mount=type=bind,from=static_clang,target=/tmp/cross-compiler,ro \ | ||
/tmp/cross-compiler/entrypoint /build_scripts/build-cpython.sh [email protected] https://accounts.google.com 3.11.13 | ||
|
||
FROM build_cpython AS build_cpython312 | ||
RUN manylinux-entrypoint /build_scripts/build-cpython.sh [email protected] https://accounts.google.com 3.12.11 | ||
ARG MANYLINUX_DISABLE_CLANG_FOR_CPYTHON | ||
RUN --mount=type=bind,from=static_clang,target=/tmp/cross-compiler,ro \ | ||
/tmp/cross-compiler/entrypoint /build_scripts/build-cpython.sh [email protected] https://accounts.google.com 3.12.11 | ||
|
||
FROM build_cpython AS build_cpython313 | ||
RUN manylinux-entrypoint /build_scripts/build-cpython.sh [email protected] https://accounts.google.com 3.13.7 | ||
ARG MANYLINUX_DISABLE_CLANG_FOR_CPYTHON | ||
RUN --mount=type=bind,from=static_clang,target=/tmp/cross-compiler,ro \ | ||
/tmp/cross-compiler/entrypoint /build_scripts/build-cpython.sh [email protected] https://accounts.google.com 3.13.7 | ||
|
||
FROM build_cpython AS build_cpython313_nogil | ||
RUN manylinux-entrypoint /build_scripts/build-cpython.sh [email protected] https://accounts.google.com 3.13.7 nogil | ||
ARG MANYLINUX_DISABLE_CLANG_FOR_CPYTHON | ||
RUN --mount=type=bind,from=static_clang,target=/tmp/cross-compiler,ro \ | ||
/tmp/cross-compiler/entrypoint /build_scripts/build-cpython.sh [email protected] https://accounts.google.com 3.13.7 nogil | ||
|
||
FROM build_cpython AS build_cpython314 | ||
RUN manylinux-entrypoint /build_scripts/build-cpython.sh [email protected] https://github.com/login/oauth 3.14.0rc2 | ||
ARG MANYLINUX_DISABLE_CLANG_FOR_CPYTHON | ||
RUN --mount=type=bind,from=static_clang,target=/tmp/cross-compiler,ro \ | ||
/tmp/cross-compiler/entrypoint /build_scripts/build-cpython.sh [email protected] https://github.com/login/oauth 3.14.0rc2 | ||
|
||
FROM build_cpython AS build_cpython314_nogil | ||
RUN manylinux-entrypoint /build_scripts/build-cpython.sh [email protected] https://github.com/login/oauth 3.14.0rc2 nogil | ||
ARG MANYLINUX_DISABLE_CLANG_FOR_CPYTHON | ||
RUN --mount=type=bind,from=static_clang,target=/tmp/cross-compiler,ro \ | ||
/tmp/cross-compiler/entrypoint /build_scripts/build-cpython.sh [email protected] https://github.com/login/oauth 3.14.0rc2 nogil | ||
|
||
|
||
FROM runtime_base | ||
COPY --from=build_tcl_tk /manylinux-rootfs / | ||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.