Skip to content

Commit 59359a0

Browse files
authored
ci-conda: stop shipping compilers, update codecov-cli to 11.2.6 (#353)
`ci-conda` images currently ship a system-installed `gcc` and `g++`. This proposes removing them. It also proposes some other minor re-organization of `ci-conda`, combining very-combinable things (like the creation of directories and files whose content rarely changes) into fewer layers. ## Why were they originally added? Those were added only because we were building the `codecov` CLI from source in these images (#147). That's no longer necessary here in 2026, because newer releases of `codecov-cli` provide `py3-none-any` wheels (e.g. https://pypi.org/project/codecov-cli/11.2.5/#files). ## Benefits of removing them? Slightly smaller images. But more importantly...stronger correctness guarantees. conda-based CI jobs in RAPIDS should never be reaching for system-installed compilers... we'd always prefer to use conda-forge's compilers for building conda packages. Removing these system-installed `gcc` and `g++` reduces the risk of accidentally picking up things from those packages that should have come from conda-provided compilers. Authors: - James Lamb (https://github.com/jameslamb) Approvers: - Gil Forsyth (https://github.com/gforsyth) URL: #353
1 parent e375390 commit 59359a0

2 files changed

Lines changed: 65 additions & 25 deletions

File tree

ci-conda.Dockerfile

Lines changed: 63 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -100,10 +100,10 @@ EOF
100100
# Ownership & permissions based on https://docs.anaconda.com/anaconda/install/multi-user/#multi-user-anaconda-installation-on-linux
101101
COPY --from=miniforge-upstream --chown=root:conda --chmod=770 /opt/conda /opt/conda
102102

103+
RUN <<EOF
103104
# Ensure new files are created with group write access & setgid. See https://unix.stackexchange.com/a/12845
104-
RUN chmod g+ws /opt/conda
105+
chmod g+ws /opt/conda
105106

106-
RUN <<EOF
107107
# Ensure new files/dirs have group write permissions
108108
umask 002
109109

@@ -130,10 +130,10 @@ find /opt/conda -follow -type f -name '*.pyc' -delete
130130
conda clean -aiptfy
131131
EOF
132132

133+
RUN <<EOF
133134
# Reassign root's primary group to root
134-
RUN usermod -g root root
135+
usermod -g root root
135136

136-
RUN <<EOF
137137
# ensure conda environment is always activated
138138
ln -s /opt/conda/etc/profile.d/conda.sh /etc/profile.d/conda.sh
139139
echo ". /opt/conda/etc/profile.d/conda.sh; conda activate base" >> /etc/skel/.bashrc
@@ -189,8 +189,6 @@ case "${LINUX_VER}" in
189189
file
190190
unzip
191191
wget
192-
gcc
193-
g++
194192
)
195193
apt-get install -y --no-install-recommends \
196194
"${PACKAGES_TO_INSTALL[@]}"
@@ -206,8 +204,6 @@ case "${LINUX_VER}" in
206204
wget
207205
which
208206
yum-utils
209-
gcc
210-
gcc-c++
211207
)
212208
dnf -y install --setopt=install_weak_deps=False \
213209
"${PACKAGES_TO_INSTALL[@]}"
@@ -221,22 +217,21 @@ case "${LINUX_VER}" in
221217
esac
222218
EOF
223219

224-
# Install prereq for envsubst
225-
RUN <<EOF
226-
rapids-mamba-retry install -y \
227-
gettext
228-
conda clean -aiptfy
229-
EOF
230-
231220
# Create condarc file from env vars
232221
ENV RAPIDS_CONDA_BLD_ROOT_DIR=/tmp/conda-bld-workspace
233222
ENV RAPIDS_CONDA_BLD_OUTPUT_DIR=/tmp/conda-bld-output
234223
COPY condarc.tmpl /tmp/condarc.tmpl
235-
RUN cat /tmp/condarc.tmpl | envsubst | tee /opt/conda/.condarc; \
236-
rm -f /tmp/condarc.tmpl
237224

238225
# Install CI tools using mamba
239226
RUN <<EOF
227+
# Install prereq for envsubst
228+
rapids-mamba-retry install -y \
229+
gettext
230+
231+
# create condarc file from env vars
232+
cat /tmp/condarc.tmpl | envsubst | tee /opt/conda/.condarc; \
233+
rm -f /tmp/condarc.tmpl
234+
240235
PYTHON_MAJOR_VERSION=${PYTHON_VERSION%%.*}
241236
PYTHON_MINOR_VERSION=${PYTHON_VERSION#*.}
242237
PYTHON_UPPER_BOUND="${PYTHON_MAJOR_VERSION}.$((PYTHON_MINOR_VERSION+1)).0a0"
@@ -269,27 +264,72 @@ rapids-mamba-retry install -y \
269264
conda clean -aiptfy
270265
EOF
271266

272-
# Install yq and awscli
267+
# Install tools
273268
ARG AWS_CLI_VER=notset
269+
ARG CODECOV_VER=notset
274270
ARG REAL_ARCH=notset
275271
ARG YQ_VER=notset
276272
RUN <<EOF
273+
# yq
277274
rapids-retry wget -q https://github.com/mikefarah/yq/releases/download/v${YQ_VER}/yq_linux_${CPU_ARCH} -O /tmp/yq
278275
mv /tmp/yq /usr/bin/yq
279276
chmod +x /usr/bin/yq
280277

278+
# AWS CLI
281279
# ref: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html#getting-started-install-instructions
282280
rapids-retry curl -o /tmp/awscliv2.zip \
283281
-L "https://awscli.amazonaws.com/awscli-exe-linux-${REAL_ARCH}-${AWS_CLI_VER}.zip"
284282
unzip -q /tmp/awscliv2.zip -d /tmp
285283
/tmp/aws/install
286284
rm -rf /tmp/aws /tmp/awscliv2.zip
287-
EOF
288285

289-
# Install codecov from source distribution
290-
ARG CODECOV_VER=notset
291-
RUN <<EOF
292-
rapids-pip-retry install codecov-cli==${CODECOV_VER}
286+
# codecov-cli
287+
#
288+
# codecov-cli is a noarch Python package, but some of its dependencies require compilation.
289+
# compilers are installed defensively here to prevent issues like "a dependency of codecov-cli
290+
# doesn't support CPU_ARCH / LINUX_VER / PYTHON_VER" from slowing down updates to RAPIDS CI.
291+
#
292+
293+
case "${LINUX_VER}" in
294+
"ubuntu"*)
295+
COMPILER_PACKAGES=(
296+
gcc
297+
g++
298+
)
299+
rapids-retry apt-get update -y
300+
apt-get install -y --no-install-recommends \
301+
"${COMPILER_PACKAGES[@]}"
302+
;;
303+
"rockylinux"*)
304+
COMPILER_PACKAGES=(
305+
gcc
306+
gcc-c++
307+
)
308+
dnf install -y \
309+
"${COMPILER_PACKAGES[@]}"
310+
;;
311+
esac
312+
313+
rapids-pip-retry install --prefer-binary \
314+
"codecov-cli==${CODECOV_VER}"
315+
316+
# remove compiler packages... conda-based CI should use conda-forge's compilers
317+
case "${LINUX_VER}" in
318+
"ubuntu"*)
319+
apt-get purge -y \
320+
"${COMPILER_PACKAGES[@]}"
321+
apt-get autoremove -y
322+
rm -rf /var/cache/apt/archives /var/lib/apt/lists/*
323+
;;
324+
"rockylinux"*)
325+
dnf remove -y \
326+
"${COMPILER_PACKAGES[@]}"
327+
dnf clean all
328+
;;
329+
esac
330+
331+
# clear the pip cache, to shrink image size and prevent unintentionally
332+
# pinning CI to older versions of things
293333
pip cache purge
294334
EOF
295335

versions.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
1-
# SPDX-FileCopyrightText: Copyright (c) 2024-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
1+
# SPDX-FileCopyrightText: Copyright (c) 2024-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
22
# SPDX-License-Identifier: Apache-2.0
33
# renovate: datasource=github-releases depName=cli/cli
44
GH_CLI_VER: 2.83.1
55
# renovate: datasource=pypi depName=codecov-cli
6-
CODECOV_VER: 11.2.5
6+
CODECOV_VER: 11.2.6
77
# renovate: datasource=github-releases depName=mikefarah/yq
88
YQ_VER: 4.49.2
99
# renovate: datasource=github-releases depName=aws/aws-cli

0 commit comments

Comments
 (0)