Skip to content

SRE-3205 ci: update for Fedora 42 #87

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 24 commits into
base: master
Choose a base branch
from
+228 −10
Open

SRE-3205 ci: update for Fedora 42 #87

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
ed02ef8
SRE-3205 ci: update for Fedora 42
grom72 Jul 17, 2025
e0e779f
Add Leap 15 build on Fedora latest
grom72 Jul 18, 2025
a14108d
Add EL8 build on Fedora 41
grom72 Jul 21, 2025
e74d276
Update to leap 15.6
grom72 Jul 21, 2025
5540c3a
Add leap 15.5 build explicitly
grom72 Jul 21, 2025
9f5966a
Use Fedora "latest" explicitly
grom72 Jul 22, 2025
cc7570b
Remove support for all EOL Fedoora versions
grom72 Jul 22, 2025
362ae05
Disable Leap 15.5 until SRE-3219 is fixed
grom72 Jul 22, 2025
a63357b
Fix archive removal
grom72 Jul 22, 2025
61a087c
Revert "Remove support for all EOL Fedoora versions"
grom72 Jul 24, 2025
6296cf4
Use Fedora 38 for Leap 15.5 build
grom72 Jul 24, 2025
cd93a33
Detect if archive repo is needed for older Fedora
grom72 Jul 24, 2025
050e014
Revert "Detect if archive repo is needed for older Fedora"
grom72 Jul 24, 2025
56e8740
Documentation update
grom72 Jul 24, 2025
07d08fe
Automate Fedora EOL detection
grom72 Jul 24, 2025
bf92be5
Missing jq in the base Fedora image
grom72 Jul 24, 2025
abf6b88
Missing jq in the base Fedora image - fix
grom72 Jul 24, 2025
39ba6a6
Use local artifactory as eol proxy
grom72 Jul 24, 2025
292f1e7
Use local artifactory as eol proxy - fix 1
grom72 Jul 24, 2025
12b11f4
Use local artifactory as eol proxy - fix 2
grom72 Jul 24, 2025
6ba2414
Use local artifactory as eol proxy - fix 3
grom72 Jul 24, 2025
bc3c2d9
Use local artifactory as eol proxy - fix 4
grom72 Jul 24, 2025
92b5666
jq is no longer required
grom72 Jul 24, 2025
6499193
jq is no longer required - fix 1
grom72 Jul 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions Dockerfile.mockbuild
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ LABEL maintainer="[email protected]"
ARG REPO_FILE_URL
ARG DAOS_LAB_CA_FILE_URL
ARG REPOSITORY_NAME

# script to install OS updates basic tools and daos dependencies
# COPY ./utils/scripts/install-fedora.sh /tmp/install.sh
# script to setup local repo if available
Expand All @@ -26,9 +27,9 @@ RUN chmod +x /tmp/repo-helper.sh && \
rm -f /tmp/repo-helper.sh

# Install basic tools
RUN dnf -y install mock make \
rpm-build createrepo rpmlint redhat-lsb-core git \
python-srpm-macros rpmdevtools && \
RUN dnf -y install mock make \
rpm-build createrepo rpmlint git \
python-srpm-macros rpmdevtools && \
dnf -y clean all

# use same UID as host and default value of 1000 if not specified
Expand Down
203 changes: 200 additions & 3 deletions Jenkinsfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ pipeline {
' --cap-add=SYS_ADMIN' +
' --privileged=true' +
' -v /scratch:/scratch'
additionalBuildArgs dockerBuildArgs() + '--build-arg PACKAGINGDIR=. '
additionalBuildArgs dockerBuildArgs() + '--build-arg FVERSION=latest --build-arg PACKAGINGDIR=. '
}
}
steps {
Expand Down Expand Up @@ -110,6 +110,55 @@ pipeline {
}
}
} //stage('Build libfabric on EL 8')
stage('Build libfabric on EL 8 (Fedora 41)') {
agent {
dockerfile {
filename 'Dockerfile.mockbuild'
label 'docker_runner'
args '--group-add mock' +
' --cap-add=SYS_ADMIN' +
' --privileged=true' +
' -v /scratch:/scratch'
additionalBuildArgs dockerBuildArgs() +
'--build-arg FVERSION=41 --build-arg PACKAGINGDIR=. '
}
}
steps {
checkoutScm url: 'https://github.com/daos-stack/libfabric.git',
checkoutDir: "libfabric",
branch: commitPragma(pragma: 'libfabric-branch', def_val: 'master')
sh label: env.STAGE_NAME,
script: updatePackaging('libfabric') + '''
rm -rf artifacts/el8/
mkdir -p artifacts/el8/
make CHROOT_NAME="rocky+epel-8-x86_64" DISTRO_VERSION=8 chrootbuild'''
}
post {
success {
sh 'ls -l /var/lib/mock/rocky+epel-8-x86_64/result/'
}
unsuccessful {
sh label: "Collect artifacts",
script: '''mockroot=/var/lib/mock/rocky+epel-8-x86_64
artdir=$PWD/libfabric/artifacts/el8
cp -af _topdir/SRPMS $artdir
(cd $mockroot/result/ &&
cp -r . $artdir)
(if cd $mockroot/root/builddir/build/BUILD/*/; then
find . -name configure -printf %h\\\\n | \
while read dir; do
if [ ! -f $dir/config.log ]; then
continue
fi
tdir="$artdir/autoconf-logs/$dir"
mkdir -p $tdir
cp -a $dir/config.log $tdir/
done
fi)'''
archiveArtifacts artifacts: 'libfabric/artifacts/el8/**'
}
}
} //stage('Build libfabric on EL 8 (Fedora 41)')
stage('Build libfabric on EL 9') {
agent {
dockerfile {
Expand Down Expand Up @@ -158,6 +207,55 @@ pipeline {
}
}
} //stage('Build libfabric on EL 9')
stage('Build libfabric on EL 9 (Fedora 42)') {
agent {
dockerfile {
filename 'Dockerfile.mockbuild'
label 'docker_runner'
args '--group-add mock' +
' --cap-add=SYS_ADMIN' +
' --privileged=true' +
' -v /scratch:/scratch'
additionalBuildArgs dockerBuildArgs() +
'--build-arg FVERSION=42 --build-arg PACKAGINGDIR=. '
}
}
steps {
checkoutScm url: 'https://github.com/daos-stack/libfabric.git',
checkoutDir: "libfabric",
branch: commitPragma(pragma: 'libfabric-branch', def_val: 'master')
sh label: env.STAGE_NAME,
script: updatePackaging('libfabric') + '''
rm -rf artifacts/el9/
mkdir -p artifacts/el9/
make CHROOT_NAME="rocky+epel-9-x86_64" DISTRO_VERSION=9 chrootbuild'''
}
post {
success {
sh 'ls -l /var/lib/mock/rocky+epel-9-x86_64/result/'
}
unsuccessful {
sh label: "Collect artifacts",
script: '''mockroot=/var/lib/mock/rocky+epel-9-x86_64
artdir=$PWD/libfabric/artifacts/el9
cp -af _topdir/SRPMS $artdir
(cd $mockroot/result/ &&
cp -r . $artdir)
(if cd $mockroot/root/builddir/build/BUILD/*/; then
find . -name configure -printf %h\\\\n | \
while read dir; do
if [ ! -f $dir/config.log ]; then
continue
fi
tdir="$artdir/autoconf-logs/$dir"
mkdir -p $tdir
cp -a $dir/config.log $tdir/
done
fi)'''
archiveArtifacts artifacts: 'libfabric/artifacts/el9/**'
}
}
} //stage('Build libfabric on EL 9 (Fedora 42)')
stage('Build mercury on EL 8') {
agent {
dockerfile {
Expand Down Expand Up @@ -216,7 +314,106 @@ pipeline {
' --privileged=true' +
' -v /scratch:/scratch'
additionalBuildArgs dockerBuildArgs() +
'--build-arg FVERSION=37 --build-arg PACKAGINGDIR=. '
'--build-arg PACKAGINGDIR=. '
}
}
steps {
checkoutScm url: 'https://github.com/daos-stack/libfabric.git',
checkoutDir: "libfabric",
branch: commitPragma(pragma: 'libfabric-branch', def_val: 'master')
sh label: env.STAGE_NAME,
script: updatePackaging('libfabric') + '''
rm -rf artifacts/leap15/
mkdir -p artifacts/leap15/
make CHROOT_NAME="opensuse-leap-15.6-x86_64" chrootbuild'''
}
post {
success {
sh 'ls -l /var/lib/mock/opensuse-leap-15.6-x86_64/result/'
}
unsuccessful {
sh label: "Collect artifacts",
script: '''mockroot=/var/lib/mock/opensuse-leap-15.6-x86_64
artdir=$PWD/libfabric/artifacts/leap15
cp -af _topdir/SRPMS $artdir
(cd $mockroot/result/ &&
cp -r . $artdir)
(if cd $mockroot/root/builddir/build/BUILD/*/; then
find . -name configure -printf %h\\\\n | \
while read dir; do
if [ ! -f $dir/config.log ]; then
continue
fi
tdir="$artdir/autoconf-logs/$dir"
mkdir -p $tdir
cp -a $dir/config.log $tdir/
done
fi)'''
archiveArtifacts artifacts: 'libfabric/artifacts/leap15/**'
}
}
} //stage('Build libfabric on Leap 15')
stage('Build libfabric on Leap 15 (Fedora 41)') {
agent {
dockerfile {
filename 'Dockerfile.mockbuild'
label 'docker_runner'
args '--group-add mock' +
' --cap-add=SYS_ADMIN' +
' --privileged=true' +
' -v /scratch:/scratch'
additionalBuildArgs dockerBuildArgs() +
'--build-arg FVERSION=41 --build-arg PACKAGINGDIR=. '
}
}
steps {
checkoutScm url: 'https://github.com/daos-stack/libfabric.git',
checkoutDir: "libfabric",
branch: commitPragma(pragma: 'libfabric-branch', def_val: 'master')
sh label: env.STAGE_NAME,
script: updatePackaging('libfabric') + '''
rm -rf artifacts/leap15/
mkdir -p artifacts/leap15/
make CHROOT_NAME="opensuse-leap-15.6-x86_64" chrootbuild'''
}
post {
success {
sh 'ls -l /var/lib/mock/opensuse-leap-15.6-x86_64/result/'
}
unsuccessful {
sh label: "Collect artifacts",
script: '''mockroot=/var/lib/mock/opensuse-leap-15.6-x86_64
artdir=$PWD/libfabric/artifacts/leap15
cp -af _topdir/SRPMS $artdir
(cd $mockroot/result/ &&
cp -r . $artdir)
(if cd $mockroot/root/builddir/build/BUILD/*/; then
find . -name configure -printf %h\\\\n | \
while read dir; do
if [ ! -f $dir/config.log ]; then
continue
fi
tdir="$artdir/autoconf-logs/$dir"
mkdir -p $tdir
cp -a $dir/config.log $tdir/
done
fi)'''
archiveArtifacts artifacts: 'libfabric/artifacts/leap15/**'
}
}
} //stage('Build libfabric on Leap 15 (Fedora 41)')
// Leap 15.5 requires Fedora 38 or older
stage('Build libfabric on Leap 15.5 (Fedora 38)') {
agent {
dockerfile {
filename 'Dockerfile.mockbuild'
label 'docker_runner'
args '--group-add mock' +
' --cap-add=SYS_ADMIN' +
' --privileged=true' +
' -v /scratch:/scratch'
additionalBuildArgs dockerBuildArgs() +
'--build-arg FVERSION=38 --build-arg PACKAGINGDIR=. '
}
}
steps {
Expand Down Expand Up @@ -254,7 +451,7 @@ pipeline {
archiveArtifacts artifacts: 'libfabric/artifacts/leap15/**'
}
}
} //stage('Build libfabric on Leap 15')
} //stage('Build libfabric on Leap 15.5')
stage('Build libfabric on Ubuntu 20.04') {
agent {
dockerfile {
Expand Down
28 changes: 24 additions & 4 deletions scripts/repo-helper-fedora.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,26 @@ set -uex
: "${FVERSION:=latest}"
: "${REPOSITORY_NAME:=artifactory}"
: "${archive:=}"
if [ "$FVERSION" != "latest" ]; then
archive="-archive"
Copy link

@ryon-jensen ryon-jensen Jul 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what happens when 42 gets moved to archive?

Copy link
Contributor Author

@grom72 grom72 Jul 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most probably we will modify this script sooner when adding support for Fedora 43, 44.

But to be honest we should remove support for any archive version of Fedora. Archive means EOL.

Copy link
Contributor Author

@grom72 grom72 Jul 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-archive option has been removed in cc7570b

Copy link
Contributor

@JohnMalmberg JohnMalmberg Jul 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding the archive option allows us to use older than latest in case we have to pin something to an archive release.

Fedora was used because that is the version of mock that has is kept up to date with the distros. Unfortunately fedora has been dropping support for archived EL-x and EOS leap-15.x.

At one point, only fedora/38 could build leap 15.5 rpms, and we needed to go to fedora/41 to build leap 15.6 rpms, and fedora/41 dropped building the leap 15.4 rpms.

The version of the image used in the common docker file can vary for the target of el-8, el-9, leap-15, and near future el-10.

We have fedora-latest pinned to fedora/41 currently, and I would like to remove that pin or move that pin to fedora/42.

Copy link
Contributor Author

@grom72 grom72 Jul 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not see a point to build anything in scope of EOL OSes.
Based on https://en.opensuse.org/Lifetime Leap 15.5 is already EOL for more than half of year.

Copy link
Contributor

@JohnMalmberg JohnMalmberg Jul 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately our paying customer has not yet migrated to SLES 15 SP6, so needs leap 15.5 built RPMs.
We run into that in a lot of cases where we can not move to the most recent version of a distro, which is one of the reasons we are building and testing on el-8.8 instead of el-8.10 and el-8.8 has been out of support for longer than leap 15.5.

Copy link
Contributor Author

@grom72 grom72 Jul 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fedora EOL is now calculated based on date retrieved from https://endoflife.date/api/fedora.json (via artifactory)

fi

is_fedora_eol() {
local eol_url fedora_version eol_date today
if [ -n "$REPO_FILE_URL" ]; then
eol_url="${REPO_FILE_URL%repo-files/}eol-proxy/fedora.json"
fedora_version=$(grep VERSION_ID /etc/os-release | cut -d= -f2 | \
tr -d '"')
eol_date=$(curl -s "$eol_url" | sed 's/},{/}\n{/g' | \
grep "cycle\":\"$fedora_version\"" | \
sed -n 's/.*"eol":"\([^"]*\)".*/\1/p')
if [[ -z "$eol_date" ]]; then
return 1 # Assume NOT EOL if data missing
fi
today=$(date +%Y-%m-%d)
[[ "$today" > "$eol_date" ]]
return $? # Return 0 if EOL, 1 if not
else
return 1 # Assume NOT EOL if url is missing
fi
}

# shellcheck disable=SC2120
disable_repos () {
Expand Down Expand Up @@ -58,11 +75,14 @@ install_optional_ca() {
if [ -n "$REPO_FILE_URL" ]; then
install_curl
install_optional_ca
if is_fedora_eol; then
archive="-archive"
fi
mkdir -p /etc/yum.repos.d
pushd /etc/yum.repos.d/
curl -k --noproxy '*' -sSf \
-o "daos_ci-fedora${archive}-${REPOSITORY_NAME}.repo" \
"{$REPO_FILE_URL}daos_ci-fedora${archive}-${REPOSITORY_NAME}.repo"
"${REPO_FILE_URL}daos_ci-fedora${archive}-${REPOSITORY_NAME}.repo"
disable_repos /etc/yum.repos.d/
popd
fi
Expand Down