Skip to content
This repository has been archived by the owner on Oct 29, 2024. It is now read-only.

A pass at bumping the buildbot versioning #237

Closed
wants to merge 2 commits into from
Closed

A pass at bumping the buildbot versioning #237

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
51878a7
A pass at bumping the buildbot versioning
rincebrain Oct 28, 2021
15e7bd8
Whoops
rincebrain Oct 28, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,10 +47,15 @@ specific builders they should be enumerated as a comma separated list.
* __amazon2__: Amazon Linux 2 (x86_64)
* __centos6__: CentOS 6 (x86_64)
* __centos7__: CentOS 7 (x86_64)
* __debian10__: Debian 10 (x86_64)
* __fedora30__: Fedora 30 (x86_64)
* __ubuntu16__: Ubuntu 16.04 LTS (x86_64)
* __centos8__: AlmaLinux 8 (x86_64)
* __centos8stream__: CentOS 8 Stream (x86_64)
* __debian8__: Debian 8 (armel, ppc64)
* __debian10__: Debian 10 (x86_64, arm64)
* __debian11__: Debian 11 (x86_64, arm64)
* __fedora34__: Fedora 34 (x86_64)
* __ubuntu16__: Ubuntu 16.04 LTS (i386)
* __ubuntu18__: Ubuntu 18.04 LTS (x86_64)
* __ubuntu20__: Ubuntu 20.04 LTS (x86_64)
* __freebsd12__: FreeBSD 12 (x86_64)
* __freebsd13__: FreeBSD 13 (x86_64)
* __freebsd14__: FreeBSD 14 (x86_64)
Expand Down
2 changes: 1 addition & 1 deletion master/github.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
from twisted.python import log

builders_common="arch,"
builders_linux="centos7,centos8,centosstream8,debian10,fedora33,builtin,"
builders_linux="centos7,centos8,centosstream8,debian10,debian11,fedora34,builtin,"
builders_freebsd="freebsd12,freebsd13,freebsd14"

builders_push_master=builders_common+builders_linux+builders_freebsd+"coverage"
Expand Down
79 changes: 61 additions & 18 deletions master/master.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -480,15 +480,17 @@ centos8_ami = "ami-0473bc9e4bfe85b1b"
centosstream8_ami = "ami-0f377b303df4963ab" # CentOS Stream 8 1-06-2021

# Provided by Debian: https://wiki.debian.org/Cloud/AmazonEC2Image
debian10_amd64_ami = "ami-0ed1af421f2a3cf40" # Debian 10 9-9-2019
debian10_arm64_ami = "ami-0a61b13f06119b46c" # Debian 10 9-28-2020
debian10_amd64_ami = "ami-024fe42989cf9e876" # Debian 10 10-11-2021
debian10_arm64_ami = "ami-0a0ab0f23e5ea98f9" # Debian 10 10-11-2021
debian11_amd64_ami = "ami-09b4378b1d3387f81" # Debian 11 10-11-2021
debian11_arm64_ami = "ami-041b92da75e2787a9" # Debian 11 10-11-2021
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, I ran in to some issues trying to get the buildslave running on Debian 11 when installed from pip. The 0.8 release requires an old version of python and I wasn't having any luck getting it to run using update-alternatives. This will require a little more investigation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That could be rough - Debian 11 dropped basically every Python 2 package except the actual interpreter, so you'd have to go slurp everything from Pip or something if you need it...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fun. Well I suppose all the more reason for us finally to roll buildbot forward to a modern version. Then why don't we put this on the back burner for now.


# Provided by Fedora: https://alt.fedoraproject.org/cloud/
fedora33_ami = "ami-03cd6589f3954c8ff" # Fedora 33 10-28-2020
fedora34_ami = "ami-0d828c0715f284b51" # Fedora 34 ??-??-????

# Provided by Cannonical: https://cloud-images.ubuntu.com/locator/ec2/
ubuntu18_ami = "ami-03c9dad75296f9e90" # Ubuntu 18.04 4-26-2018
ubuntu20_ami = "ami-036e9fc1037b68e21" # Ubuntu 20.04 6-19-2020
# Provided by Canonical: https://cloud-images.ubuntu.com/locator/ec2/
ubuntu18_ami = "ami-0461281d457e8aaeb" # Ubuntu 18.04 9-28-2021
ubuntu20_ami = "ami-053ac55bdcfe96e85" # Ubuntu 20.04 10-21-2021

# Custom AMIs
coverage_ami = "ami-a97867c9" # Ubuntu 17.04 5-2-2018
Expand Down Expand Up @@ -539,6 +541,13 @@ debian10_arm64_slave = [
arch="arm64"
) for i in range(0, numarchslaves)
]
debian11_arm64_slave = [
ZFSEC2BuildSlave(
name="Debian-11-arm64-buildslave%s" % (str(i+1)),
ami=debian11_arm64_ami,
arch="arm64"
) for i in range(0, numarchslaves)
]

debian8_ppc64_slave = [
BuildSlave("Debian-8-ppc64-buildslave1",
Expand All @@ -551,6 +560,7 @@ arch_slaves = \
ubuntu16_i386_slave + \
debian8_arm_slave + \
debian10_arm64_slave + \
debian11_arm64_slave + \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously we just kept one builder per architecture to help catch anything major for that architecture. But since these builds are quick I think we can keep both in this case. It'll be nice to verify both kernels.

debian8_ppc64_slave

#
Expand Down Expand Up @@ -593,10 +603,17 @@ debian10_x86_64_testslave = [
) for i in range(0, numtestslaves)
]

fedora33_x86_64_testslave = [
debian11_x86_64_testslave = [
ZFSEC2TestSlave(
name="Debian-11-x86_64-testslave%s" % (str(i+1)),
ami=debian11_amd64_ami
) for i in range(0, numtestslaves)
]

fedora34_x86_64_testslave = [
ZFSEC2TestSlave(
name="Fedora-33-x86_64-testslave%s" % (str(i+1)),
ami=fedora33_ami
name="Fedora-34-x86_64-testslave%s" % (str(i+1)),
ami=fedora34_ami
) for i in range(0, numtestslaves)
]

Expand Down Expand Up @@ -641,7 +658,8 @@ test_slaves = \
centos8_x86_64_testslave + \
centosstream8_x86_64_testslave + \
debian10_x86_64_testslave + \
fedora33_x86_64_testslave + \
debian11_x86_64_testslave + \
fedora34_x86_64_testslave + \
ubuntu18_x86_64_testslave + \
ubuntu20_x86_64_testslave + \
freebsd12_amd64_testslave + \
Expand Down Expand Up @@ -876,6 +894,13 @@ arch_builders = [
tags=arch_tags,
properties=builder_arch_properties,
),
ZFSBuilderConfig(
name="Debian 11 arm64 (BUILD)",
factory=build_factory,
slavenames=[slave.slavename for slave in debian11_arm64_slave],
tags=arch_tags,
properties=builder_arch_properties,
),
ZFSBuilderConfig(
name="Debian 8 ppc64 (BUILD)",
factory=build_factory,
Expand Down Expand Up @@ -938,11 +963,21 @@ debian_10_builders = [
),
]

fedora_33_builders = [
debian_11_builders = [
ZFSBuilderConfig(
name="Fedora 33 x86_64 (TEST)",
name="Debian 11 x86_64 (TEST)",
factory=test_factory,
slavenames=[slave.name for slave in fedora33_x86_64_testslave],
slavenames=[slave.name for slave in debian11_x86_64_testslave],
tags=platform_tags,
properties=builder_default_properties,
),
]

fedora_34_builders = [
ZFSBuilderConfig(
name="Fedora 34 x86_64 (TEST)",
factory=test_factory,
slavenames=[slave.name for slave in fedora34_x86_64_testslave],
tags=platform_tags,
properties=builder_default_properties,
),
Expand Down Expand Up @@ -1004,7 +1039,8 @@ test_builders = \
centos_8_builders + \
centosstream_8_builders + \
debian_10_builders + \
fedora_33_builders + \
debian_11_builders + \
fedora_34_builders + \
ubuntu_18_builders + \
ubuntu_20_builders + \
freebsd_12_builders + \
Expand All @@ -1015,7 +1051,8 @@ nightly_builders = \
centos_7_builders + \
centos_8_builders + \
debian_10_builders + \
fedora_33_builders + \
debian_11_builders + \
fedora_34_builders + \
ubuntu_18_builders + \
freebsd_12_builders + \
freebsd_13_builders
Expand Down Expand Up @@ -1181,10 +1218,16 @@ c['schedulers'].append(CustomSingleBranchScheduler(
change_filter=filter.ChangeFilter(category_re=".*debian10.*")))

c['schedulers'].append(CustomSingleBranchScheduler(
name="pull-request-fedora-33-scheduler",
builderNames=[builder.name for builder in fedora_33_builders],
name="pull-request-debian-11-scheduler",
builderNames=[builder.name for builder in debian_11_builders],
codebases=default_codebases,
change_filter=filter.ChangeFilter(category_re=".*debian11.*")))

c['schedulers'].append(CustomSingleBranchScheduler(
name="pull-request-fedora-34-scheduler",
builderNames=[builder.name for builder in fedora_34_builders],
codebases=default_codebases,
change_filter=filter.ChangeFilter(category_re=".*fedora33.*")))
change_filter=filter.ChangeFilter(category_re=".*fedora34.*")))

c['schedulers'].append(CustomSingleBranchScheduler(
name="pull-request-ubuntu-18-scheduler",
Expand Down
2 changes: 1 addition & 1 deletion scripts/bb-bootstrap.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -323,7 +323,7 @@ Fedora*)
if test $BB_USE_PIP -ne 0; then

# Python 2 has been removed from Fedora 32. The required pip2
# pacakages are still provided by the UnitedRPMs repository.
# packages are still provided by the UnitedRPMs repository.
if test $VERSION -ge 32; then
rpm --import https://raw.githubusercontent.com/UnitedRPMs/unitedrpms/master/URPMS-GPG-PUBLICKEY-Fedora
dnf -y install https://github.com/UnitedRPMs/unitedrpms/releases/download/17/unitedrpms-$(rpm -E %fedora)-17.fc$(rpm -E %fedora).noarch.rpm
Expand Down
7 changes: 5 additions & 2 deletions scripts/known-issues.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -238,8 +238,11 @@ function build_url()
Debian_10_x86_64__TEST_)
encoded_name="Debian%2010%20x86_64%20%28TEST%29"
;;
Fedora_33_x86_64__TEST_)
encoded_name="Fedora%2033%20x86_64%20%28TEST%29"
Debian_11_x86_64__TEST_)
encoded_name="Debian%2011%20x86_64%20%28TEST%29"
;;
Fedora_34_x86_64__TEST_)
encoded_name="Fedora%2034%20x86_64%20%28TEST%29"
;;
FreeBSD_stable_12_amd64__TEST_)
encoded_name="FreeBSD%20stable%2F12%20amd64%20%28TEST%29"
Expand Down