Draft: Bullseye->Bookworm migration #1759

alexAubin · 2024-01-02T20:00:35Z

cf YunoHost/issues#2050

Notes to test migration on a fresh VPS

curl https://install.yunohost.org | bash -s -- -a
# Drink coffee
yunohost tools postinstall -d bullseye-bookworm-migration.test -u camille -F Camille -p Yunohost
# Sip more coffee

# Install your favorite apps, for example:
# Nextcloud (PHP+mysql)
yunohost app install nextcloud --args "domain=bullseye-bookworm-migration.test&path=/cloud&admin=camille&init_main_permission=visitors&user_home=true&system_addressbook_exposed=yes"
# Wekan (Nodejs + mongodb) 
yunohost app install wekan --args "domain=bullseye-bookworm-migration.test&path=/wekan&admin=camille&init_main_permission=visitors"
# Terraforming mars (nodejs+postgresql) Actually requires a few GB of RAM to npm build ? :|
# yunohost app install terraforming-mars --args "domain=bullseye-bookworm-migration.test&path=/tfm&init_main_permission=visitors"
# Coin (Python + postgresql)
yunohost domain add coin.bullseye-bookworm-migration.test
yunohost app install coin --args "domain=coin.bullseye-bookworm-migration.test&admin=camille&[email protected]&isp_name=YunoHost&isp_site=yunohost.org"

# Gotta make sure we get to unstable, because only unstable packages of yunohost 12.x are available now
sed -i 's/stable/stable testing unstable/g' /etc/apt/sources.list.d/yunohost.list

# Dirty/manual apply of this PR
apt install aptitude -y
curl https://raw.githubusercontent.com/YunoHost/yunohost/migrate-to-bookworm/locales/en.json > /usr/share/yunohost/locales/en.json
curl https://raw.githubusercontent.com/YunoHost/yunohost/migrate-to-bookworm/src/utils/system.py > /usr/lib/python3/dist-packages/yunohost/utils/system.py
curl https://raw.githubusercontent.com/YunoHost/yunohost/migrate-to-bookworm/src/migrations/0027_migrate_to_bookworm.py > /usr/lib/python3/dist-packages/yunohost/migrations/0027_migrate_to_bookworm.py

# Disable check that system is up to date before migration
sed -i 's/raise.*not_fully_up_to_date.*/pass/g' /usr/lib/python3/dist-packages/yunohost/migrations/0027_migrate_to_bookworm.py

# Validate the migration is there
yunohost tools migrations list --pending

# Run the migration from the command line (OR restart yunohost-api and run the migration from the webadmin?)
yunohost tools migrations run --accept-disclaimer --debug

# Here be dragons

alexAubin · 2024-01-02T23:40:35Z

So it's working~ish on a clean install, next step is to check upgrading with a bunch of apps installed, trying to trigger tricky cases, polishing

kay0u · 2024-01-17T07:55:55Z

src/migrations/0027_migrate_to_bookworm.py

+        os.system(f'wget --timeout 900 --quiet "{new_apt_key}" --output-document=- | gpg --dearmor >"/usr/share/keyrings/yunohost-bookworm.gpg"')
+
+        # Add Sury key even if extra_php_version.list was already there,
+        # because some old system may be using an outdated key not valid for Bookworm


Just reading the comments without understanding what an old system and an outdated key mean, but I feel that's not really linked to the migration and YNH should handle this key somewhere else.
I mean, we didn't change how sury is integrated to YNH in bookworm compared to bullseye?

kay0u · 2024-01-17T07:58:29Z

src/migrations/0027_migrate_to_bookworm.py

+        # Critical fix for RPI otherwise network is down after rebooting
+        # https://forum.yunohost.org/t/20652
+        #
+        # FIXME : this is from buster->bullseye, do we still needed it ?


Just a reminder, we should check if the problem is still here

kay0u · 2024-01-17T08:04:48Z

src/migrations/0027_migrate_to_bookworm.py

+        # Stupid hack for stupid dnsmasq not picking up its new init.d script then breaking everything ...
+        # https://forum.yunohost.org/t/20676
+        #
+        # FIXME : this is from buster->bullseye, do we still needed it ?


To be checked during the testing phase

kay0u · 2024-01-17T08:06:51Z

src/migrations/0027_migrate_to_bookworm.py

+        #cmd = "LC_ALL=C"
+        #cmd += " DEBIAN_FRONTEND=noninteractive"
+        #cmd += " APT_LISTCHANGES_FRONTEND=none"
+        #cmd += " apt dist-upgrade "
+        #cmd += " --quiet -o=Dpkg::Use-Pty=0 --fix-broken --dry-run"
+        #cmd += " | grep -q 'ynh-deps'"
+
+        #logger.info("Simulating upgrade...")
+        #if os.system(cmd) == 0:
+        #    raise YunohostError(
+        #        "The upgrade cannot be completed, because some app dependencies would need to be removed?",
+        #        raw_msg=True,
+        #    )
+
+        # FIXME :
+        #postupgradecmds = "rm -f /usr/sbin/policy-rc.d\n"
+        #postupgradecmds += "echo 'Restarting nginx...' >&2\n"
+        #postupgradecmds += "systemctl restart nginx\n"


Wut, can we remove that?

kay0u · 2024-01-17T08:12:13Z

src/migrations/0027_migrate_to_bookworm.py

+        conflicts = check_output("dpkg-query -s yunohost | grep '^Conflicts:'").strip()
+        if conflicts:
+            # We want to keep conflicting with apache/bind9 tho
+            new_conflicts = "Conflicts: apache2, bind9"


What about other conflicts ? In yunohost, admin, ssowat etc?

tituspijean · 2024-05-10T20:43:25Z

I have just upgraded a server from YunoHost 11.2.11.3 to 12 after following the instructions of the first message.

Migration runs smoothly, but the logging has a bug:

...
690523 SUCCESS Migration 0027_migrate_to_bookworm completed
690547 DEBUG action [68666.1] executed in 690.171s
690548 DEBUG lock has been released
Traceback (most recent call last):
  File "/usr/bin/yunohost", line 77, in <module>
    yunohost.cli(
  File "/usr/lib/python3/dist-packages/yunohost/__init__.py", line 41, in cli
    ret = moulinette.cli(
  File "/usr/lib/python3/dist-packages/moulinette/__init__.py", line 110, in cli
    load_only_category = args[0] if args and not args[0].startswith("-") else None
  File "/usr/lib/python3/dist-packages/moulinette/interfaces/cli.py", line 503, in run
    raise MoulinetteError("operation_interrupted")
  File "/usr/lib/python3/dist-packages/moulinette/actionsmap.py", line 580, in process
    - top_parser -- A BaseActionsMapParser-derived instance to use for
  File "/usr/lib/python3/dist-packages/yunohost/tools.py", line 825, in tools_migrations_run
    return sorted(migrations, key=lambda m: m.id)
  File "/usr/lib/python3/dist-packages/yunohost/log.py", line 671, in success
    """
  File "/usr/lib/python3/dist-packages/yunohost/log.py", line 709, in close
    msg = m18n.n("log_link_to_log", name=self.name, desc=desc)
  File "/usr/lib/python3/dist-packages/yunohost/log.py", line 797, in _get_description_from_name
    args = parts[1:]
ModuleNotFoundError: No module named '_strptime'

centralscrutinizer21 · 2024-06-21T19:24:06Z

ok since I broke my matrix server for the n time I am going to be bad with my server and try this upgrade with several apps installed:

calibreweb
bludit
bookwyrm
castopod
cinny
collabora
coturn
element-call
invidious
libreddit
mautrix_signal
mautrix_telegram
mautrix_whatsapp
miniflux
nextcloud
yourls
pleroma
peertube
my_webapp
pixelfed
rss-bridge
navidrome
umami
vaultwarden
wallabag2
webtrees
wireguard
searxng
ntfy

It's a sacrifice for the greater good.
I have backups for the most important apps and media files in separate storages
I'll report the results

Salamandar · 2024-06-26T17:56:28Z

I have just upgraded a server from YunoHost 11.2.11.3 to 12 after following the instructions of the first message.

Migration runs smoothly, but the logging has a bug:

Same, and nginx is left dead

Jun 26 17:45:22 intra.syslinbit.com systemd[1]: Reload failed for nginx.service - A high performance web server and a reverse proxy server.

Restarting the nginx service works.

Roundcube seems broken though.

EDIT : Ah I had roundcube and snappymail installed before the migration, and snappymail is still working :)

…tion doesnt appear as failed from the webamin

…r some reason it gets reset?

Salamandar · 2024-07-04T17:44:03Z

Webadmin issue : Clicking on this button changes the URL visible in the browser but doesn't actually redirect to the page…

src/migrations/0027_migrate_to_bookworm.py

src/utils/system.py

… the migration

…'s causing too many issues in dependency resolution idk

…, can't use raw bash redirects, gotta use stdin= from subprocess ... and we want only a limited number of 'yes' and not an infinite yes like the -y option does resuling in conflict resolution loops

Salamandar · 2024-07-08T21:18:55Z

As is, this PR can be merged without risk.

alexAubin · 2024-07-08T21:21:22Z

https://www.youtube.com/watch?v=AWM5ZNdWlqw

alexAubin added the 🏗️ Major project Big decision label Jan 2, 2024

alexAubin marked this pull request as draft January 2, 2024 20:01

alexAubin force-pushed the migrate-to-bookworm branch 4 times, most recently from c07aefa to 4bdf982 Compare January 2, 2024 23:05

alexAubin mentioned this pull request Jan 14, 2024

Support Debian 12 "Bookworm" YunoHost/issues#2050

Closed

38 tasks

kay0u reviewed Jan 17, 2024

View reviewed changes

First draft for migrate_to_bookworm

383fd6f

alexAubin force-pushed the migrate-to-bookworm branch from 4bdf982 to 383fd6f Compare April 10, 2024 19:14

Merge remote-tracking branch 'origin/dev' into migrate-to-bookworm

307ed10

alexAubin added 4 commits July 3, 2024 17:24

Merge remote-tracking branch 'origin/dev' into migrate-to-bookworm

9a5aff9

Backport i18n string + code for bookworm migration

c6aec68

bullseye->bookorm: delay the yunohost-api restart such that the migra…

772e772

…tion doesnt appear as failed from the webamin

bullseye->bookworm: force-regen the nsswitch configuration because fo…

c694ea2

…r some reason it gets reset?

Salamandar reviewed Jul 4, 2024

View reviewed changes

src/migrations/0027_migrate_to_bookworm.py Outdated Show resolved Hide resolved

Salamandar reviewed Jul 4, 2024

View reviewed changes

src/utils/system.py Show resolved Hide resolved

Fix missing import of moulinette.Moulinette

f344cb0

Salamandar force-pushed the migrate-to-bookworm branch from b8c25db to f344cb0 Compare July 4, 2024 19:02

Salamandar and others added 5 commits July 4, 2024 21:26

Add missing from time import sleep ; also restart nginx at the end of…

90d4cd9

… the migration

bullseye->bookworm: dirty hack to explicitly remove rspamd because it…

2763e04

…'s causing too many issues in dependency resolution idk

bullseye->bookworm: more tweaks for the 'assume yes' in aptitude call…

0f34d7e

…, can't use raw bash redirects, gotta use stdin= from subprocess ... and we want only a limited number of 'yes' and not an infinite yes like the -y option does resuling in conflict resolution loops

Add aptitude to deps for the migration to bookworm

26fba08

Merge branch 'dev' into migrate-to-bookworm

b289de3

Salamandar mentioned this pull request Jul 8, 2024

Prepare migration to bookworm #1900

Closed

Disable migration to bookworm until it is ready

4996114

alexAubin marked this pull request as ready for review July 8, 2024 21:21

alexAubin merged commit a735a6d into dev Jul 8, 2024
2 of 3 checks passed

alexAubin deleted the migrate-to-bookworm branch July 8, 2024 21:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Draft: Bullseye->Bookworm migration #1759

Draft: Bullseye->Bookworm migration #1759

alexAubin commented Jan 2, 2024 •

edited

Loading

alexAubin commented Jan 2, 2024

kay0u Jan 17, 2024

kay0u Jan 17, 2024

kay0u Jan 17, 2024

kay0u Jan 17, 2024

kay0u Jan 17, 2024

tituspijean commented May 10, 2024

centralscrutinizer21 commented Jun 21, 2024

Salamandar commented Jun 26, 2024 •

edited

Loading

Salamandar commented Jul 4, 2024

Salamandar commented Jul 8, 2024

alexAubin commented Jul 8, 2024

Draft: Bullseye->Bookworm migration #1759

Draft: Bullseye->Bookworm migration #1759

Conversation

alexAubin commented Jan 2, 2024 • edited Loading

Notes to test migration on a fresh VPS

alexAubin commented Jan 2, 2024

kay0u Jan 17, 2024

Choose a reason for hiding this comment

kay0u Jan 17, 2024

Choose a reason for hiding this comment

kay0u Jan 17, 2024

Choose a reason for hiding this comment

kay0u Jan 17, 2024

Choose a reason for hiding this comment

kay0u Jan 17, 2024

Choose a reason for hiding this comment

tituspijean commented May 10, 2024

centralscrutinizer21 commented Jun 21, 2024

Salamandar commented Jun 26, 2024 • edited Loading

Salamandar commented Jul 4, 2024

Salamandar commented Jul 8, 2024

alexAubin commented Jul 8, 2024

alexAubin commented Jan 2, 2024 •

edited

Loading

Salamandar commented Jun 26, 2024 •

edited

Loading