CI: Test migrations

[4ydan]

#134

Basics

• I added a line to /doc/CHANGELOG.md
• The PR is rebased with current master.
• Details of what you changed are in commit messages.
• References to issues, e.g. close #X, are in the commit messages.
• The buildserver is happy.

Checklist

• I have installed and I am using pre-commit hooks
• I fully described what my PR does in the documentation
  (not in the PR description)
• I fixed all affected documentation
• I fixed all affected decisions
• I added unit tests for my code
• I added code comments, logging, and assertions as appropriate
• I translated all strings visible to the user
• I mentioned every code or binary not directly written or done by me in reuse syntax
• I created left-over issues for things that are still to be done
• Code is conforming to our Architecture
• Code is conforming to our Guidelines
  (exceptions are documented)
• Code is consistent to our Design Decisions

Review

• I've tested the code
• I've read through the whole code
• Examples are well chosen and understandable

[markus2330]

Good to finally get this! 🚀

[4ydan]

Can we do the migration check before building?

[markus2330]

Probably yes, but I don't know.

[4ydan]

Ok so now we need proper rights, is that to be fixed in a separate PR and we wait for the merge?
Or how should I modify the script in CI.

[markus2330]

Ok so now we need proper rights, is that to be fixed in a separate PR and we wait for the merge?

I don't think it is a permission problem. It rather looks like the migrations are broken: the PostGIS extension is tried to be removed but not yet all objects that use PostGIS are removed. (At least I interpret the error message like this.)

I tried to reproduce and got another problem. I created #674 to track these issues.

Probably, we need to wait for #644 (or even longer) until these problems are resolved...

[markus2330]

jenkins build please

[markus2330]

Let us see if #644 improved the situation...

[markus2330]

jenkins build please

[markus2330]

We again have Failed to run 2023-07-01-152628_squashed_setup with: cannot drop extension postgis because other objects depend on it here. I cannot reproduce it locally. Maybe it is a permission problem after all?

@temmey do you maybe know what this error is?

[4ydan]

Changing DROP EXTENSION postgis; to DROP EXTENSION postgis CASCADE; solves the problem, but not sure if we want this. I also can't reproduce it locally.

[markus2330]

Great that you found a way!

According to https://www.postgresql.org/docs/current/sql-dropextension.html

Automatically drop objects that depend on the extension, and in turn all objects that depend on those objects (see Section 5.14).

As we also reset the whole database on pr builds, afaik there cannot be any harm be removing any objects in the database. On dev (master) or production builds, it might be a problem but there we don't test redo -a anyway and then it doesn't occur, does it?

[4ydan]

Yes I agree, but somehow it still bothers me that there is something.

it might be a problem but there we don't test redo -a anyway and then it doesn't occur, does it?

True, it only occured with make migration-redo-a.

[4ydan]

I guess it will help more than it could do bad. Because finding the issue for that error might take some time in which other PR's could fail migrations maybe? Maybe we can leave an open issue that aims to remove the CASCADE again at some point?

[markus2330]

Yes, let us merge this, a rebase is needed, though.

I don't think an issue is needed. It is okay if data gets lost on pr or dev.

[4ydan]

Ready to be merged.

[markus2330]

Thank you! ❤️