Skip to content

Latest commit

 

History

History
165 lines (118 loc) · 7.39 KB

File metadata and controls

165 lines (118 loc) · 7.39 KB

🗄️ Database Changes

ThesisManagement uses Liquibase for managing database schema changes. All schema modifications must be tracked and versioned properly to ensure consistency across development, testing, and production environments.

📋 How to Apply a Schema Change

  1. Create a new changeset by adding a SQL script in the /server/src/main/resources/db/changelog/changes directory.
  2. Use descriptive and unique filenames (e.g. 09_new_feature_name.sql).
  3. Include the new file in the db.changelog-master.xml to ensure it is executed.
  4. Stick to the Liquibase formatted SQL format:
    --liquibase formatted sql
    --changeset yourname:unique-id

🏷️ Naming Conventions

  • Prefix filenames with a two-digit incremental number to preserve execution order.
  • Use lowercase and underscores (e.g. 10_add_user_flags.sql).
  • Use your name or GitHub handle in the changeset author (e.g. --changeset marc:10-add-user-flags).

🧪 Local Development Notes

  • Liquibase will auto-run on application start.
  • Always test schema changes with a local PostgreSQL instance before pushing.

🧑‍💻 Manual SQL Scripts

Some SQL scripts are not part of the Liquibase process. These are intended for manual execution in specific environments (e.g. TUM-specific data).

📄 Example:

This file manually adds the "Applied Education Technologies (AET)" research group, only applicable for TUM environments (DEV & PROD). Do not include in Liquibase.

🔄 Major Version Upgrade (pg_dump/pg_restore)

PostgreSQL major versions (e.g. 17 → 18) use different internal data formats, so the new server cannot read data files written by the old one. You must export the data with pg_dump and re-import it with pg_restore.

PGDATA Path Change in PG 18+

Starting with PostgreSQL 18, the official Docker image changed the default PGDATA from /var/lib/postgresql/data to /var/lib/postgresql/18/docker (docker-library/postgres#1259). Our production compose file explicitly sets PGDATA=/var/lib/postgresql/data to keep existing volume mounts working.

Canonical PostgreSQL Version

The PostgreSQL image tag is 18.4-alpine. The committed .env (POSTGRES_IMAGE_TAG) is the single source of truth: Docker Compose loads it automatically and the Gradle build parses it for the integration-test Testcontainers. Every consumer falls back to the same hard-coded default if the variable is unset, so the workspace builds even without .env:

Consumer Reads the version from Fallback default
docker-compose.yml, docker-compose.prod.yml, docker-compose.showcase.yml committed .env (auto-loaded); .env.prod for production the ${POSTGRES_IMAGE_TAG:-18.4-alpine} inline fallback in each compose file
Integration-test Testcontainers (BaseIntegrationTest, BaseKeycloakIntegrationTest via TestContainerImages) committed .env, parsed in server/build.gradle (or a POSTGRES_IMAGE_TAG env var) the System.getProperty(…, "18.4-alpine") fallback in TestContainerImages
.github/workflows/e2e_tests.yml (CI service container) POSTGRES_IMAGE_TAG repository variable (Settings → Secrets and variables → Actions → Variables) — the workflow services.image field cannot read .env the `${{ vars.POSTGRES_IMAGE_TAG

POSTGRES_IMAGE_TAG is not a secret — the committed .env must hold non-secret config only.

To bump the version, edit POSTGRES_IMAGE_TAG in the committed .env — that updates both Docker Compose and the Gradle tests at once. For full reproducibility also update the matching hard-coded fallbacks (the ${POSTGRES_IMAGE_TAG:-…} in the three compose files, the workflow's || '18.4-alpine', and the default in TestContainerImages), then run the major-upgrade procedure below for production. The GitHub repository variable is optional and only needed to override CI without a code change.

Upgrade Procedure

Always test the upgrade on the dev environment first before applying to production.

All commands must be run as the thesistrack user in /home/thesistrack/ on the VM. SSH in, then switch to the correct user:

sudo su
su thesistrack
cd ~

Then run the upgrade

# 1. Stop the application server (keep DB running)
docker compose -f docker-compose.prod.yml --env-file=.env.prod stop server client

# 2. Create a full database dump
docker exec thesis-management-db pg_dump -Fc -U "thesistrack" "thesistrack" > thesis_dump.dump

# 3. Stop and remove the old DB container
docker compose -f docker-compose.prod.yml --env-file=.env.prod down db

# 4. Back up the old data directory (do NOT delete it yet)
mv ./postgres_data ./postgres_data_backup

# 5. Point the deployment at the new PostgreSQL image tag. All compose files read
#    ${POSTGRES_IMAGE_TAG} (see "Canonical PostgreSQL Version" above), so set it once in .env.prod
#    instead of editing the compose file. Replace 18.4-alpine with the target tag.
echo 'POSTGRES_IMAGE_TAG=18.4-alpine' >> .env.prod
#    (or edit the existing POSTGRES_IMAGE_TAG line if it is already present)

# 6. Start only the database service
docker compose -f docker-compose.prod.yml --env-file=.env.prod up -d db

# 7. Wait for the database to be ready
until docker exec thesis-management-db pg_isready -U "thesistrack"; do sleep 1; done

# 8. Copy the dump into the new container and restore
docker cp thesis_dump.dump thesis-management-db:/tmp/thesis_dump.dump
docker exec thesis-management-db pg_restore -U "thesistrack" -d "thesistrack" --no-owner --no-acl --exit-on-error /tmp/thesis_dump.dump

# 9. Verify data integrity
docker exec thesis-management-db psql -U "thesistrack" -d "thesistrack" -c "\dt+"

# 10. Start the full application stack
docker compose -f docker-compose.prod.yml --env-file=.env.prod up -d

# 11. Verify Hibernate validation and Liquibase pass
docker logs -f thesis-management-server  # check for errors

# 12. After 1-2 weeks of stable operation, remove the old data directory and dump
rm -rf ./postgres_data_backup thesis_dump.dump

Rollback

If the upgrade fails, restore the previous version:

docker compose -f docker-compose.prod.yml --env-file=.env.prod down db
rm -rf ./postgres_data
mv ./postgres_data_backup ./postgres_data
# Revert docker-compose.prod.yml changes (PG image tag and PGDATA override)
docker compose -f docker-compose.prod.yml --env-file=.env.prod up -d

References

🗺️ Database Schema

Database Schema

Always regenerate this diagram after structural changes to ensure documentation accuracy.