Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

migration bug fixed V0_7_7__update.sql #1394

Closed
wants to merge 1 commit into from
Closed

migration bug fixed V0_7_7__update.sql #1394

wants to merge 1 commit into from

Conversation

mahmudarslan
Copy link

release version : steve-3.6.0

docker-compose up
steve-migration-7

2024-02-23 17:31:30 [INFO] BUILD FAILURE
2024-02-23 17:31:30 [INFO] ------------------------------------------------------------------------
2024-02-23 17:31:30 [INFO] Total time: 56.074 s
2024-02-23 17:31:30 [INFO] Finished at: 2024-02-23T14:31:30Z
2024-02-23 17:31:30 [INFO] ------------------------------------------------------------------------
2024-02-23 17:31:30 [WARNING]
2024-02-23 17:31:30 [WARNING] Plugin validation issues were detected in 2 plugin(s)
2024-02-23 17:31:30 [WARNING]
2024-02-23 17:31:30 [WARNING] * pl.project13.maven:git-commit-id-plugin:4.9.10
2024-02-23 17:31:30 [WARNING] * org.flywaydb:flyway-maven-plugin:7.15.0
2024-02-23 17:31:30 [WARNING]
2024-02-23 17:31:30 [WARNING] For more or less details, use 'maven.plugin.validation' property with one of the values (case insensitive): [BRIEF, DEFAULT, VERBOSE]
2024-02-23 17:31:30 [WARNING]
2024-02-23 17:31:30 [ERROR] Failed to execute goal org.flywaydb:flyway-maven-plugin:7.15.0:migrate (default) on project steve: org.flywaydb.core.internal.command.DbMigrate$FlywayMigrateException: Migration V0_7_7__update.sql failed
2024-02-23 17:31:30 [ERROR] -----------------------------------
2024-02-23 17:31:30 [ERROR] SQL State : HY000
2024-02-23 17:31:30 [ERROR] Error Code : 1833
2024-02-23 17:31:30 [ERROR] Message : Cannot change column 'chargeBoxId': used in a foreign key constraint 'stevedb/FK_chargeBoxId_c' of table 'stevedb/connector'
2024-02-23 17:31:30 [ERROR] Location : /code/src/main/resources/db/migration/V0_7_7__update.sql (/code/src/main/resources/db/migration/V0_7_7__update.sql)
2024-02-23 17:31:30 [ERROR] Line : 1
2024-02-23 17:31:30 [ERROR] Statement : ALTER TABLE chargebox
2024-02-23 17:31:30 [ERROR] CHANGE COLUMN chargeBoxId chargeBoxId VARCHAR(255) NOT NULL COMMENT '' ,
2024-02-23 17:31:30 [ERROR] CHANGE COLUMN endpoint_address endpoint_address VARCHAR(255) NULL DEFAULT NULL COMMENT '' ,
2024-02-23 17:31:30 [ERROR] CHANGE COLUMN ocppProtocol ocppProtocol VARCHAR(255) NULL DEFAULT NULL COMMENT '' ,
2024-02-23 17:31:30 [ERROR] CHANGE COLUMN chargePointVendor chargePointVendor VARCHAR(255) NULL DEFAULT NULL COMMENT '' ,
2024-02-23 17:31:30 [ERROR] CHANGE COLUMN chargePointModel chargePointModel VARCHAR(255) NULL DEFAULT NULL COMMENT '' ,
2024-02-23 17:31:30 [ERROR] CHANGE COLUMN chargePointSerialNumber chargePointSerialNumber VARCHAR(255) NULL DEFAULT NULL COMMENT '' ,
2024-02-23 17:31:30 [ERROR] CHANGE COLUMN chargeBoxSerialNumber chargeBoxSerialNumber VARCHAR(255) NULL DEFAULT NULL COMMENT '' ,
2024-02-23 17:31:30 [ERROR] CHANGE COLUMN fwVersion fwVersion VARCHAR(255) NULL DEFAULT NULL COMMENT '' ,
2024-02-23 17:31:30 [ERROR] CHANGE COLUMN fwUpdateStatus fwUpdateStatus VARCHAR(255) NULL DEFAULT NULL COMMENT '' ,
2024-02-23 17:31:30 [ERROR] CHANGE COLUMN iccid iccid VARCHAR(255) NULL DEFAULT NULL COMMENT '' ,
2024-02-23 17:31:30 [ERROR] CHANGE COLUMN imsi imsi VARCHAR(255) NULL DEFAULT NULL COMMENT '' ,
2024-02-23 17:31:30 [ERROR] CHANGE COLUMN meterType meterType VARCHAR(255) NULL DEFAULT NULL COMMENT '' ,
2024-02-23 17:31:30 [ERROR] CHANGE COLUMN meterSerialNumber meterSerialNumber VARCHAR(255) NULL DEFAULT NULL COMMENT '' ,
2024-02-23 17:31:30 [ERROR] CHANGE COLUMN diagnosticsStatus diagnosticsStatus VARCHAR(255) NULL DEFAULT NULL COMMENT ''
2024-02-23 17:31:30 [ERROR]
2024-02-23 17:31:30 [ERROR] -> [Help 1]
2024-02-23 17:31:30 [ERROR]
2024-02-23 17:31:30 [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
2024-02-23 17:31:30 [ERROR] Re-run Maven using the -X switch to enable full debug logging.
2024-02-23 17:31:30 [ERROR]
2024-02-23 17:31:30 [ERROR] For more information about the errors and possible solutions, please read the following articles:
2024-02-23 17:31:30 [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

@goekay
Copy link
Member

goekay commented Feb 25, 2024

why do you think there is a problem? what specific problem does the commit fix?

@fulup-bzh
Copy link

fulup-bzh commented Feb 26, 2024 •
edited
Loading

I have the same error as you, on a vanilla new installation. I aligned on your branch to get your patch but I still have an error.

  • Debian-12
  • jdk-17.0.10
  • mariadb-10.11

Error: ERROR] Migration of schema stevedb to version "0.8.6 - update" failed! Please restore backups and roll back database

Detail logs

My source

commit c26e0b336f62b5e946e81afe4fb5e4bb2ca79f49 (HEAD -> mahmudarslan/master, mahmudarslan/master)
Author: mahmudarslan <[email protected]>
Date:   Fri Feb 23 17:28:08 2024 +0300
    migration bug fixed V0_7_7__update.sql

Command log

./mvnw package
[INFO] Scanning for projects...
[INFO] 
[INFO] -------------------------< de.rwth.idsg:steve >-------------------------
[INFO] Building SteVe 3.6.0
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- enforcer:3.4.1:enforce (enforce-java) @ steve ---
[INFO] Rule 0: org.apache.maven.enforcer.rules.version.RequireJavaVersion passed
[INFO] Rule 1: org.apache.maven.enforcer.rules.version.RequireMavenVersion passed
[INFO] 
[INFO] --- git-commit-id:7.0.0:revision (default) @ steve ---
[INFO] 
[INFO] --- properties:1.2.1:read-project-properties (default) @ steve ---
[INFO] Loading 24 properties from File: /root/ocpp-steve/steve/src/main/resources/config/prod/main.properties
[INFO] 
[INFO] --- flyway:10.8.1:migrate (default) @ steve ---
[INFO] Database: jdbc:mysql://localhost:3306/stevedb (MySQL 5.5)
[INFO] Schema history table `stevedb`.`schema_version` does not exist yet
[INFO] Successfully validated 36 migrations (execution time 00:00.012s)
[INFO] Creating Schema History table `stevedb`.`schema_version` ...
[INFO] Current version of schema `stevedb`: << Empty Schema >>
[WARNING] outOfOrder mode is active. Migration of schema `stevedb` may not be reproducible.
[INFO] Migrating schema `stevedb` to version "0.6.6 - inital"
[WARNING] DB: Event scheduler is switched off, use SET GLOBAL event_scheduler=ON to enable it. (SQL State: HY000 - Error Code: 1105)
[INFO] Migrating schema `stevedb` to version "0.6.7 - update"
[WARNING] DB: Event scheduler is switched off, use SET GLOBAL event_scheduler=ON to enable it. (SQL State: HY000 - Error Code: 1105)
[INFO] Migrating schema `stevedb` to version "0.6.8 - update"
[INFO] Migrating schema `stevedb` to version "0.6.9 - update"
[INFO] Migrating schema `stevedb` to version "0.7.0 - update"
[INFO] Migrating schema `stevedb` to version "0.7.1 - update"
[INFO] Migrating schema `stevedb` to version "0.7.2 - update"
[INFO] Migrating schema `stevedb` to version "0.7.3 - update"
[INFO] Migrating schema `stevedb` to version "0.7.6 - update"
[INFO] Migrating schema `stevedb` to version "0.7.7 - update"
[INFO] Migrating schema `stevedb` to version "0.7.8 - update"
[INFO] Migrating schema `stevedb` to version "0.7.9 - update"
[INFO] Migrating schema `stevedb` to version "0.8.0 - update"
[INFO] Migrating schema `stevedb` to version "0.8.1 - update"
[INFO] Migrating schema `stevedb` to version "0.8.2 - update"
[INFO] Migrating schema `stevedb` to version "0.8.4 - update"
[INFO] Migrating schema `stevedb` to version "0.8.5 - update"
[INFO] Migrating schema `stevedb` to version "0.8.6 - update"
[ERROR] Migration of schema `stevedb` to version "0.8.6 - update" failed! Please restore backups and roll back database and code!
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  5.000 s
[INFO] Finished at: 2024-02-26T15:30:44+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.flywaydb:flyway-maven-plugin:10.8.1:migrate (default) on project steve: org.flywaydb.core.internal.exception.FlywaySqlException: Error while retrieving the list of applied migrations from Schema History table `stevedb`.`schema_version`
[ERROR] ----------------------------------------------------------------------------------------------------------
[ERROR] SQL State  : HY000
[ERROR] Error Code : 1100
[ERROR] Message    : Table 'schema_version' was not locked with LOCK TABLES

@mahmudarslan
Copy link
Author

mahmudarslan commented Feb 26, 2024 •
edited
Loading

[ERROR] Migration of schema stevedb to version "0.8.6 - update" failed! Please restore backups and roll back database and code!

to solve this problem, modify the docker-compose.yml as follows ( image: mariadb:10.4.30)

version: "3.0"

volumes:
db-data:
external: false

services:

db:
# Pinning MariaDB to point release 10.4.30 works around the issues with the
# database migrations seen with 10.4.31 in issue #1212.
#
# TODO: Get database migrations to work with the latest point releases of
# MariaDB 10.4.
image: mariadb:10.4.30
ports:
- 3306:3306
environment:
MYSQL_RANDOM_ROOT_PASSWORD: "yes"
MYSQL_DATABASE: stevedb
MYSQL_USER: steve
MYSQL_PASSWORD: changeme

app:
build: .
links:
- "db:mariadb"
volumes:
- .:/code
ports:
- "8180:8180"
- "8443:8443"
depends_on:
- db

I have the same error as you, on a vanilla new installation. I aligned on your branch to get your patch but I still have an error.

  • Debian-12
  • jdk-17.0.10
  • mariadb-10.11

Error: ERROR] Migration of schema stevedb to version "0.8.6 - update" failed! Please restore backups and roll back database

Detail logs

My source

commit c26e0b336f62b5e946e81afe4fb5e4bb2ca79f49 (HEAD -> mahmudarslan/master, mahmudarslan/master)
Author: mahmudarslan <[email protected]>
Date:   Fri Feb 23 17:28:08 2024 +0300
    migration bug fixed V0_7_7__update.sql

Command log

./mvnw package
[INFO] Scanning for projects...
[INFO] 
[INFO] -------------------------< de.rwth.idsg:steve >-------------------------
[INFO] Building SteVe 3.6.0
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- enforcer:3.4.1:enforce (enforce-java) @ steve ---
[INFO] Rule 0: org.apache.maven.enforcer.rules.version.RequireJavaVersion passed
[INFO] Rule 1: org.apache.maven.enforcer.rules.version.RequireMavenVersion passed
[INFO] 
[INFO] --- git-commit-id:7.0.0:revision (default) @ steve ---
[INFO] 
[INFO] --- properties:1.2.1:read-project-properties (default) @ steve ---
[INFO] Loading 24 properties from File: /root/ocpp-steve/steve/src/main/resources/config/prod/main.properties
[INFO] 
[INFO] --- flyway:10.8.1:migrate (default) @ steve ---
[INFO] Database: jdbc:mysql://localhost:3306/stevedb (MySQL 5.5)
[INFO] Schema history table `stevedb`.`schema_version` does not exist yet
[INFO] Successfully validated 36 migrations (execution time 00:00.012s)
[INFO] Creating Schema History table `stevedb`.`schema_version` ...
[INFO] Current version of schema `stevedb`: << Empty Schema >>
[WARNING] outOfOrder mode is active. Migration of schema `stevedb` may not be reproducible.
[INFO] Migrating schema `stevedb` to version "0.6.6 - inital"
[WARNING] DB: Event scheduler is switched off, use SET GLOBAL event_scheduler=ON to enable it. (SQL State: HY000 - Error Code: 1105)
[INFO] Migrating schema `stevedb` to version "0.6.7 - update"
[WARNING] DB: Event scheduler is switched off, use SET GLOBAL event_scheduler=ON to enable it. (SQL State: HY000 - Error Code: 1105)
[INFO] Migrating schema `stevedb` to version "0.6.8 - update"
[INFO] Migrating schema `stevedb` to version "0.6.9 - update"
[INFO] Migrating schema `stevedb` to version "0.7.0 - update"
[INFO] Migrating schema `stevedb` to version "0.7.1 - update"
[INFO] Migrating schema `stevedb` to version "0.7.2 - update"
[INFO] Migrating schema `stevedb` to version "0.7.3 - update"
[INFO] Migrating schema `stevedb` to version "0.7.6 - update"
[INFO] Migrating schema `stevedb` to version "0.7.7 - update"
[INFO] Migrating schema `stevedb` to version "0.7.8 - update"
[INFO] Migrating schema `stevedb` to version "0.7.9 - update"
[INFO] Migrating schema `stevedb` to version "0.8.0 - update"
[INFO] Migrating schema `stevedb` to version "0.8.1 - update"
[INFO] Migrating schema `stevedb` to version "0.8.2 - update"
[INFO] Migrating schema `stevedb` to version "0.8.4 - update"
[INFO] Migrating schema `stevedb` to version "0.8.5 - update"
[INFO] Migrating schema `stevedb` to version "0.8.6 - update"
[ERROR] Migration of schema `stevedb` to version "0.8.6 - update" failed! Please restore backups and roll back database and code!
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  5.000 s
[INFO] Finished at: 2024-02-26T15:30:44+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.flywaydb:flyway-maven-plugin:10.8.1:migrate (default) on project steve: org.flywaydb.core.internal.exception.FlywaySqlException: Error while retrieving the list of applied migrations from Schema History table `stevedb`.`schema_version`
[ERROR] ----------------------------------------------------------------------------------------------------------
[ERROR] SQL State  : HY000
[ERROR] Error Code : 1100
[ERROR] Message    : Table 'schema_version' was not locked with LOCK TABLES

@fulup-bzh
Copy link

I'm not sure to understand you. I'm not using Docker. I'm on a plain Debian machine on which one I only have Mariadb-10.5 or 10.11.

Look like I'm suck until this bug get properly fixed.
Thank you for your quick response.

@mahmudarslan
Copy link
Author

why do you think there is a problem? what specific problem does the commit fix?

Fixes bug in version 3.6.0 when we want to run the application with docker file

@fnkbsi
Copy link
Contributor

fnkbsi commented Mar 30, 2024

why do you think there is a problem? what specific problem does the commit fix?

Fixes bug in version 3.6.0 when we want to run the application with docker file

Newer version of mariadb are much more strict about changing data-typs or character-sets if the data is uses in a foreign key. That's why V0_7_7_update.sql and V0.8.6_update.sql are throwing errors on newer maraidb version (I assume since 10.5).

@mahmudarslan: Would you please also change V0_8_6_update.sql. Drop the foreign keys which throw errors and restore them at the end of the script. Than this pk should also work vor mariadb v10.5 and newer

Know problem with this recommended solution:
Changing the migration scripts makes an update of Steve much more tricky, because the checksum in the schema table has to be adapted before compiling the new version.

@fnkbsi fnkbsi mentioned this pull request Mar 30, 2024
Open
@juherr juherr mentioned this pull request Mar 31, 2024
Closed
@Jenf01
Copy link

Jenf01 commented Mar 31, 2024

Sorry for the maybe stupid question but I am completely new to the Linux and GitHub world.
Do I understand you correctly that if I also have the problem migrating I have to wait until an update comes or can I do something myself to get the Steve project migrated?
As already mentioned, I have absolutely no experience with Linux or GitHub.
I am unfortunately only dependent on Steve otherwise my wallbox will not work as it should.
Thank you in advance for your answer.

@juherr
Copy link
Contributor

juherr commented Apr 2, 2024

@Jenf01 Your approach is good, and trying to fix is appreciated. Sadly, the fix you proposed will break servers in production and cannot be merged.

As said by @fnkbsi something should be done on checksums.

Maybe repair or baseline flyway commands are possible but I'm not sure they are part of the Flyway OSS version.
This blog post looks to be a good introduction: https://medium.com/att-israel/flyway-squashing-migrations-2993d75dae96

If a modification is planned in migration files, then maybe it is a perfect time to change the baseline and drop the need for extra rights like I tried in #1140

@rpseng
Copy link

rpseng commented Apr 17, 2024

Hello, just to let you know that the build also fails right now with Ubuntu 22.04.4 LTS since it comes with mysql Ver 15.1 Distrib 10.6.16-MariaDB

@fnkbsi fnkbsi mentioned this pull request Apr 20, 2024
Merged
@goekay
Copy link
Member

goekay commented Jun 27, 2024

what is the state of this PR after merging #1439 ?

@goekay
Copy link
Member

goekay commented Jul 27, 2024

i am closing this issue due to inactivity. feel free to reopen if it is still relevant.

@goekay goekay closed this Jul 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@mahmudarslan @goekay @fulup-bzh @fnkbsi @Jenf01 @juherr @rpseng