WIP: Store migrates in db #1318

daniel-garcia · 2025-09-12T02:07:45Z

No description provided.

- Add support for viper

Enable configuration

- Add support for dsn and individual parameters

Configuration support for database string

Merge latest changes from opensource migrate

* Update alpine and golang versions * Refresh go modules

* 'master' of github.com:golang-migrate/migrate: (418 commits) Run gofmt on internal build dir go mod tidy Resolves golang-migrate#647 - Fixes typos in Mongo advisory locking parameters (golang-migrate#648) Bump version of autorest/adal Set syntax highlighting for pkger example Add pkger to README change github auth to use oauth token instead of basic. Use the recommended v4 in mysql README go mod tidy fix test Delete all rows Use ParseBool Support for AWS Keyspaces Update gosnowflake from v1.4.3 to v1.6.3 Update docker client usage with breaking change Update dktest to v0.3.7 to fix security warnings revert binary file location change in docker image fix source/file driver Update build constraints Update golangci-lint config ...

* upgrade alpine 3.13 -> 3.18 * add --pull to docker build

* adding pgx as the supported hotload base driver * updating go.mod file

* pgx migrate test (#29) * add hotload support (#15) * import fsync * Upgrade alpine 3.13 -> 3.18 (#21) * upgrade alpine 3.13 -> 3.18 * add --pull to docker build * adding pgx as the supported hotload base driver (#26) * adding pgx as the supported hotload base driver * updating go.mod file * removing the pgx register in migrate (#27) * register hotload with pgx driver --------- Co-authored-by: Daniel Garcia <[email protected]> Co-authored-by: Ying Chen <[email protected]> Co-authored-by: Supriya Gowda <[email protected]> * register pgx in main cli, not internal (#30) * register pgx in main cli, not internal * go fmt --------- Co-authored-by: Sujay Kumar Suman <[email protected]> Co-authored-by: Ying Chen <[email protected]> Co-authored-by: Supriya Gowda <[email protected]>

Vulns

* First pass at downmigrate * First pass at downmigrate * downmigrate changes + UTs + cleanups * address comments

…own feature (#37) * address comments * address comments + new UTs * address comments * remove build files

* Update go version and dependencies * Update go versions in ci

- Add new 'install-to DIR' subcommand that copies the running binary to a specified directory - Uses atomic operations: copy to temp file, then rename for atomicity - Preserves executable permissions (755) from original binary - Handles errors gracefully with proper exit codes and cleanup - Validates destination directory exists and is actually a directory - Integrated with existing help system and command structure Implementation details: - New file: internal/cli/commands_install_to.go - core functionality - New file: internal/cli/commands_install_to_test.go - comprehensive test suite - Modified: internal/cli/main.go - CLI integration and command parsing Testing: - 7 test functions covering unit and integration testing - Tests atomic operations, error handling, permission preservation - Tests CLI interface, help system, and edge cases - All tests passing This feature enables easy binary deployment and self-updating workflows while maintaining compatibility with upstream migrate project structure.

feat: add install-to subcommand for binary self-installation

Copilot

Pull Request Overview

This is a WIP pull request that adds the ability to store migration scripts directly in the database instead of relying on external file storage for dirty state handling. This enables better migration management in environments where shared file storage is not available.

Key changes include:

Introduction of database-based migration storage capabilities through a new MigrationStorageDriver interface
Enhanced dirty state handling with support for both file-based and database-based approaches
Addition of an "install-to" command for copying the migrate binary to specified directories

Reviewed Changes

Copilot reviewed 22 out of 23 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
migrate.go	Core migration logic enhanced with database storage sync, dirty state handling, and file cleanup operations
migrate_test.go	Comprehensive test coverage for new dirty state handling, file operations, and configuration features
database/storage.go	New interface defining database-based migration storage capabilities
database/postgres/storage.go	PostgreSQL implementation of migration storage with enhanced schema table support
database_source.go	Source driver implementation for reading migrations from database storage
internal/cli/main.go	CLI enhancements with viper configuration, database driver support, and new install-to command
cmd/migrate/main.go	Updated main entry point with hotload driver registration and configuration setup

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-09-12T02:09:15Z

internal/cli/main.go

+	if version == "" {
 		fmt.Fprintln(os.Stderr, version)
 		os.Exit(0)
 	}


The logic is inverted - if version is empty, it prints the empty version and exits. This should check if a version flag was set to true or if version is non-empty to print and exit.

Copilot · 2025-09-12T02:09:15Z

migrate.go

+		}
+		scheme := "file"
+		if uri.Scheme != "file" && uri.Scheme != "" {
+			return "", "", fmt.Errorf("unsupported scheme: %s", scheme)


The error message uses variable scheme but should use uri.Scheme since scheme is declared as a constant string 'file' on line 227.

Suggested change

return "", "", fmt.Errorf("unsupported scheme: %s", scheme)

return "", "", fmt.Errorf("unsupported scheme: %s", uri.Scheme)

Copilot · 2025-09-12T02:09:16Z

migrate_test.go

+	m, _ := setupMigrateInstance(tempDir)
+
+	tests := []struct {
+		lastSuccessFulVersion int


The field name contains a typo - 'lastSuccessFulVersion' should be 'lastSuccessfulVersion' to match the spelling used elsewhere in the codebase.

Copilot · 2025-09-12T02:09:16Z

migrate_test.go

+			if !tt.wantErr && m.dirtyStateConf == tt.wantConf {
+				t.Errorf("dirtyStateConf = %v, want %v", m.dirtyStateConf, tt.wantConf)
+			}


The comparison logic is incorrect. It should use != instead of == to properly compare if the actual configuration doesn't match the expected configuration when no error is expected.

Copilot · 2025-09-12T02:09:16Z

migrate.go

+						// this condition is required if the first migration fails
+						if lastCleanMigrationApplied == 0 {
+							lastCleanMigrationApplied = migr.TargetVersion
+						}


This logic is problematic - if the first migration fails, lastCleanMigrationApplied is set to the current failing migration's target version, but no migration has actually been successfully applied yet. This should use the previous version or handle the zero case differently.

Suggested change

// this condition is required if the first migration fails

if lastCleanMigrationApplied == 0 {

lastCleanMigrationApplied = migr.TargetVersion

}

// Do not update lastCleanMigrationApplied on failure; it should remain zero if no migration has succeeded

Copilot · 2025-09-12T02:09:17Z

database/postgres/storage.go

+			if err.Error() == "file does not exist" { // Handle os.ErrNotExist
+				break
+			}


Comparing error strings is fragile and unreliable. Use errors.Is(err, os.ErrNotExist) or check for the specific error type instead of string comparison.

coveralls · 2025-09-12T02:28:50Z

coverage: 50.017% (-4.0%) from 54.037%
when pulling 0f032ae on infobloxopen:storeMigratesInDB
into 8b9c5f7 on golang-migrate:master.

kumaya and others added 30 commits September 17, 2018 10:02

Modify logging to support logrus

f916ad6

Modify logging to support logrus

2513950

Merge branch 'master' of https://github.com/golang-migrate/migrate

f6ccfd4

Merge branch 'master' of https://github.com/golang-migrate/migrate

24a850c

Enable configuration

4499bdf

- Add support for viper

Merge pull request #3 from kumaya/config

2ed505f

Enable configuration

Configuration support for database string

adeea01

- Add support for dsn and individual parameters

Merge pull request #4 from kumaya/dsn

49df9a4

Configuration support for database string

Update README.md

1875527

Merge branch 'master' of https://github.com/golang-migrate/migrate

aa78373

Merge pull request #5 from kumaya/master

8a93a8f

Merge latest changes from opensource migrate

Update go and alpine versions (#6)

b200546

* Update alpine and golang versions * Refresh go modules

Add Jenkinsfile (#7)

5564ef4

Fix test issues

079cf15

Skip tests, just build

577e929

Docker credentials around build, too

8e2cb3a

Only push to harbor

2299517

URL escape password and username on connection strings (#10)

c2c9739

merge upstream

418501b

add hotload support

8445c30

add hotload support (#15)

f1ec28e

build in ib-branch

b23a55e

Merge branch 'hotload' into ib

2786533

import fsync

2d48d39

Upgrade alpine 3.13 -> 3.18 (#21)

f746be2

* upgrade alpine 3.13 -> 3.18 * add --pull to docker build

adding pgx as the supported hotload base driver (#26)

31fc03e

* adding pgx as the supported hotload base driver * updating go.mod file

removing the pgx register in migrate (#27)

f83486d

resolved upstream conflicts in go.*

d6166ab

daniel-garcia and others added 20 commits December 8, 2023 17:12

patch vulns

b82c168

merge upstream master

af3698e

Merge pull request #33 from infobloxopen/vulns

8447691

Vulns

fixed merged conflict

d47e437

upgrade go->1.22, alpine->latest (#34)

6caa1d9

Goto command support to handle dirty Database state (#36)

13318b2

* First pass at downmigrate * First pass at downmigrate * downmigrate changes + UTs + cleanups * address comments

Minor code updates from missed open comments from #36 to the migrated…

5d974f1

…own feature (#37) * address comments * address comments + new UTs * address comments * remove build files

Update go version and dependencies (#42)

9070a97

* Update go version and dependencies * Update go versions in ci

Build docker image using scratch as final base layer (#44)

0ccdb3b

Update jenkins go version to 1.24 (#45)

144e706

Try to unset go flags to make mod work maybe (#46)

3ed052a

Try using go mod download instead of vendor (#47)

7efd19e

gofmt and remove lint

028c114

limit tests and test on 1.24

7e04814

do not test sqlserver

6f0ba2b

Merge pull request #49 from infobloxopen/installTo

c84938c

feat: add install-to subcommand for binary self-installation

Merge branch 'master' into ib

7351dcb

disable linting for infobloxopen

67ef559

store db migrates in the database if it supports it

0f032ae

Copilot AI review requested due to automatic review settings September 12, 2025 02:07

daniel-garcia closed this Sep 12, 2025

Copilot AI reviewed Sep 12, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

WIP: Store migrates in db #1318

WIP: Store migrates in db #1318

Uh oh!

daniel-garcia commented Sep 12, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Sep 12, 2025

Uh oh!

Copilot AI Sep 12, 2025

Uh oh!

Copilot AI Sep 12, 2025

Uh oh!

Copilot AI Sep 12, 2025

Uh oh!

Copilot AI Sep 12, 2025

Uh oh!

Copilot AI Sep 12, 2025

Uh oh!

coveralls commented Sep 12, 2025

Uh oh!

Uh oh!

	return "", "", fmt.Errorf("unsupported scheme: %s", scheme)
	return "", "", fmt.Errorf("unsupported scheme: %s", uri.Scheme)

WIP: Store migrates in db #1318

WIP: Store migrates in db #1318

Uh oh!

Conversation

daniel-garcia commented Sep 12, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Sep 12, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 12, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 12, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 12, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 12, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 12, 2025

Choose a reason for hiding this comment

Uh oh!

coveralls commented Sep 12, 2025

Uh oh!

Uh oh!