How to handling Parallel GitHub Actions and Migrations in Django + React Projects

[vucreations]

I'm working on a Django + React project, and we use GitHub Actions for CI/CD. The typical workflow when a pull request is raised involves running Selenium tests, which require a large dataset in the database to execute effectively. Once the tests pass, I merge the branch, triggering an AWS CodePipeline that goes through source, manual approval, and deployment stages.

The problem I'm facing is related to database migrations when multiple developers raise pull requests simultaneously. If I run the workflows in parallel, I end up with migration conflicts that cause issues down the road. However, if I run the workflows sequentially, the deployment process gets delayed, especially when there are several pull requests in the queue.

Current Setup:

1. GitHub Actions: Runs on pull requests, runs Selenium tests, and applies migrations to a testing database.
2. AWS CodePipeline: After tests pass and PR is merged, it handles manual approval and deployment to production.
3. Database: I'm using a large dataset, and migrations are applied to both a testing database and the live database in different stages.

Concerns:

• Migration Conflicts: When workflows run in parallel and developers raise pull requests with migration changes, I face conflicts when these workflows try to run migrations on the test or live databases simultaneously.
• Sequential Workflow: Running workflows one after another avoids conflicts, but significantly delays deployment when there are multiple pull requests waiting to be processed.

My Questions:

1. What’s the best practice for managing migrations when multiple pull requests are raised with migration changes?
2. Is there a way to efficiently run parallel workflows without causing migration issues but still avoiding delays in deployment?
3. How can I ensure database migrations run smoothly and prevent conflicts when multiple developers contribute changes that require schema updates?

Any tips or suggestions on how to handle this situation would be really helpful!

[igorpereirag]

Managing database migrations in a project where multiple developers raise pull requests simultaneously can be challenging, especially when it impacts continuous integration and deployment (CI/CD). Here are some best practices and suggestions to help mitigate migration conflicts while balancing parallel and sequential workflows.

1. Use Feature Branches and Rebase Migrations:

• Feature Branches: Each pull request should be based on a feature branch. This helps isolate changes and makes it easier to resolve migration conflicts.
• Rebase Before Merge: Before merging a pull request, make sure to rebase it against the main branch to incorporate any recent migrations. This ensures that migrations from other pull requests are applied before the branch is merged, reducing conflicts.

2. Use Squash Migrations:

• Squashing Migrations: If your project allows, you can squash migrations before merging. Squashing reduces the number of migration files, which decreases the chance of conflicts. You can configure Django’s squashmigrations to combine multiple migration files into one.
• Squashing should be done in coordination with your team to avoid breaking schema consistency.

3. Parallel Testing with Isolated Databases:

• Instead of using a shared testing database, consider using isolated databases for each workflow. Each pull request can spin up its own Docker container or a dedicated test database for migrations, ensuring parallel execution without conflict.
• Tools like Docker Compose or services like RDS or Amazon Aurora can be leveraged to create test databases on demand.

4. Staggered Deployment of Migrations:

• In a production environment, you can implement staggered deployments. Run migrations in a pre-deployment phase (using AWS CodePipeline) and ensure that your app can handle the updated schema before the final deployment.
• This ensures that schema migrations don’t block deployment but can still be reviewed and applied manually if needed.

5. Use a Dedicated Migration Approval Process:

• If migration conflicts are frequent, you can create a manual migration review stage before merging PRs. This stage will check for migration conflicts by running migrations from each PR in a controlled environment.
• This can be automated as part of the CI pipeline or done as a manual approval step.

6. Apply Optimistic Locking for Sequential Migrations:

• Use optimistic locking or a lock mechanism for database migrations. When one pull request is merged, it locks the migration process until it finishes, reducing the chance of conflicts. This will allow workflows to run in parallel but manage migrations sequentially.

7. Decouple Migration Process from Tests:

• Consider decoupling the migration process from running tests. Instead of running migrations directly during tests, run tests on a pre-migrated database (like a snapshot) and run migrations as a separate task in the pipeline. This will help avoid conflicts while still ensuring tests run smoothly.

8. Version Control for Migrations:

• Utilize a migration naming convention or version control for your migrations. Ensure migrations are named uniquely and clearly, to avoid naming conflicts, and use version tags or commit hashes to track applied migrations.

9. Continuous Communication and Review:

• Regularly communicate migration changes during pull requests. You can introduce a migration review checklist for developers to ensure that migration conflicts are identified early and resolved in collaboration.

Workflow Example:

1. GitHub Actions:

   • On a pull request, spin up an isolated test environment (with a separate database).
   • Run migrations and Selenium tests in parallel without impacting other branches.

2. CodePipeline:

   • After merging, apply migrations in a staggered pre-deployment phase.
   • Use manual approval to review migrations before pushing to production.

By adopting these strategies, you can mitigate migration conflicts while maintaining the efficiency of your CI/CD pipeline.

[itisrude]

Hey VuCreations,

In parallel workflows, these migrations can get hairy-especially when there are multiple PRs. A couple tips that may help with these include:

1. Best Practice to Manage Migrations with Multiple PRs

The typical solution is to introduce a "lock". This means you would lock migrations so that only one can apply at a time. You might do this by creating a database table tracking migration status, and every time running a migration check whether another migration is currently running. If it is, queue them all to run after the current finishes.

2. Running Parallel Workflows Efficiently

You could split workflows into two steps:

• Run the tests in parallel for every pull request without running the migrations. That way, several workflows would run simultaneously.

• Do the migrations and deployment when the tests pass. The migration stage can be queued on PRs containing changes to the database so that they come one after another, avoiding conflicts but keeping your test pipeline parallel.

3. Ensuring Smooth Migrations Without Conflicts

One approach is to have all the migrations applied in a staging environment before pushing into production. You can also add feature flags that decouple schema changes from code changes. You would apply schema migrations first, but only merge code depending on those schema changes when the migrations are proved to work.

Further Consideration

You could also think about adding pre-migration checks that detect and prevent conflicts before they happen by running a "dry run" of the migrations against another database that's a copy of your production or staging environment.

If any of these tips help, please let me know! Feel free to give me more detailed information about your setup if you'd like specific advice.

[github-actions[bot]]

🕒 Discussion Activity Reminder 🕒

This Discussion has been labeled as dormant by an automated system for having no activity in the last 60 days. Please consider one the following actions:

1️⃣ Close as Out of Date: If the topic is no longer relevant, close the Discussion as out of date at the bottom of the page.

2️⃣ Provide More Information: Share additional details or context — or let the community know if you've found a solution on your own.

3️⃣ Mark a Reply as Answer: If your question has been answered by a reply, mark the most helpful reply as the solution.

Note: This dormant notification will only apply to Discussions with the Question label. To learn more, see our recent announcement.

Thank you for helping bring this Discussion to a resolution! 💬