Skip to content

Conversation

abondar
Copy link
Member

@abondar abondar commented Apr 5, 2025

Context:

Right now aerich creates migrations based on diff between latest content in aerich table in db.

This creates several problems, for example:
#309
#270

Because of such a way of handling it - it creates problems with setting up existing project on new env/new db, as new user does not have any knowledge about models state at given moment of time, and in best case scenario - you can run upgrade and it will save current state of models to all aerich table rows, which is wrong by itself
Also it creates problems with sequentially creating several migrations without upgrading database in between.

Overall - this methodic of storing model state is problematic and inconvenient

This PR aims to tackle those issues

Description:

New approach saves current models state as encoded string within file itself, enabling offline migration management, without comparing with version in database. That solves problems for setting up application in new env and improves experience for people who develop their apps without available connection to db, which can be common for bigger projects

It also provides method fix-migrations to update migrations to new format, although that requires existing db with migrations applied to work properly

Known issues

Right now - this approach is not fully "offline" as it still need to connection to db for two things:

  • Init tortoise application, to populate current models state
  • We query mysql server for it's version, to determine which column rename syntax we should use

I believe both of this issues are solvable

  • For tortoise init - we can make special method "_init_offline" or something like that, which will init all models, but not create real connection to db
  • We could either drop special support for mysql5, or we could configure dialect for migrations in tortoise config

I'll try to work on both of this solutions in upcoming days

Alternatives

I think storing information about models state through files is necessary one way or another. Better approach would be storing only diff between models, but if we go down this route - it's even better to calculate models state on the go, by parsing migration changes themself.
But such approach leads us into rabbit hole of making full fledged django like migration system, which is neat, but far bigger than all existing aerich code, so I don't think it's feasible for me in the moment

@abondar abondar requested review from waketzheng and henadzit April 5, 2025 16:25
@waketzheng
Copy link
Contributor

I tried to find out the reason why ci failed, and found that the change of aerich.migrate.Migrate.get_last_version will let the aerich heads get wrong result when version in aerich table is not the same as the latest file in migrations/ (e.g.: downgrade version in aerich table by aerich downgrade -v xxx --fake)

if version == -1:
versions = [specified_version]
else:
versions = await Aerich.filter(app=self.app, pk__gte=specified_version.pk)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also changed this line from gte to gt, as it seemed strange to me that "downgrading to certain version" also ran downgrade for chosen version, so you actually got base at "chosen migration - 1"

Although if you have counterpoints to why it should stay the way it were, I can rollback it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that the downgrade command seems strange.
Could you roll back the unnecessary changes? Let’s keep things simple, zero in on the new feature. We can move the refactoring work to another pull request (PR).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants