Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pretty print auto-migration plans #2065

Draft
wants to merge 8 commits into
base: master
Choose a base branch
from
Draft

Conversation

kazimuth
Copy link
Contributor

@kazimuth kazimuth commented Dec 16, 2024

Description of Changes

Implements https://github.com/orgs/clockworklabs/projects/22/views/16?filterQuery=migration&pane=issue&itemId=91111218.

While implementing this, I discovered auto-migration plans applied their steps in a nondeterministic order (I was iterating a hashmap and had forgotten to sort.)

This interacted in a particularly bad way with row-level-security policies:

  • Row level security policies are removed and re-added in every automigration, to allow the core crate to re-initialize the relevant queries. (I disagree with this -- it should be an implementation detail of the core crate, not mentioned in auto-migrations -- but I'm not changing it in this PR; see my comments about it in the code.)
  • Since ordering was random, the Add step for the policy could be run before the Remove step.
  • Adding an already existing policy was implemented in an idempotent way by the core crate.
  • So, any auto-migration could nondeterministically drop row-level-security policies!

This is now fixed by simply sorting the steps of an auto-migration in a prescribed way.

API and ABI breaking changes

This is a minor breaking change to the client API since it adds data to the result of an update call.

Expected complexity level and risk

0

Testing

Added a smoketest.
TODO: I want to add to the integration tests of the schema crate as well.

@kazimuth kazimuth force-pushed the jgilles/pretty_print branch from d66684b to c0b04e1 Compare December 30, 2024 20:52
@kazimuth kazimuth force-pushed the jgilles/pretty_print branch from c3de71d to 64ff276 Compare January 9, 2025 19:17
@kazimuth kazimuth changed the title Pretty print auto-migration plans Pretty print auto-migration plans, and make auto-migrations deterministic Jan 9, 2025
@kazimuth kazimuth requested a review from Centril January 9, 2025 19:29
@@ -189,6 +190,9 @@ pub async fn exec(config: Config, args: &ArgMatches) -> Result<(), anyhow::Error
} else {
println!("{} database with identity: {}", op, database_identity);
}
if let Some(update_summary) = update_summary {
println!("{}", update_summary);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: I guess I should check if the terminal supports ANSI color codes here, since the summary may include those.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this TODO meant for this PR, or should it be a comment/ticket?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure that this is the best UX. It seems like these update summaries can be pretty large, and I don't think that most users are interested in reviewing them. Could we put this behind a CLI flag --show-migration-summary or something like that?

Copy link
Contributor Author

@kazimuth kazimuth Jan 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a reminder to myself. Also yeah I can put it behind a flag, sure.

TBH I'm not sure a flag is needed -- our tools are already pretty chatty and I don't think this adds an overwhelming amount of information. But probably a better long-term solution would be a history audit tool on the website; that would be a lot less ephemeral than console output. It's a larger project though.

I do think it's particularly important to print an explanation when automatic migrations fail -- but, come to think of it, this PR hasn't implemented that, so I suppose I should do it as well, either here or in another PR.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that reminder meant to be implemented in this PR? (i.e. should I hold off on reviewing?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this PR yeah, I'll mark as draft until it's ready again.

@kazimuth kazimuth marked this pull request as ready for review January 9, 2025 19:38
@gefjon
Copy link
Contributor

gefjon commented Jan 9, 2025

Please separate the bugfix from the new feature. These should be separate PRs.

@kazimuth
Copy link
Contributor Author

kazimuth commented Jan 9, 2025

Sure, #2104

@kazimuth kazimuth changed the title Pretty print auto-migration plans, and make auto-migrations deterministic Pretty print auto-migration plans Jan 9, 2025
@bfops bfops added the release-any To be landed in any release window label Jan 13, 2025
@bfops
Copy link
Collaborator

bfops commented Jan 13, 2025

Not my place but:

This is a minor breaking change to the client API since it adds data to the result of an update call.

Is that a breaking a change? It sounds purely additive, which I would've called not-api-breaking?

@bfops
Copy link
Collaborator

bfops commented Jan 13, 2025

@bfops bfops marked this pull request as draft January 13, 2025 18:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-any To be landed in any release window
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants