Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add handling of sequences which are not associated with a table column #97

Open
bonesmoses opened this issue Jan 30, 2024 · 2 comments
Open
Labels
enhancement New feature or request

Comments

@bonesmoses
Copy link

I ran across this project in the ever-expanding ecosystem of Postgres tools. Given it's a replication / sync tool, I took a look at the sequence handling code, as this is an area that's often covered improperly.

I noticed in your refresh_sequences method that you're only "refreshing" sequences which are directly associated with a table column, and doing so by implicitly using the max() function on the underlying table.column.

This does not cover cases where bare sequences are referenced by application code and are thus not associated with a table at all. A safer approach would be to obtain the current values of the upstream publishing system and adjust all sequences based on those values. You can do this by leveraging the last_value column in the sequence itself.

@shayonj shayonj added the bug Something isn't working label Jan 30, 2024
@shayonj
Copy link
Owner

shayonj commented Jan 30, 2024

Thank you so much for taking the time and opening this report. I think because writes are stopped on the source (upstream publisher) database, its safe to read the sequence value from there.

For all the use cases so far, its been sequences that are directly tied to the tables. I will put a PR (sometime in the next week) to refresh sequences for all sequences regardless of the association with the table. Great catch.

@shayonj
Copy link
Owner

shayonj commented Feb 4, 2024

Finally, getting around to this. I think i misunderstood the original request at first. Its actually intentional that we refresh sequences by table, because pg_easy_replicate today doesn't support replicating sequences (same as logical replication), hence why we are not refreshing or porting over sequences from the source DB (reference to limits in the readme).

That said, it could be a nice feature to support this. That is, recreate sequences when sync begins and then post switchover refresh them, like you mentioned.

I think we can accept a --replicate-all-sequences (default true) or similar option in CLI to begin this stream of work. Will try to give it a spin.

@shayonj shayonj added enhancement New feature or request and removed bug Something isn't working labels Feb 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants