Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delta storage layer migration 0.3.x #324

Closed
gruuya opened this issue Mar 23, 2023 · 0 comments
Closed

Delta storage layer migration 0.3.x #324

gruuya opened this issue Mar 23, 2023 · 0 comments

Comments

@gruuya
Copy link
Contributor

gruuya commented Mar 23, 2023

As of version 0.3.x Seafowl doesn't use a custom MVCC implementation anymore, but instead relies on delta-rs to accommodate Delta tables for it's new underlying storage layer.

Consequently, there are few changes/guidelines that are worth pointing out:

  • Writes are no longer possible to the preexisting tables which were created prior to Seafowl v0.3.x (aka legacy tables).
  • The only exception are VACUUM and DROP commands, in order to facilitate easier cleanup of legacy tables.
  • Reading (including time-travel) is still supported for legacy tables.
  • All newly created tables will be Delta tables, and all functionalities are supported for them as they were for legacy tables (including the full range of DDL & DML statements as before; see the updated docs).
  • The idea is to enable migration of data from legacy tables to new tables throughout all 0.3.x versions—for instance via CREATE TABLE AS statement where the subquery targets a legacy table.
  • At some point the legacy tables will be completely phased out, and we'll bump the minor version again (0.4.x versions).

Finally, If you find any problems related to migration or working with new Delta tables please file a new issue or report it in this pinned issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant