Skip to content

Rebase truncate changes on top of PR #969#1

Open
jprafael wants to merge 9 commits into
pr-969-base-for-825from
pr-825-rebased-on-969
Open

Rebase truncate changes on top of PR #969#1
jprafael wants to merge 9 commits into
pr-969-base-for-825from
pr-825-rebased-on-969

Conversation

@jprafael
Copy link
Copy Markdown
Owner

Summary

This is a fork-only comparison PR.

Because the base branch already contains duckdb#969, this PR shows only the remaining differences from duckdb#825 after rebasing on top of duckdb#969.

Included commits

  • Add DuckLake truncate operator
  • Assert DuckLake truncate plan in tests
  • Assert delete count invariant in truncate
  • Add more unit tests and cleanup implementation
  • Cleanups
  • Fix truncate local inlined data after rebase
  • Fix truncate row count on repeated deletes
  • Fix delete history and diagnostics after rebase

Notes

This PR is intended for comparison/review in jprafael/ducklake only and does not target duckdb/ducklake directly.

Comment on lines +1798 to +4413
@@ -1900,8 +1900,7 @@ ORDER BY data.begin_snapshot, data.row_id_start, data.data_file_id, del.begin_sn
delete_file.data = ReadDeleteFile(table, row, col_idx, IsEncrypted());
file_entry.delete_files.push_back(std::move(delete_file));
}

// Load inlined deletions for active files so rewrite compaction can treat them the same as delete files
// Load inlined deletions for active files so rewrite compaction can treat them the same as delete files.
auto inlined_deletions = ReadInlinedFileDeletions(table_id, snapshot);
for (auto &file : files) {
auto entry = inlined_deletions.find(file.file.id.index);
@@ -4410,8 +4409,8 @@ WHERE end_snapshot IS NOT NULL AND NOT EXISTS(

void DuckLakeMetadataManager::DeleteInlinedData(const DuckLakeInlinedTableInfo &inlined_table) {
auto result = transaction.Query(StringUtil::Format(R"(
DELETE FROM {METADATA_CATALOG}.%s
)",
DELETE FROM {METADATA_CATALOG}.%s
)",
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove whitespace/comment only changes.

Comment thread src/storage/ducklake_truncate.cpp Outdated
}
case DuckLakeDataType::INLINED_DATA: {
auto row_ids = GetVisibleInlinedRowIds(transaction, read_info.snapshot, file_info.file.path);
transaction.AddNewInlinedDeletes(table.GetTableId(), file_info.file.path, std::move(row_ids));
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The whole purpose of this PR is so that TRUNCATE doesn't have to iterate through rows.
Why can't this just set the entire inline "file" as deleted?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant