Ducklake truncate operator

[jprafael]

Current implementation of DELETE FROM table or TRUNCATE table scans the entire table to be deleted and captures row_ids in memory which for large tables is expensive.

This is an optimization for the case where the full table needs to be deleted, in which case we can do it by manipulating metadata alone.

This was LLM generated.

[Mytherin]

Thanks! Looks good - good idea. Can we add some extra tests?

• Verify ROLLBACK works correctly
• Verify transactionality works correctly (i.e. other connections don't yet see the changes)
• Add tests verifying this works correctly with inlined data
• Look at the return value of TRUNCATE to verify it's returning the correct number of rows, also when a table already has deletes within a file
• Verify time travel works

[jprafael]

Thanks! Looks good - good idea. Can we add some extra tests?

• Verify ROLLBACK works correctly
• Verify transactionality works correctly (i.e. other connections don't yet see the changes)
• Add tests verifying this works correctly with inlined data
• Look at the return value of TRUNCATE to verify it's returning the correct number of rows, also when a table already has deletes within a file
• Verify time travel works

Thanks for the test pushback! Indeed there were some issues with the code that warranted a rewrite. The new version adds tests for the cases you mentioned and is more organized in the way that DuckLakeTruncate iterates through the data sources (files, inline, local inline). There is also