Skip to content

Add ducklake_drop_data_files procedure#1172

Open
jdctinuiti wants to merge 6 commits into
duckdb:mainfrom
jdctinuiti:drop-data-files-procedure
Open

Add ducklake_drop_data_files procedure#1172
jdctinuiti wants to merge 6 commits into
duckdb:mainfrom
jdctinuiti:drop-data-files-procedure

Conversation

@jdctinuiti

@jdctinuiti jdctinuiti commented May 18, 2026

Copy link
Copy Markdown

Stacked on #1171. Adds ducklake_drop_data_files, a procedure for retiring data files by typed partition values (e.g. partition_values => struct_pack(year := 2024, month := 1)). Supports dry_run, identity/transformed partition keys, transaction-local files, and concurrent conflicts. Please review only the top commit until #1171 is merged.

@jdctinuiti jdctinuiti marked this pull request as ready for review May 19, 2026 16:01
@jdctinuiti jdctinuiti force-pushed the drop-data-files-procedure branch 2 times, most recently from 11726ce to 6f6a984 Compare May 21, 2026 21:22
@jdctinuiti

jdctinuiti commented May 21, 2026

Copy link
Copy Markdown
Author

Added a follow-up commit that extracts the partition_values parsing/matching code into a shared DuckLakePartitionFilter helper. This keeps ducklake_drop_data_files behavior the same, but makes the partition filter spec-aware by checking file partition_id when available. Also opened #1182 stacked on this helper to reuse it for ducklake_merge_adjacent_files and ducklake_rewrite_data_files partition filtering.

@jdctinuiti jdctinuiti force-pushed the drop-data-files-procedure branch from 6bf1761 to bb8d0ef Compare June 4, 2026 00:38
@jdctinuiti jdctinuiti mentioned this pull request Jun 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant