Skip to content

Add bucket-transform-aware partition pruning#1176

Merged
pdet merged 3 commits into
duckdb:v1.5-variegatafrom
guillesd:bucket-pruning-fix
May 21, 2026
Merged

Add bucket-transform-aware partition pruning#1176
pdet merged 3 commits into
duckdb:v1.5-variegatafrom
guillesd:bucket-pruning-fix

Conversation

@guillesd

@guillesd guillesd commented May 20, 2026

Copy link
Copy Markdown
Collaborator

Equality and IN-list predicates on bucket()-partitioned columns now prune files via ducklake_file_partition_value at scan-planning time. Bucket hashes are computed in C++ so the catalog SQL stays backend-agnostic. Range predicates fall through to the existing zone-map path.

Basically makes lookup queries useful with buckets. Zone-maps are not really effective for hashed transforms.

@guillesd

Copy link
Copy Markdown
Collaborator Author

We could alternatively make BuildBucketPartitionPruningClause part of GenerateFilterPushdownComponents and centralize everything there. I'd just need to change the signature of the function to get DuckLakeTableEntry to have the partition info. Let me know what you think @pdet

@pdet

pdet commented May 21, 2026

Copy link
Copy Markdown
Member

Thanks, LGTM, can you just fix the failing CI?

@guillesd

Copy link
Copy Markdown
Collaborator Author

@pdet done! Basically all partitioning tests are skipped in Quack in the first place because of a known limitation. So I just added the new test to the skip list only for Quack!

@pdet

pdet commented May 21, 2026

Copy link
Copy Markdown
Member

Thanks!

@pdet pdet merged commit 2ab3b75 into duckdb:v1.5-variegata May 21, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants