perf: share a scan's pruning predicates between its files' FilePruners

[asg0451]

🤖 Written by Claude (an AI assistant) and posted from my account.

Which issue does this PR close?

• Part of Cache parquet pruning setup across files with the same physical schema #21554.

Rationale for this change

A Parquet scan with a predicate builds a PruningPredicate for each file's FilePruner when it opens the file, and again whenever a dynamic filter in the predicate changes. A build snapshots the dynamic filters, simplifies the predicate and rewrites it in terms of statistics, which costs much more than evaluating the result against one file's statistics. Files with the same partition values build the same pruning predicate, so a scan of many small files does that work once a file.

It shows most under a TopK over many files that carry statistics: the first files fill the TopK, and its dynamic filter then prunes most of the rest at open, each after building the pruning predicate from scratch. In a service of ours whose query reads ~350 PartitionedFiles (row ranges of Parquet files, each with min/max statistics for the sort column, in one partition) under a newest-first TopK, FilePruner::should_prune was 45% of a query's CPU: 31% building the pruning predicate, 13% evaluating it.

What changes are included in this PR?

• PruningPredicateCache in datafusion-pruning: one slot holding the last pruning predicate built, with the predicate it was built from (after partition values are replaced with literals), the file schema, and the predicate's snapshot_generation. Clones share the slot.
• FilePruner::with_cache, additive: should_prune takes the pruning predicate from the cache, and builds it only when the predicate, the schema or the generation differs.
• ParquetMorselizer keeps one cache per partition and gives it to each file's pruner.

For review:

• The generation is read before the build, so a cached pruning predicate was built from that generation of the dynamic filters or a later one. A later dynamic filter is as correct to prune with; it's what a file opened a moment later would get.
• One slot, not a map: files with the same partition values are usually next to each other in a partition's file group, and one slot bounds the memory at one pruning predicate a partition. Files whose partition values alternate build each time, as now.
• The slot's lock is held while building, so two files of a partition don't build the same one at once.
• predicate_creation_errors now counts a failed build once, not once for each file that would have built it again.
• Cache reusable Parquet pruning setup across files with the same schema #21566 caches the row-group pruning setup for static predicates, and skips dynamic filters and literal replacement. This covers the FilePruner, dynamic filters included, so the two don't overlap.

What is the testing strategy for this PR?

• file_pruner::tests::files_with_the_same_predicate_share_its_pruning_predicate: under a DynamicFilterPhysicalExpr, a second file reuses the first's pruning predicate; after update, the next check builds a new one, which the files after it reuse; each file is pruned or kept as before.
• file_pruner::tests::files_with_different_predicates_build_their_own.
• opener::test::test_prune_on_partition_values_of_each_file_of_a_scan: one morselizer opens files with different partition values, interleaved, under a dynamic filter, and each is pruned or kept by its own partition value.

Benchmark: parquet_pruning_setup_cache/same_schema_files (128 files, static predicate), main (5f21582) against this branch, four runs each taking turns on a laptop, LTO off: median 11.1 ms → 10.1 ms (−9%), with about ±10% between runs.

Not known:

• A local TopK variant of that bench (SELECT id FROM t ORDER BY id LIMIT 10 over the same files) was within noise. I haven't checked whether its files reach the FilePruner, or are skipped before it.

Are there any user-facing changes?

PruningPredicateCache and FilePruner::with_cache are new public API in datafusion-pruning. Nothing breaks.

[asg0451]

apologies -- the bot posted this without my permission :/ i do not vouch for this code, feel free to ignore