[VL][Delta] Add JVM-side Delta DV scan handoff by malinjawi · Pull Request #3 · malinjawi/incubator-gluten

malinjawi · 2026-05-13T10:09:35Z

What changes are proposed in this pull request?

This draft PR is the third step in the split Delta deletion-vector stack.

It depends on apache#12040 and contains the JVM-side Delta scan and deletion-vector handoff pieces split out from apache#11963.

Main changes:

add Delta-version-specific metadata utilities for extracting materialized deletion-vector payloads
preprocess Delta scans with DVs so native Delta scans receive metadata columns and serialized DV payload handles
register task-local DV payloads through DeltaDeletionVectorRegistry and retain direct ByteBuffers for the native kernel lifetime
extend the Java/JNI/native split path so per-split payload buffers are passed beside LocalFiles split info
map Delta scan split metadata to the native Delta connector and HiveDeltaSplit path introduced by PR2
add Delta deletion-vector handoff suites for Delta 3.3 and Delta 4.0 layouts

This PR intentionally builds on PR2 rather than reintroducing the roaring bitmap or native Delta reader infrastructure.

How was this patch tested?

./build/mvn -q spotless:apply -pl gluten-arrow,gluten-substrait,gluten-delta,backends-velox -Pbackends-velox -Pdelta -Pspark-3.5
./build/mvn -q spotless:check -pl gluten-arrow,gluten-substrait,gluten-delta,backends-velox -Pbackends-velox -Pdelta -Pspark-3.5
./build/mvn -q -pl gluten-arrow,gluten-substrait,gluten-delta,backends-velox -am -Pbackends-velox -Pdelta -Pspark-3.5 -DskipTests -DskipUTs -DskipITs test-compile
PATH=/opt/homebrew/opt/llvm@15/bin:$PATH /tmp/gluten-format-venv/bin/python dev/check.py format tree <touched C++ files>
git diff --check && git diff --cached --check

Was this patch authored or co-authored using generative AI tooling?

Generated-by: IBM BOB

github-actions · 2026-05-13T10:10:08Z