[SkipRecovery] Re-enable mixed-delete tests on Iceberg 1.10.2+ [reduced-it] [databricks] by wjxiz1992 · Pull Request #15964 · NVIDIA/cudf-spark

wjxiz1992 · 2026-09-11T04:00:12Z

JaCoCo production line coverage: not fully measurable locally — no compatible Scala 2.13 / shim 411 nightly baseline is published.

Contributes to #12885.

Description

This test-only recovery restores enforced CPU/GPU result checks for mixed Iceberg position and equality deletes on Iceberg 1.10.2 and newer. Before this change, the format-v2 and format-v3 cases were permanent expected failures with fixed data seeds even on runtimes containing the upstream equality-delete cache fix. The affected older releases are now skipped instead of spending test time on known failures; production behavior is unchanged.

Apache Iceberg #15514, released in 1.11.0 and backported to 1.10.2, fixes equality-delete records being cached in a query-dependent field order. This PR:

Skips both mixed-delete tests on Iceberg versions older than 1.10.2, removes their expected-failure markers and permanent seed overrides on fixed versions, and fails pytest configuration when --iceberg lacks an expected runtime version.
Passes the expected Iceberg version through the Spark 4.0 extra-classpath pre-merge launcher and documents the required environment variable.
Keeps every count, full-row, and GPU-scan assertion. For the format-v3 count query only, disables adaptive query execution so plan validation sees the completed scan, then resets session configuration before fixture cleanup.

Older supported dependencies are not fixed by this PR, so #12885 should remain open. Iceberg 1.10.2 was policy-checked but was not GPU-runtime-tested locally. Shared pytest configuration and pre-merge launcher paths changed, so Databricks remains in CI scope.

Validation

Spark 4.1.1 / Iceberg 1.11.0 / Python 3.10: all six focused format-v2/format-v3 CPU/GPU cases passed across the three reader modes (6 passed, 45 deselected in 219.85 seconds). Full-row equality and the format-v3 GpuBatchScanExec assertion were exercised.
Spark 4.0.0 / Iceberg 1.10.1 / Python 3.10: all six focused cases were selected and skipped (6 skipped, 45 deselected in 1.73 seconds).
Seven release-boundary checks and two decorator-policy checks passed. The checked cases cover missing version, 1.6.1, 1.9.2, 1.10.0, 1.10.1, 1.10.2, and 1.11.0.
Unset EXPECTED_ICEBERG_VERSION with --iceberg failed during pytest configuration with exit 4 and the expected error. bash -n and a focused launcher probe confirmed that the Spark 4.0 extra-classpath path forwards Iceberg 1.10.1.
Python 3.10 compilation and git diff --check passed.

No production performance change is expected.

AI assistance: The change and PR description were prepared with Codex assistance.

Checklists

Documentation

Updated for new or modified user-facing features or behaviors
No user-facing change

Testing

Added or modified tests to cover new code paths
Covered by existing tests
(Please provide the names of the existing tests in the PR description.)
Not required

Performance

Tests ran and results are added in the PR description
Issue filed with a link in the PR description
Not required

Signed-off-by: Allen Xu <allxu@nvidia.com>

Copilot

🟢 Approval recommended

No unresolved blocking issues were identified.

Pull request overview

Re-enables mixed Iceberg delete validation for Iceberg 1.11+ while preserving quarantine behavior for older or unknown versions.

Changes:

Gates xfail and fixed-seed markers by Iceberg version.
Disables AQE for the v3 count-plan assertion.
Resets Spark configuration before cleanup.

File summaries

File	Description
`integration_tests/src/main/python/iceberg/iceberg_merge_on_read_test.py`	Updates mixed-delete test gating and session cleanup.

Review details

Files reviewed: 1/1 changed files
Comments generated: 0
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

greptile-apps · 2026-09-11T04:03:03Z

Confidence Score: 5/5

The PR appears safe to merge; no actionable correctness or repository-rule violations remain.

Findings

Pre-merge Iceberg run fails ▶

Summary

Requires an explicit expected Iceberg version when Iceberg tests are enabled.
Propagates that version through the Spark 4.0 extra-classpath pre-merge launcher.
Skips affected mixed-delete cases before Iceberg 1.10.2 and restores normal assertions on fixed versions.
Disables AQE for the v3 count-plan assertion and resets session configuration before fixture cleanup.

_{Reviews (5) · Last reviewed commit: "Skip mixed-delete tests on affected Iceb..."}

wjxiz1992 · 2026-09-11T04:26:54Z

build

firestarman · 2026-09-11T07:23:39Z


 pytestmark = iceberg_unsupported_mark

+# Iceberg 1.11.0 fixes cached equality-delete records being interpreted in the wrong field order:


Can the quarantine gate account for Iceberg 1.10.2 as a fixed runtime? Apache backported the equality-delete schema-ordering fix to the 1.10.x branch and released it in 1.10.2.

Signed-off-by: Allen Xu <allxu@nvidia.com>

wjxiz1992 · 2026-09-11T07:38:28Z

build

wjxiz1992 · 2026-09-11T09:17:10Z

build

liurenjie1024 · 2026-09-14T03:46:50Z

+# field order: https://github.com/apache/iceberg/pull/15514 (backport: #15605).
+# Keep the quarantine for older/unknown runtimes.
+_iceberg_eq_delete_cache_bug = (
+    runtime_iceberg_version is None or


I think we should throw exception if runtime_iceberg_version not detected?

Signed-off-by: Allen Xu <allxu@nvidia.com>

greptile-apps · 2026-09-14T04:09:49Z

+    if config.getoption('iceberg') and not os.environ.get('EXPECTED_ICEBERG_VERSION'):
+        raise pytest.UsageError(
+            "EXPECTED_ICEBERG_VERSION must be set when running Iceberg tests")


Pre-merge Iceberg run fails

The new check rejects every --iceberg invocation that does not set EXPECTED_ICEBERG_VERSION. The existing run_iceberg_extra_classpath_tests path passes --iceberg without exporting that variable, even though it already receives the exact Iceberg version. As a result, the Spark 4.0.x extra-classpath pre-merge run exits with UsageError during pytest configuration before collecting any tests. Please export the function's iceberg_version argument as EXPECTED_ICEBERG_VERSION in that launcher.

wjxiz1992 · 2026-09-14T04:12:58Z

build

Signed-off-by: Allen Xu <allxu@nvidia.com>

wjxiz1992 · 2026-09-14T06:16:09Z

build

Recover mixed-delete tests with Iceberg 1.11

f73aae1

Signed-off-by: Allen Xu <allxu@nvidia.com>

Copilot AI lite review requested due to automatic review settings September 11, 2026 04:00

wjxiz1992 added the test Only impacts tests label Sep 11, 2026

wjxiz1992 self-assigned this Sep 11, 2026

wjxiz1992 requested a review from liurenjie1024 September 11, 2026 04:00

Copilot started reviewing on behalf of wjxiz1992 September 11, 2026 04:00 View session

Copilot AI reviewed Sep 11, 2026

View reviewed changes

firestarman reviewed Sep 11, 2026

View reviewed changes

Include Iceberg 1.10.2 in mixed-delete test recovery

404e998

Signed-off-by: Allen Xu <allxu@nvidia.com>

wjxiz1992 changed the title ~~[SkipRecovery] Re-enable mixed-delete tests on Iceberg 1.11+ [reduced-it]~~ [SkipRecovery] Re-enable mixed-delete tests on Iceberg 1.10.2+ [reduced-it] Sep 11, 2026

firestarman previously approved these changes Sep 11, 2026

View reviewed changes

liurenjie1024 reviewed Sep 14, 2026

View reviewed changes

Fail fast when Iceberg test version is missing

28d9bdd

Signed-off-by: Allen Xu <allxu@nvidia.com>

wjxiz1992 dismissed firestarman’s stale review via 28d9bdd September 14, 2026 04:07

greptile-apps Bot reviewed Sep 14, 2026

View reviewed changes

Pass Iceberg version to extra-classpath tests

3cfaea8

Signed-off-by: Allen Xu <allxu@nvidia.com>

wjxiz1992 requested a review from a team as a code owner September 14, 2026 04:16

Skip mixed-delete tests on affected Iceberg versions

cf808ed

Signed-off-by: Allen Xu <allxu@nvidia.com>

wjxiz1992 changed the title ~~[SkipRecovery] Re-enable mixed-delete tests on Iceberg 1.10.2+ [reduced-it]~~ [SkipRecovery] Re-enable mixed-delete tests on Iceberg 1.10.2+ [reduced-it] [databricks] Sep 14, 2026

liurenjie1024 approved these changes Sep 14, 2026

View reviewed changes


		pytestmark = iceberg_unsupported_mark

		# Iceberg 1.11.0 fixes cached equality-delete records being interpreted in the wrong field order:

Uh oh!

Conversation

wjxiz1992 commented Sep 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Validation

Checklists

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟢 Approval recommended

Uh oh!

greptile-apps Bot commented Sep 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Findings

Summary

Uh oh!

wjxiz1992 commented Sep 11, 2026

Uh oh!

firestarman Sep 11, 2026

Choose a reason for hiding this comment

Uh oh!

wjxiz1992 Sep 11, 2026

Choose a reason for hiding this comment

Uh oh!

wjxiz1992 commented Sep 11, 2026

Uh oh!

wjxiz1992 commented Sep 11, 2026

Uh oh!

Uh oh!

liurenjie1024 Sep 14, 2026

Choose a reason for hiding this comment

Uh oh!

wjxiz1992 Sep 14, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Sep 14, 2026

Choose a reason for hiding this comment

Uh oh!

wjxiz1992 Sep 14, 2026

Choose a reason for hiding this comment

Uh oh!

wjxiz1992 commented Sep 14, 2026

Uh oh!

wjxiz1992 commented Sep 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

wjxiz1992 commented Sep 11, 2026 •

edited

Loading

greptile-apps Bot commented Sep 11, 2026 •

edited

Loading