You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This PR implements comprehensive ONNX Resize operator semantics for burn::tensor::module::interpolate as requested in #6147.
Previously, InterpolateOptions relied solely on align_corners: bool, while scale_factor only calculated output shape without driving coordinate transformations. This prevented burn-onnx from expressing several ONNX Resize attributes and blocked 19 ONNX backend conformance tests (burn-onnx#546).
This change introduces full support for ONNX coordinate transformation modes, nearest rounding tie-breaking modes, scale-driven coordinate mapping, and bicubic convolution options across Burn's tensor stack, reference CPU implementation (burn-flex), IR, and dispatch pipeline.
Key Changes
1. Core Abstractions & Type Definitions (burn-std, burn-backend, burn-tensor)
CoordinateTransformMode: Added support for HalfPixel, AlignCorners, Asymmetric, PytorchHalfPixel, HalfPixelSymmetric, and TfCropAndResize { roi: [[f32; 2]; 2], extrapolation_value: f32 }.
NearestRoundingMode: Added RoundPreferFloor, RoundPreferCeil, Floor, and Ceil.
CubicOptions: Added configurable cubic spline parameter a (default -0.75, TensorFlow -0.5) and exclude_outside weight zeroing/renormalization.
InterpolateOptions Backward Compatibility:
Mode-dependent defaults configured in InterpolateOptions::new(mode) guarantee zero silent numerical regressions:
Nearest defaults to CoordinateTransformMode::Asymmetric and NearestRoundingMode::Floor.
NearestExact defaults to CoordinateTransformMode::HalfPixel and NearestRoundingMode::RoundPreferCeil (matching PyTorch nearest-exact symmetric upsampling).
Bilinear / Bicubic / Lanczos3 default to CoordinateTransformMode::AlignCorners and NearestRoundingMode::RoundPreferFloor.
Preserved with_align_corners(bool) without #[deprecated] to maintain compatibility with existing internal callers and benchmarks, while guarding it to prevent inadvertently overriding Nearest and NearestExact coordinate mappings.
Builder State Safety:
Added mutual exclusion between with_output_size and with_scale_factor so setting one explicitly clears the other.
Added input validation ensuring all components of scale_factor are strictly positive (> 0.0).
2. Reference CPU Implementation (burn-flex)
High-Precision Coordinate Mapping (f64):
Evaluates all coordinate mapping, scales, and ROI arithmetic strictly in f64, preventing single-precision boundary truncation (e.g. $14 \to 2$ downsampling selecting index 7 rather than 6).
Direct length arithmetic (x_out * l_in / l_out) used for size-based asymmetric mapping when scale_factor is omitted, eliminating division-order floating point drift (e.g. $14 \to 18$).
Prioritizes options.scale_factor when present over out_len / in_len, resolving non-integer downsampling coordinate drift.
Boundary & Edge-Case Protection:
Added explicit branches for single-pixel output (out_len == 1) under AlignCorners and TfCropAndResize to prevent division-by-zero.
Allowed inverted ROIs (r_start > r_end) in TfCropAndResize to support flipped-crop operations, checking extrapolation strictly against physical tensor bounds.
Early skip for zero-weight taps (weight == 0.0) in bicubic sampling and adjoint accumulation, preventing IEEE 754 NaN * 0.0 = NaN contamination when dummy clamped boundary indices read non-finite inputs.
Added fallback boundary clamping when valid tap weights sum to <= 1e-6 under bicubic exclude_outside = true to prevent NaN propagation.
Full Precision & Autodiff Coverage:
Forward operations implemented across f32, f64, f16, and bf16.
Backward gradient passes (interpolate_bilinear_backward_* and bicubic equivalents) mirror forward fractional coordinates and zero out gradient accumulation for extrapolated sample points.
3. High-Level Neural Network Modules (burn-nn)
Updated Interpolate1dConfig and Interpolate2dConfig to expose the new builders.
Forwarded both optional sizing fields in Interpolate1d so setting both triggers the shared mutual-exclusion validation check consistently with Interpolate2d.
4. IR, Routing, and GPU Scope Boundary (burn-ir, burn-router, burn-cubecl)
IR Float Hashing: Canonicalized signed zeros (+0.0 vs -0.0) to identical bit patterns prior to hashing roi, extrapolation_value, CubicOptionsIr::a, and scale_factor, maintaining PartialEq and Hash consistency for burn-fusion plan caching.
CubeCL Compatibility:
Permitted compatible scale_factor calls for AlignCorners (which depend only on tensor dimensions) as well as calls where scales match resolved output/input ratios in exact f64 precision.
Corrected backward dimension unpacking in interpolate_backward after permute_nchw_to_nhwc ([_, in_height, in_width, _]), ensuring backward scale checks evaluate physical spatial dimensions.
nearest_interpolate.rs: Validated exact 0.5 coordinate tie-breaking across all 4 rounding modes, asymmetric vs. half-pixel shifts, f64 precision coordinate mapping ($14 \to 2$ downsample), size-based asymmetric precision ($14 \to 18$), and NearestExact symmetric upsampling.
bilinear_interpolate.rs: Validated scale-driven downsampling divergence, single-pixel output handling, HalfPixelSymmetric centering adjustment, normalized ROIs, inverted ROI flip-crops, and extrapolation value fill (inlined).
bicubic_interpolate.rs: Validated spline coefficients (a = -0.5 vs. a = -0.75), border tap zeroing under exclude_outside, and NaN skip protection on dummy zero-weight boundary taps.
autodiff/nearest_interpolate.rs: Validated backward scatter routing for precision coordinates ($14 \to 2$ and $14 \to 18$), bilinear crop extrapolation zero-gradients, non-integer scale gradient flows, and bicubic excluded boundary tap gradient skips.
- Add CoordinateTransformMode, NearestRoundingMode, and CubicOptions to burn-std
- Implement scale-driven mapping, nearest tie-breaks, and bicubic edge renormalization in burn-flex
- Support inverted ROIs, single-pixel outputs, and extrapolation in TfCropAndResize
- Wire options through burn-ir, burn-router, burn-dispatch, and burn-nn
- Ensure builder mutual exclusion between output_size and scale_factor
- Add comprehensive unit, boundary, and regression tests in burn-backend-tests
Fixestracel-ai#6147
❌ Patch coverage is 80.73529% with 262 lines in your changes missing coverage. Please review.
✅ Project coverage is 66.39%. Comparing base (5965468) to head (b45e72b). ⚠️ Report is 15 commits behind head on main.
❌ Your project check has failed because the head coverage (66.39%) is below the target coverage (80.00%). You can increase the head coverage or adjust the target coverage.
This PR extends Burn’s interpolation stack with ONNX Resize options needed by burn-onnx, addressing the coordinate mapping and sampling gaps described in #6147.
…scales, and tap handling
- Support compatible scale factors in CubeCL for AlignCorners and exact ratios
- Perform coordinate mapping and tap accumulation in f64 in burn-flex
- Skip zero-weight taps in bicubic interpolation to avoid NaN poisoning
- Pair NearestExact defaults with RoundPreferCeil for symmetric upsampling
- Guard with_align_corners to preserve Asymmetric mapping on Nearest
- Canonicalize signed zero before bit-hashing float fields in burn-ir
- Forward both output_size and scale_factor in Interpolate1d to trigger mutual exclusion checks
- Add regression tests across burn-backend-tests, burn-ir, and burn-nn
All 7 items flagged by Copilot are resolved and backed by dedicated regression tests:
CubeCL Compatible Scales: burn-cubecl now accepts scale_factor for AlignCorners mappings (which depend only on tensor dimensions) as well as HalfPixel/Nearest calls where the scale matches the resolved output/input ratio within tolerance. Only genuinely divergent scale-driven requests panic pending upstream cubek support.
f64 Coordinate Math: map_coordinate and tap calculations in burn-flex now evaluate in f64, resolving the 14 -> 2 downsample truncation where index 6 was selected instead of 7.
Bicubic Zero-Weight Tap Skip: Loops in burn-flex now skip taps with weight 0.0 before indexing, preventing NaN * 0.0 = NaN poisoning from dummy boundary reads when exclude_outside = true.
NearestExact Defaults: Updated NearestExact defaults to (HalfPixel, RoundPreferCeil) so upsampling [10.0, 20.0] to length 4 produces symmetric [10.0, 10.0, 20.0, 20.0].
with_align_corners Preservation: Guarded with_align_corners so it only modifies bilinear, bicubic, and lanczos modes, preserving Nearest's default Asymmetric mapping.
IR Signed Zero Canonicalization: Normalized +0.0 and -0.0 to identical bit representations prior to hashing roi, extrapolation_value, and CubicOptionsIr::a, maintaining PartialEq and Hash consistency for burn-fusion plan caching.
1D Size and Scale Validation: Forwarded both sizing options in Interpolate1d so configuring both triggers the shared mutual-exclusion panic.
All unit, backend, and autodiff interpolation tests pass locally with clean formatting and zero clippy warnings.
Ready for another look @antimora whenever you get a chance.
[P2] These checks accept distinct nearest semantics but lower them to identical CubeK options. Resizing [10, 20] to width 4 with HalfPixel + Floor requires [10, 10, 10, 20], yet Nearest lowers it like Asymmetric + Floor, and NearestExact lowers it like HalfPixel + RoundPreferCeil; both require [10, 10, 20, 20]. NearestExact + Asymmetric is also accepted without preserving its transform. Validate the complete mode/transform/rounding tuple and reject unsupported combinations until CubeK can represent them.
Normalize nonzero negative retained weight sums
crates/burn-flex/src/ops/interpolate.rs:672
[P2] A negative retained weight sum still needs normalization; the API does not restrict the cubic coefficient. For HalfPixel resizing [10, 20] to one pixel with a = 8 and exclude_outside = true, both valid weights are -0.5. Normalizing their sum of -1 gives 15, but this branch falls back to 20. Check the magnitude of the sum so nonzero negative sums are normalized in both forward and backward.
All four follow-up review items have been resolved and verified:
CubeCL Backward NHWC Indexing: Fixed dimension destructuring in crates/burn-cubecl/src/kernel/interpolate/base.rs from [_, _, in_height, in_width] to [_, in_height, in_width, _] after permute_nchw_to_nhwc, ensuring spatial dimensions are accurately read during backward scale checks.
Exact Scale Equality in CubeCL: Replaced the float epsilon check with exact f64 ratio comparison before discarding scale factors, preventing unintended coordinate shifts near integer boundaries.
Flex Asymmetric Precision ($14 \to 18$): Updated map_coordinate to compute x_out * l_in / l_out directly when no scale factor is provided, eliminating intermediate float division truncation and correctly mapping coordinate 9 to index 7.
Autodiff Coverage: Added backward gradient tests inline for $14 \to 18$ scatter routing, bilinear crop extrapolation zero-gradients, non-integer scales, and bicubic excluded border tap gradients.
All 37 module interpolation tests and 14 autodiff interpolation tests pass locally with clean formatting and zero clippy warnings.
Ready for another look @antimora whenever you get a chance.
All items from the latest review pass are resolved (2 previously missed issues):
CubeCL Nearest Semantics: In crates/burn-cubecl/src/kernel/interpolate/base.rs, strictly validated the full (mode, coordinate_transform, nearest_mode) tuple. Only (Nearest, Asymmetric, Floor) and (NearestExact, HalfPixel, RoundPreferCeil) are permitted to lower to CubeK; all unsupported combinations panic with actionable guidance.
Bicubic Retained Weight Normalization: In crates/burn-flex/src/ops/interpolate.rs, changed the tap sum check in bicubic_axis_taps to sum.abs() > 1e-6. Non-zero negative weight sums (such as with non-standard spline coefficients like $a = -8.0$ under exclude_outside) are now correctly normalized across forward and backward passes.
Regression Tests: Added regression tests inline for negative weight sum normalization and CubeCL combination rejection, with zero modifications to test modules or extra files.
All 39 tensor and 14 autodiff interpolation tests pass locally with clean formatting and zero clippy warnings.
- Remove lazy-dependent integration test from shared backend test harness
- Pass options by reference in map_options to eliminate redundant clones
- Add direct unit tests for unsupported nearest option combinations in burn-cubecl
The reason will be displayed to describe this comment to others. Learn more.
Preserve half-pixel rounding ties when resizing by output size. Computing coordinates through a rounded scale ratio can select the wrong nearest neighbor.
The reason will be displayed to describe this comment to others. Learn more.
Can we distinguish zero weight sums from small nonzero sums here? The cutoff appears to replace potentially valid interpolation weights with a boundary sample.
The reason will be displayed to describe this comment to others. Learn more.
This test uses AlignCorners, which maps coordinates from tensor sizes, so it does not exercise scale-driven gradients.
The test should probably use scales where input_size * scale is nonintegral, and assert expected gradient values rather than only positivity and finiteness.
The reason will be displayed to describe this comment to others. Learn more.
This duplicates test_nearest_downsample_f64_precision_backward in burn-backend-tests, which checks the same gradient routing through the public API. Please remove this local test.
Thanks for the thorough review @laggui! All 6 comments have been addressed:
Changelog: Added an entry under ## Unreleased in burn-book/src/changelog.md detailing the ONNX Resize operator additions, coordinate transformation modes, rounding modes, cubic options, scale-driven mapping, and f64 precision improvements.
Half-Pixel Rounding Ties: Updated CoordinateTransformMode::HalfPixel and PytorchHalfPixel to evaluate (x_out + 0.5) * l_in / l_out - 0.5 directly when resizing without scale factors, eliminating division ratio drift on exact half-pixel ties.
Bicubic Normalization: Replaced sum.abs() > 1e-6 with if sum != 0.0 && sum.is_finite(), ensuring valid small non-zero weight sums are properly normalized and only true zeros fall back to boundary clamping.
Scale-Driven Autodiff Gradients: Updated test_bilinear_backward_non_integer_scale_gradients to use CoordinateTransformMode::HalfPixel and non-integral scale [1.0, 0.6] on length 4, asserting exact analytical gradients [2/3, 1/3, 1.0, 0.0].
Deduplication: Removed the redundant local backward unit test in burn-flex and the duplicate builder mutual exclusion test in burn-backend-tests.
All 37 module interpolation tests and 14 autodiff tests pass locally with clean formatting and zero clippy warnings.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #6147
Description
This PR implements comprehensive ONNX
Resizeoperator semantics forburn::tensor::module::interpolateas requested in #6147.Previously,
InterpolateOptionsrelied solely onalign_corners: bool, whilescale_factoronly calculated output shape without driving coordinate transformations. This preventedburn-onnxfrom expressing several ONNX Resize attributes and blocked 19 ONNX backend conformance tests (burn-onnx#546).This change introduces full support for ONNX coordinate transformation modes, nearest rounding tie-breaking modes, scale-driven coordinate mapping, and bicubic convolution options across Burn's tensor stack, reference CPU implementation (
burn-flex), IR, and dispatch pipeline.Key Changes
1. Core Abstractions & Type Definitions (
burn-std,burn-backend,burn-tensor)CoordinateTransformMode: Added support forHalfPixel,AlignCorners,Asymmetric,PytorchHalfPixel,HalfPixelSymmetric, andTfCropAndResize { roi: [[f32; 2]; 2], extrapolation_value: f32 }.NearestRoundingMode: AddedRoundPreferFloor,RoundPreferCeil,Floor, andCeil.CubicOptions: Added configurable cubic spline parametera(default-0.75, TensorFlow-0.5) andexclude_outsideweight zeroing/renormalization.InterpolateOptionsBackward Compatibility:InterpolateOptions::new(mode)guarantee zero silent numerical regressions:Nearestdefaults toCoordinateTransformMode::AsymmetricandNearestRoundingMode::Floor.NearestExactdefaults toCoordinateTransformMode::HalfPixelandNearestRoundingMode::RoundPreferCeil(matching PyTorchnearest-exactsymmetric upsampling).Bilinear/Bicubic/Lanczos3default toCoordinateTransformMode::AlignCornersandNearestRoundingMode::RoundPreferFloor.with_align_corners(bool)without#[deprecated]to maintain compatibility with existing internal callers and benchmarks, while guarding it to prevent inadvertently overridingNearestandNearestExactcoordinate mappings.with_output_sizeandwith_scale_factorso setting one explicitly clears the other.scale_factorare strictly positive (> 0.0).2. Reference CPU Implementation (
burn-flex)f64):f64, preventing single-precision boundary truncation (e.g.x_out * l_in / l_out) used for size-based asymmetric mapping whenscale_factoris omitted, eliminating division-order floating point drift (e.g.options.scale_factorwhen present overout_len / in_len, resolving non-integer downsampling coordinate drift.out_len == 1) underAlignCornersandTfCropAndResizeto prevent division-by-zero.r_start > r_end) inTfCropAndResizeto support flipped-crop operations, checking extrapolation strictly against physical tensor bounds.weight == 0.0) in bicubic sampling and adjoint accumulation, preventing IEEE 754NaN * 0.0 = NaNcontamination when dummy clamped boundary indices read non-finite inputs.<= 1e-6under bicubicexclude_outside = trueto prevent NaN propagation.f32,f64,f16, andbf16.interpolate_bilinear_backward_*and bicubic equivalents) mirror forward fractional coordinates and zero out gradient accumulation for extrapolated sample points.3. High-Level Neural Network Modules (
burn-nn)Interpolate1dConfigandInterpolate2dConfigto expose the new builders.Interpolate1dso setting both triggers the shared mutual-exclusion validation check consistently withInterpolate2d.H = 1 -> 1) maintains neutral scaling (scale = 1.0,roi[0] = [0.0, 1.0]), preventing unintentional extrapolation.4. IR, Routing, and GPU Scope Boundary (
burn-ir,burn-router,burn-cubecl)+0.0vs-0.0) to identical bit patterns prior to hashingroi,extrapolation_value,CubicOptionsIr::a, andscale_factor, maintainingPartialEqandHashconsistency forburn-fusionplan caching.scale_factorcalls forAlignCorners(which depend only on tensor dimensions) as well as calls where scales match resolved output/input ratios in exactf64precision.interpolate_backwardafterpermute_nchw_to_nhwc([_, in_height, in_width, _]), ensuring backward scale checks evaluate physical spatial dimensions.cubekintegration.Verification & Testing
burn-backend-tests:nearest_interpolate.rs: Validated exact 0.5 coordinate tie-breaking across all 4 rounding modes, asymmetric vs. half-pixel shifts,f64precision coordinate mapping (NearestExactsymmetric upsampling.bilinear_interpolate.rs: Validated scale-driven downsampling divergence, single-pixel output handling,HalfPixelSymmetriccentering adjustment, normalized ROIs, inverted ROI flip-crops, and extrapolation value fill (inlined).bicubic_interpolate.rs: Validated spline coefficients (a = -0.5vs.a = -0.75), border tap zeroing underexclude_outside, and NaN skip protection on dummy zero-weight boundary taps.autodiff/nearest_interpolate.rs: Validated backward scatter routing for precision coordinates (cargo check -p burn-backend-tests --benches: Passed (0 warnings, 0 errors)cargo test-flex --test tensor interpolate: Passed (37 tests)cargo test-flex --test autodiff interpolate: Passed (14 tests)cargo test -p burn-tensor interpolate: Passed (8 tests)cargo test -p burn-nn --features burn-core/flex interpolate: Passed (6 tests)cargo test -p burn-ir interpolate: Passedcargo test -p burn-flex interpolate: Passed (10 tests)cargo check -p burn-cubecl --features wgpu --tests: Passedcargo clippy --all-targets -- -D warnings: Passed (0 errors, 0 warnings)cargo fmt --check: Clean