Fix render test false negative

[NathanMOlson]

Fixes #4862. Merging this change will cause the terrain/fog render test to fail (as it should), due to #4859.

The pixelmatch library does not detect a difference between transparent and white. To work around this, the horizon and fog colors have been changed to not be white.

• Confirm your changes do not include backports from Mapbox projects (unless with compliant license) - if you are not sure about this, please ask!
• Briefly describe the changes in this PR.
• Link to related issues.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.95%. Comparing base (86ed382) to head (b869ce9).
Report is 194 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4864      +/-   ##
==========================================
- Coverage   87.66%   86.95%   -0.71%     
==========================================
  Files         265      265              
  Lines       37937    37937              
  Branches     2407     2485      +78     
==========================================
- Hits        33257    32988     -269     
- Misses       3608     3828     +220     
- Partials     1072     1121      +49     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[HarelM]

I think we should not merge a failing render test though, we should update the image or fix the root cause of this test as oart of this PR.

[HarelM]

@NathanMOlson any updates on this PR? Can we move it forward somehow?

[NathanMOlson]

This PR can be moved forward by fixing #4859. I don't know how to fix #4859.

Also, a better solution than the one in this PR would be to update the pixelmatch library to include the changes here: mapbox/pixelmatch#142. These changes have been approved by the maintainer, but have not been merged because they add processing time. I do not see a way to fix the pixelmatch library without adding processing time. In my opinion, MapLibre would be better served with a version of pixelmatch that detects transparency differences, even if it is slightly slower.

[HarelM]

In the past, a similar problem was solved by adding a border around the tiles to stitch different zoom level for tiles when terrain is on.
Is it possible that the border needs to be "bigger" to solve this issue (the tile is very far away).
I'm not sure it's the right fix, but wondering if it works solve the issue...

[NathanMOlson]

mapbox/pixelmatch#142 has been merged in the pixelmatch library.

[HarelM]

I believe I addressed the change here:

• Bump pixelmatch from 6.0.0 to 7.1.0 #5541

Is there anything else needed as part of this PR?

[NathanMOlson]

@HarelM #5541 updated several render test so that the expected result includes visible defects.

high-pitch/pitch95 has 3 white pixels near the horizon in the upper left.
terrain/fog has a transparent section near the horizon in the upper right
terrain/symbol-add-terrain the tile "skirt" is visible

It feels wrong to test against defective results, would it be better to remove/disable these tests? I think the fact that prior to #5541, the expected results didn't have the defects, suggests that these are regressions.

#5541 fixes the problem that I was pointing out with this PR, which is that pixelmatch could pass render tests that should fail. However, it does not fix the defective terrain/fog rendering.

I'm fine with this being closed, or left open as a demonstration of the rendering defect in terrain/fog.

[HarelM]

The render tests simply define the current behavior, if there's a fix that changes the results of the render test, we'll update the image, that's fine.
It's even better to see that if a fix was done the relevant render test works change for the best.

As for skipped tests, there were a bunch of skipped tests when I started maintaining this library, and I removed all of them as keeping skipped tests doesn't really help anyone.

If you want to indicate there's a behavior that needs fixing, an issue with a simple reproduction (using the style.json for the render test even) would be a much better approach.

[HarelM]

As far as I can tell the following issue is the tracking one:

• Terrain "hole" rendered when terrain tiles are missing #4859

Closing.