Textures appear black and reflections are off on scenes with reflection probes

[matthewbcool]

Description
Environment map reflections appear incorrectly positioned and textures are black in high quality mode

To Reproduce
Steps to reproduce the behavior:

1. Go to https://hubs.mozilla.com/scenes/gHuNHUI/cliffsideretreat
2. Spawn in a camera, 3d object with PBR materials and observe

Expected behavior
Materials reflections/materials should be more consistently lit with env maps IBL.

Screenshots

90025f635997a4dc53a9a98ea22c7827.mp4

Hardware

• Device: [MacBook M1]
• Browser: [Chrome, Firefox]

Additional context
also tested on PC, windows Firefox

[takahirox]

Thanks for the problem report. Probably it was broken by the recent Three.js upgrade #5488. Let me investigate...

[matthewbcool]

Scenes with only an env map and no reflection probes don't seem to have this problem: https://hubs.mozilla.com/scenes/HsSRCtj/merry-go

[takahirox]

Yes, light probe is suspicious. And I noticed that commenting this line out makes the objects non-black https://github.com/mozilla/hubs/blob/d815053b1938068d54fda517c0f9af0760423b09/src/gltf-component-mappings.js#L613

[takahirox]

Reflection Probe texture (envMap) passed to WebGL is very suspicious...

Left r131, right r141.

[takahirox]

/cc @netpro2k

[emclaren]

is this possibly related to #5571

[takahirox]

is this possibly related to #5571

Maybe not

[takahirox]

Ah, probably I figured out the root issue finally. The reflection proble patch has conflicts in the logic (algorithm) with Three.js r141. This is the reason why I don't like applying patches to Three.js, it's very hard to figure out the root issue if having no conflict in the code but having conflicts in logic (algorithm)... (I have spent about a week for this problem.)

I'll share the details soon.

[takahirox]

If I'm right the root issue seems to be switching environment maps after setting up WebGL programs in the renderer.

The renderer and other webgl modules generate shader code, get program cache, decide whether changing program, and so on, from material (and other objects) properties.

• https://github.com/MozillaReality/three.js/blob/7c6fb585ffa7737544871d2bbd4b0864c316f148/src/renderers/WebGLRenderer.js
• https://github.com/MozillaReality/three.js/blob/262eb259a1f92c3554a11353b7f52c13900f879e/src/renderers/webgl/WebGLProgram.js
• https://github.com/MozillaReality/three.js/blob/12f550e0f44f68e5b6c946dd242ee08e42d209e4/src/renderers/webgl/WebGLPrograms.js

Switching envMap after setting up them can cause mismatch.

More proper implementation may be setting up envMapA/B first and include them for decision of shader code generation, program cache, and so on.

[netpro2k]

Looks like the root of this regression was this change in Three 138 mrdoob/three.js#23322 ... Previously PMREM textures had a fixed size. They are now variable so we need to further extend our multi-environment map support to account for this.

I think we can just have it be a know limitation for now that all reflection probes and environment maps have to be the same resolution. We can enforce this in the Blender addon.

Ultimately we probably can support this when we do another pass at reflection probes.

[takahirox]

A workaround PR for Hubs Blender add-on has been opened Hubs-Foundation/hubs-blender-exporter#88

[takahirox]

Opened an issue for the root issue of Reflection Probe implementation Hubs-Foundation/three.js#66

We may close this issue after Hubs-Foundation/hubs-blender-exporter#88 lands.

[takahirox]

Closing this issue because Hubs-Foundation/hubs-blender-exporter#88 has been merged.

Please re-export your scenes if you encounter the problem.