MSAA does not always work when doing RTT.

[LR17]

Description

I would expect to have same output image when rendering to screen with antialias: true or when using EffectComposer + RenderPass + OuputPass and a render target with 4 samples

Reproduction steps

Open https://jsfiddle.net/5kczseda/1/.
The left cube is rendered to screen while the right cube uses postprocessing and has aliasing

However I found that this behaviour changes depending on material type and clear color opacity. You can reproduce the conditions by using the checkboxes in the gui. The "appyPatch" checkbox premultiplies alpha inside OutputPass shader, before tonemapping and colorspace conversion.

1. basic material + transparent clear color = aliasing
2. basic material + transparent clear color + patch = ok
3. standard material + transparent clear color = aliasing
4. standard material + transparent clear color + patch = aliasing
5. basic material + solid clear color = ok
6. standard material + solid clear color = aliasing

I would also like to point out that setting the clear color on the RenderPass produces a different color

Code

See the live example

Live example

• https://jsfiddle.net/5kczseda/1/

Screenshots

Version

157

Device

No response

Browser

No response

OS

No response

[Mugen87]

Just to clarify: There are no issues when setting alpha to false, right?

The problem is the title of the issue is a bit misleading since RTT with multisampled render targets should work (see https://threejs.org/examples/webgl2_multisampled_renderbuffers).

[Mugen87]

Wait, I can see there is aliasing when using alpha: false and lit materials. Let's focus on this issue first.

Here is a simplified version of your fiddle reproducing the issue: https://jsfiddle.net/n41advb2/1/

What I can see is that when reducing the ambient light intensity to a more common value, the aliasing on the right cube disappears: https://jsfiddle.net/n41advb2/2/

So this is maybe some sort of precision related issue 🤔 .

BTW: The aliasing is unrelated to OutputPass. Even a simple copy shader produces the aliasing: https://jsfiddle.net/n4pu9286/

[LR17]

It is a bit subtle but switching to standard material shows something even in the official example:
https://jsfiddle.net/e6zraqct/

The lines on the right seem to me a bit "fatter", also red color seems slightly different

[Mugen87]

The color differences you see is related to fog and color spaces. It is discussed in #24362.

[Mugen87]

Interesting! The aliasing also disappears when using THREE.UnsignedByteType for the multisampled render target: https://jsfiddle.net/6q07hpa2/

My theory is that what you see is a special case related to value ranges that MSAA can't handle. You don't see it when rendering to the default framebuffer because values get clamped (the default framebuffer is RGBA8). When using render targets with higher precision (FP16 or FP32), the clamping does not happen and MSAA can't mitigate the aliasing.

[LR17]

Good catch! I changed the original fiddle to use THREE.UnsignedByteType:
https://jsfiddle.net/bwfjrysz/

Notable changes :
4. standard material + transparent clear color + patch = aliasing with THREE.HalfFloatType, ok with THREE.UnsignedByteType
6. standard material + solid clear color = aliasing -> aliasing with THREE.HalfFloatType, ok with THREE.UnsignedByteType

[WestLangley]

Using MSAA when values can be large can be problematic. One very bright sample can dominate the result.

See https://mynameismjp.wordpress.com/2012/10/24/msaa-overview/, particularly the section "Working with HDR and Tone Mapping"

[Mugen87]

The other problem in this topic is related to premultiplied alpha. Similar to CopyShader, I think output shader should apply premultiply alpha as well. This solves the rendering issue of the original use case, see https://jsfiddle.net/dqLv3guz/. In this fiddle I just replaced OutputPass with a simple copy pass. Both wireframes are not rendered correctly.

@WestLangley We have discussed this topic in the past so I want to ask if it's okay with you to keep both passes consistent and premultiply alpha (see #26179).

[WestLangley]

@Mugen87. Agreed, we should be consistent. Also, I always prefer adding an inline comment for clarity in such cases.

[Mugen87]

@LR17 In your fiddle, you have added premultiplied alpha right after the sampling of the texture. Meaning before tone mapping and sRGB color space conversion. Shouldn't premultiplied alpha be the last step in the fragment shader?

[LR17]

I did find that aliasing was going away only if premultiplying alpha before tonemapping and color space conversion.

Here the same advice is given, although I really don't know :)

[LR17]

After tonemapping and before colorspace conversion seem slightly better in the project I'm working on

[Mugen87]

This needs more investigation especially since enabling premultiplied alpha in OutputPass affects existing examples/apps so this is a breaking change.

[Mugen87]

@WestLangley Something still confuses me. In this issue and in #28331 you only get a visual correct result when premultiplying alpha happens at a specific place.

In the built-in shaders, the <premultiplied_alpha_fragment> chunk comes after tone mapping and color space conversion. If I do the same in the fiddle from #28331, I get a black outline around meshes in the multisampled code path, see https://jsfiddle.net/8chto6vp/.

Premultiplying alpha before tone mapping and color space conversion produce the expected result: https://jsfiddle.net/Lwsavq7e/

Do you know why this happens?

[WestLangley]

The input to OutputPass already has premultiplied alpha in your example, so it is not clear why one would premultiply again...

In any event, I don't think you are getting "the expected result"; I think application of the OETF is just making the color lighter, and thus obfuscating the issue. I think application of the OETF to RGB values premultiplied by alpha is not correct.

[TriggerBlazer]

@Mugen87 I have another question.
Rendering a transparent object directly to the screen, the color is consistent with the alpha-blending formula,see https://jsfiddle.net/trigger0607/qrz87dgm/5/
resultColor = rgb(0x7A * 0.2 + 0x66 * (1-0.2),...,...) = rgb(106,106,106)

But if use using EffectComposer + RenderPass + OuputPass, the output color is rgb(104,104,104) ,see https://jsfiddle.net/trigger0607/ars47no9/7/
The result was close, but not the same,does it have anything to do with precision?

[Mugen87]

see #27184