Working with sRGBEncoding

[shaoruu]

Hi, great work! How could I get this to work with THREE.sRGBEncoding for renderer.outputEncoding?

[stevinz]

Thanks!

The examples in this repo use WboitPass as a stand-alone renderer, however, it is designed as a Pass for EffectComposer. WebGLRenderer.outputEncoding is only respected when rendering to screen. EffectComposer (and in turn WboitPass) use render targets encoded in the linear space. EffectComposer expects processing in the linear space to reduce color conversions during every pass. (source) (further reading). When using post processing, it is possible to use a gamma correction pass at the end of your pass chain to ensure a sRGB encoded output (source).

I have added an example (https://stevinz.github.io/three-wboit/EffectComposer.html) that uses this EffectComposer workflow to demonstrate. You can cycle the gamma correction ShaderPass on / off in the options. Hope this helps.

[shaoruu]

Thank you for the detailed explanation!

I added in the gamma correction shader, but things appear lighter than they used to be. Do you have any idea why this is happening?

Before:

After (with wOIT and gamma correction)

I tried changing the weight from 0 to 1, but it didn't help.

[stevinz]

That is interesting. Initial thoughts is that the textures on the trees aren't sRGB encoded, or that lighting is enabled on the tree material? I have updated the EffectComposer.html example (source - live link) to use sRGB encoded textures.

I am having trouble trying to recreate the issue. I would be willing to spend more time looking into it if you could provide a minimal example demonstrating the problem. (codepen, jsfiddle, etc.)

[shaoruu]

Thanks for the response! I just found out something weird... The screenshot I produced is on one of my dual monitor setups. I switched it to the other monitor, and everything looked correct. What could be the cause of this?

Things look great on this monitor.

Link to the monitor that creates weird looking leaves.

[stevinz]

Hmm, tricky. Monitors have encoding settings as well. I know on mine when I open the monitor settings with the physical buttons on the side / rear of the monitors, under 'Color Settings' I can select 'sRGB', 'P3', 'Warm', 'Cool', etc. Could be a hardware setting? These settings are external of the computer / OS / graphics card...

[shaoruu]

Weird... I can't seem to find color settings by pressing the buttons. I do see an "input color format" of "YCbCr", does that have to do with the issue?

[stevinz]

Hard to say, even with different monitor settings the difference you posted the first time looks pretty extreme. Ignoring the monitor issue for a moment... When you're using wboit + gamma correction do you have renderer.outputEncoding set to Linear or sRGB? Now that the sRGB encoding is happening in the Effect Composer the renderer should be set to Linear, otherwise it's possible the encoding is happening twice in some cases.

[shaoruu]

I just checked and I do have it on sRGBEncoding, but after changing it to LinearEncoding, the effects still persist. Below is my custom shader material:

const material = new ShaderMaterial({
  vertexColors: true,
  fragmentShader,
  vertexShader,
  uniforms: {
    ...UniformsUtils.clone(ShaderLib.basic.uniforms),
    // map: this.uniforms.atlas,
    uSunlightIntensity: this.uniforms.sunlightIntensity,
    uAOTable: this.uniforms.ao,
    uMinBrightness: this.uniforms.minBrightness,
    uFogNear: this.uniforms.fogNear,
    uFogFar: this.uniforms.fogFar,
    uFogColor: this.uniforms.fogColor,
    uTime: this.uniforms.time,
    ...uniforms,
  },
}) as CustomShaderMaterial;

WboitUtils.patch(material);

export const DEFAULT_CHUNK_SHADERS = {
  vertex: ShaderLib.basic.vertexShader
    .replace(
      "#include <common>",
      `
attribute int light;

varying float vAO;
varying vec4 vLight;
varying vec4 vWorldPosition;
uniform vec4 uAOTable;
uniform float uTime;

vec4 unpackLight(int l) {
  float r = float((l >> 8) & 0xF) / 15.0;
  float g = float((l >> 4) & 0xF) / 15.0;
  float b = float(l & 0xF) / 15.0;
  float s = float((l >> 12) & 0xF) / 15.0;
  return vec4(r, g, b, s);
}

#include <common>
`
    )
    .replace(
      "#include <color_vertex>",
      `
#include <color_vertex>

int ao = light >> 16;

vAO = ((ao == 0) ? uAOTable.x :
    (ao == 1) ? uAOTable.y :
    (ao == 2) ? uAOTable.z : uAOTable.w) / 255.0; 

vLight = unpackLight(light & ((1 << 16) - 1));
`
    )
    .replace(
      "#include <worldpos_vertex>",
      `
vec4 worldPosition = vec4( transformed, 1.0 );
#ifdef USE_INSTANCING
  worldPosition = instanceMatrix * worldPosition;
#endif
worldPosition = modelMatrix * worldPosition;
vWorldPosition = worldPosition;
`
    ),
  fragment: ShaderLib.basic.fragmentShader
    .replace(
      "#include <common>",
      `
uniform vec3 uFogColor;
uniform float uFogNear;
uniform float uFogFar;
uniform float uSunlightIntensity;
uniform float uMinBrightness;
uniform float uTime;
varying float vAO;
varying vec4 vLight; 
varying vec4 vWorldPosition;

#include <common>
`
    )
    .replace(
      "#include <envmap_fragment>",
      `
#include <envmap_fragment>

// Intensity of light is wavelength ** 2 
float s = min(vLight.a * vLight.a * uSunlightIntensity * (1.0 - uMinBrightness) + uMinBrightness, 1.0);
float scale = 2.0;

outgoingLight.rgb *= vec3(s + pow(vLight.r, scale), s + pow(vLight.g, scale), s + pow(vLight.b, scale));
outgoingLight *= vAO;
`
    )
    .replace(
      "#include <fog_fragment>",
      `
vec3 fogOrigin = cameraPosition;

float depth = sqrt(pow(vWorldPosition.x - fogOrigin.x, 2.0) + pow(vWorldPosition.z - fogOrigin.z, 2.0));
float fogFactor = smoothstep(uFogNear, uFogFar, depth);

gl_FragColor.rgb = mix(gl_FragColor.rgb, uFogColor, fogFactor);
`
    ),
};

For all the textures used in the project, I made sure that they had encoding set as sRGBEncoding.

This is the link to the shader source, and this (and this) is to the material code.

[shaoruu]

I don't think the shader code matters. I just tried using a MeshBasicMaterial with the same texture atlas and the weird brightness still exists. Could this be related to using CanvasTexture for the generated texture atlas?

[stevinz]

It could be, is the CanvasTexture encoding set to sRGB?

Now that you're using an EffectComposer workflow, try replacing the WboitPass with a standard render pass and see if the problem persists. That would at least isolate the problem between WboitPass and something else...

import { RenderPass } from 'three/addons/postprocessing/RenderPass.js';

// replace
// composer.addPass( new WboitPass( renderer, scene, camera ) );

// with
composer.addPass( new RenderPass( scene, camera ) );

[shaoruu]

Things look normal with a RenderPass:

The CanvasTexture is also set to sRGB 🤔

[stevinz]

After running WboitUtils.patch(material), if you then set material.wboitEnabled = false, does the problem still occur on transparent objects? During the WboitPass, normal transparent objects and wboitEnabled transparent objects are rendered separately, I'm wondering if it could be a blending issue.

[shaoruu]

This is interesting... The problem still occurs on transparent objects.

[stevinz]

Hmm, another thought. Are you doing any passes in EffectComposer before the WboitPass? If not, you should probably have WboitPass clear the write buffer by setting a clear color (if you're not already). In order to have WboitPass perform a clear, you need to pass a THREE.Color the constructor. For clearing to black it would simply be:

const wboitPass = new WboitPass( renderer, scene, camera, new THREE.Color() );

Does that make a difference?

[shaoruu]

I do not have any passes before the WboitPass, and passing a clear color didn't change anything :(

The only two passes I have are these:

[shaoruu]

Seems like this problem happens to other people too. This is @donmccurdy's solution: link. I will try to dig deeper into three-wboit's WboitPass and convert it into postprocessing's syntax locally.

[stevinz]

I'm not sure that is the same issue, your colors seem off differently than the colors in that post. Do you have a link to a specific texture that you are having this issue with? It would be nice to have a minimal example...

It shouldn't be necessary, or desired, but someone also suggested to try setting EffectComposer's render target's encoding:

composer.renderTarget1.texture.encoding = THREE.sRGBEncoding;
composer.renderTarget2.texture.encoding = THREE.sRGBEncoding;

I would be curious if that made a difference. You could try changing the encoding on the main render target in WboitPass:

const wboitPass = new WboitPass( renderer, scene, camera );
wboitPass.baseTarget.texture.encoding = THREE.sRGBEncoding;

[stevinz]

Okay, I think I've solved it.

The blending methods used in WboitPass result in visually accurate colors, but with alpha value's that vary. For the sRGB conversion the color values should have an alpha value of 1.0... I have created a new shader to encode from Linear space to sRGB space that premultiplies the alpha. I'm almost positive it will fix your issue.

I have updated the library and the build on npm. To use:

import { sRGBShader } from 'three-wboit';

// replace
// composer.addPass( new ShaderPass( GammaCorrectionShader ) );

// with
composer.addPass( new ShaderPass( sRGBShader ) );

[shaoruu]

Thanks for your help so far!

I just tried out the sRGBShader replacing it with the GammaCorrectionShader, but it seems like transparent objects have turned black, and I'm not sure why :/

Here's the source code for my example:

• PR

Let me see if I can setup a minimized example somewhere so it's easier to reproduce the problem.

[stevinz]

Very interesting..

It could be an issue with WboitUtils.patch. I haven't tested it with ShaderMaterial. I can check it out, but in the meantime you could try adding the shader code directly into you shader chuck. When you build your shader material, add uniforms renderStage and weight

At then in your fragment shader (shaders.ts)

Add the uniforms:

    uniform float renderStage;
    uniform float weight;

And at the very end, add the wboit code:

    if ( renderStage == 1.0 ) {
       vec4 accum = gl_FragColor.rgba;

	#ifndef PREMULTIPLIED_ALPHA
	    accum.rgb *= accum.a;
	#endif

	float z = gl_FragCoord.z;
        float scaleWeight = 0.7 + ( 0.3 * weight );
        float w = clamp( pow( ( accum.a * 8.0 + 0.001 ) * ( - z * scaleWeight + 1.0 ), 3.0 ) * 1000.0, 0.001, 300.0 );
        gl_FragColor = accum * w;
    } else if ( renderStage == 2.0 ) {
        gl_FragColor = vec4( gl_FragColor.a * gl_FragCoord.z );
    }

And then in index.ts, change

// Old
// WboitUtils.patch(material);

// New
material.wboitEnabled = true;

[stevinz]

Hmm, well I just tested it with ShaderMaterial and that's not it...

Does the darkness / blackness go away if you set opacity to a lower value, like 0.5?

[shaoruu]

Let me try. I'm creating a tiny project and am trying to reproduce the problem. Thanks for your help man 💯

[shaoruu]

For my actual project, setting opacity to 0.5 doesn't change. I'm starting to wonder whether or not it's the TextureAtlas class's fault 🤔

[shaoruu]

Ok here's the project: link

On the left is using the same texture generated from the texture atlas, but without doing patch. On the right is wboit patched.

[stevinz]

Okay, thanks I'll take a look

[stevinz]

Okay, so using the original GammaCorrectionShader I do see what you were talking about. The object on the right (the Wboit patched object) looks a little lighter / brighter.

However, I think that's because in your file style.css you have background-color: #242424;. Wboit rendering uses special blend modes and the final rendered image has partially transparent pixels, it needs the background to be black. When I changed it to background-color: #000000;, the colors look identical between the two objects. Does that work for you?

[shaoruu]

Hmm, that's weird. I removed the background color and pretty much everything in the styles.css, and this is what I got:

The mesh on the right is too bright to be visible. BTW, I've pushed my changes to the same repo.
Do you see the same effect?

[stevinz]

The background has to be black, not white. Try adding at top of style.css:

:root {
  background-color: #000000;
}

[shaoruu]

Here's what it looks like in black:

[stevinz]

Interesting, here's on my machine:

[stevinz]

And lighter background for comparison:

[shaoruu]

Hmmm.. I see on your machine the one on the right is also slightly brighter. Could it be that my monitor gets affected more by the Wboit pass? I found out that if I turn the material to double side, it would become even brighter, but when I turn the opacity down a bit it isn't as bright.

[stevinz]

It is strange.

But, the one on the right is brighter because the renderer's write buffer has partially transparent pixels from the WboitPass and the background color is mixing with the renderers output. That's why setting the background to black should fix the issue.

It's possible your monitor is then taking the resulting pixels and applying more correction on them. Even though visually the two objects should look the same on a black background. The one on the left has an alpha value of 1.0, where the one on the right varies, but in that picture has an alpha value of 0.5. In which case for two pixels to look the same with different alpha values the RGB values must be greater, and then in turn get affected more by any correction your monitor is doing after. The point of the sRGBShader I added was to convert the alpha value to 1. I am going back to it and working on some more tests...

[stevinz]

Alright, how does (https://stevinz.github.io/three-wboit/vox/VoxTest.html) look on your machine? Try adjusting the opacity slider as well.

[shaoruu]

Wait it looks exactly the same on both sides! How is this done?

[stevinz]

Well, the blending methods used in wboit create alpha values that are different then standard blending. They seemed to be causing a problem on your monitor (and I'm guessing others would have the same issue).

So before doing gamma correction in the sRGBShader, I am setting the alpha channel to fully opaque (1.0).

I have temporarily put the source on the repo (https://github.com/stevinz/three-wboit/tree/master/example/vox)

[stevinz]

This is not related to the issue, but I wanted to mention it while I was thinking about it. Since you're working with sRGB encoded textures, and then doing some manual rebuilding into a texture atlas and working with THREE.Color, you should let Three.js know or you may end with colors that look off. Somewhere, when you load Three.js, you should include:

THREE.ColorManagement.legacyMode = false;

• Source

After setting legacyMode to false, while using THREE.Color functions (setRGB, getHex, etc.), they will automatically encode / decode values into the sRGB space. You may already be aware of this, just thought I'd mention it.

[shaoruu]

Got it. Thanks so much for the help!