OBJ / MTLLoader: Label and convert textures and colors correctly in MTLLoader

[gkjohnson]

Related issue: #23283

Description

Convert colors to Linear and label color textures as sRGB when loading MTL materials. Also update the OBJ demo with MTL files to use outputEncoding = sRGBEncoding.

https://raw.githack.com/gkjohnson/three.js/srgb-obj/examples/webgl_loader_obj_mtl.html

[gkjohnson]

Looks like with linear lighting the specular look changes quite a bit in the face especially. I'm not sure if this would be considered an issue?

BEFORE	AFTER

[Mugen87]

Please be aware of #7290.

Since three.js does not interpret specular maps as color maps so far (they are in fact reflectivityMap maps), it's not right to configure them with sRGBEncoding. MeshPhysicalMaterial.specularColorMap is the only exception though.

[gkjohnson]

Yes specular maps aren't adjusted here. Only the specular color is.

[mrdoob]

Interesting. I always wondered why it was so plasticky...

Could you re-generate the webgl_loader_obj_mtl screenshot too?

[gkjohnson]

Could you re-generate the webgl_loader_obj_mtl screenshot too?

Hmm. I ran npm run make-screenshot -- webgl_loader_obj_mtl and the screenshot looks like this:

I'm running on Windows 10. Is there anything else I need to do?

[Mugen87]

Does it work if you navigate into the test directory and execute npm i once? Maybe dependencies are out of date.

[gkjohnson]

Does it work if you navigate into the test directory and execute npm i once?

Thanks but still no luck. Looks the same. Could the change in texture format be causing this? When I revert the texture.encoding = sRGBEncoding the texture displays (though it's bright of course due to the renderer output encoding)

[mrdoob]

Ah... Seems like the issue is that SwiftShader doesn't support WEBGL_compressed_texture_s3tc_srgb...

I checked out your branch and changed the puppeteer script to not be headless and I found this in devtools:

@Mugen87 Any suggestions? Would it be better if the renderer was tolerant about this and rendered these textures wrong, better than failing and rendering black?

[Mugen87]

Would it be better if the renderer was tolerant about this and rendered these textures wrong, better than failing and rendering black?

I would not do that. Displaying the uncompressed data means undefined result. What the engine currently does (reporting errors/warning) is the best.

BTW: I can generate the screenshot on my iMac without issues. This is the result:

[mrdoob]

BTW: I can generate the screenshot on my iMac without issues. This is the result:

Oh... The we can merge and you can generate the new screenshot.

But we'll probably have to add an exception in the e2e script:

three.js/test/e2e/puppeteer.js

Lines 25 to 55 in be9d017

	const exceptionList = [
	'index',
	'css3d_youtube', // video tag not deterministic enough
	'webaudio_visualizer', // audio can't be analyzed without proper audio hook
	'webgl_loader_imagebitmap', // takes too long to load?
	'webgl_loader_texture_lottie', // not sure why this fails
	'webgl_loader_texture_pvrtc', // not supported in CI, useless
	'webgl_materials_standard_nodes', // puppeteer does not support import maps yet
	'webgl_morphtargets_face', // To investigate...
	'webgl_postprocessing_crossfade', // fails for some misterious reason
	'webgl_raymarching_reflect', // exception for Github Actions
	'webgl_test_memory2', // gives fatal error in puppeteer
	'webgl_tiled_forward', // exception for Github Actions
	'webgl_video_kinect', // video tag not deterministic enough
	'webgl_video_panorama_equirectangular', // video tag not deterministic enough?
	'webgl_worker_offscreencanvas', // in a worker, not robust
	// webxr
	'webxr_ar_lighting',
	// webgpu
	'webgpu_compute',
	'webgpu_instance_uniform',
	'webgpu_lights_custom',
	'webgpu_lights_selective',
	'webgpu_materials',
	'webgpu_nodes_playground',
	'webgpu_rtt',
	'webgpu_sandbox',
	'webgpu_skinning_points',
	'webgpu_skinning'
	].concat( ( process.platform === 'win32' ) ? [

[mrdoob]

Actually, I can just photoshop/gimp it 🤓

I'll take care of this.

[mrdoob]

Thanks!

[mrdoob]

Actually, a better solution would be to remove the dds textures from this example. Will do that.

[Mugen87]

Actually, a better solution would be to remove the dds textures from this example.

Indeed! Let's focus on S3TC in the webgl_loader_texture_dds example.

[Mugen87]

BTW: OBJLoader has vertex color support.

three.js/examples/jsm/loaders/OBJLoader.js

Line 754 in c77a176

buffergeometry.setAttribute( 'color', new Float32BufferAttribute( geometry.colors, 3 ) );

It seems this change is only consistent if the attribute is converted to sRGB, too.

[gkjohnson]

Oh oops -- you're right. My understanding is that the vertex colors of a geometry need to be in Linear color since they undergo no transformation before or in the shader. I'll make a PR in a bit.

[spencer-sherk-ai]

gkjohnson Just a little curious here for my own understanding -- why do we need to convert the colors from SRGB to linear, in order to be compatible with a renderer using SRGB encoding? This seems counter-intuitive to me ( I would think this would happen the other way around ) but probably just some knowledge gap in my understanding about the lower level of renderer color encoding.

[gkjohnson]

why do we need to convert the colors from SRGB to linear, in order to be compatible with a renderer using SRGB encoding

Only the final image is output and displayed in sRGB. All of the math being done on the GPIU happens in Linear sRGB color space.