Feat: SDXL Color Compensation

[dunkeroni]

Summary

Adds an option to the SD1.5/SDXL Image-To-Latents node to apply color compensation to the float tensor of the image before encoding. Counteracts brightness drift and haze associated with inpainting, particularly on dark/solid backgrounds.

At the moment, it only applies dynamic range adjustments to all colors at once, as that seemed to be the most reliable method in my testing. It can be revised to adjust colors independently or apply some more elaborate correction method if one is discovered.

For clarity, the option is a Modal of either "None" or "SDXL" on the node. In the Canvas it is a boolean switch with a tooltip to explain its function. The node defaults to "None" and only the SDXL graphs change this.

(Hopefully github image compression doesn't make this impossible to see...)
Following images show the difference after 2 consecutive light inpaint passes on the same pair of seeds. Fixed version on the right.

Prevents build-up of errors after successive passes. The following images show the difference after 6 encodes without any denoise (same mask is used on both images, dots along the right and bottom sides, heart and smiley in the middle).

It does not solve all VAE issues; highly saturated sRGB colors are still outside of the VAE's representative range. It also does poorly at correcting highly saturated shadows, and white values >=250 are made worse. For these reasons, the setting is defaulted to Off for the canvas, though most normal inpainting will typically benefit from it being turned on.

It also does not prevent the SD model from changing an inpaint area away from the surroundings with high enough denoise (~0.6 and up). The intention is to prevent VAE drift on inpaint only and not touch the output side of image generation.

Related Issues / Discussions

Analysis and testing thread: https://discord.com/channels/1020123559063990373/1430012534923989053/1430012534923989053

Other mentions of the effect:
https://discord.com/channels/1020123559063990373/1193632033193336953/1427920397864534169
https://discord.com/channels/1020123559063990373/1149510134058471514/1424466794470445267
https://discord.com/channels/1020123559063990373/1149506274971631688/1344784624068464717

QA Instructions

Lots of pixel peeping, inpainting, and workflows with custom nodes to output graphs for value histogram shifts and trend differences across encode cycles. I gave up on fixing this a year ago because I couldn't find something that worked in enough scenarios. I'm still not fully satisfied with the results here, but it is at least a quantifiable step in the right direction.

Merge Plan

Easy to merge: all settings and code are self-contained and don't interact with other parts of the UI aside from adding the new options.

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Tests added / updated (if applicable)
• ❗Changes to a redux slice have a corresponding migration
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

[lstein]

Tested and works as expected, although to my eye the color changes are very subtle. I had to pass the test image through multiple rounds of encoding and decoding before I could spot visible changes.

I've got a few suggestions for helping the user know not to turn on color compensation for SD-1.5 VAEs, as well as for handling the case that the wrong type of VAE is passed to the encoder/decoder node.

[lstein]

I find it counterintuitive to offer menu items of "None" and "SDXL". Since only SDXL is supported, wouldn't it be better to use a boolean field here labeled "SDXL Color Compensation"?

[dunkeroni]

Partly to telegraph usage, but mostly for future proofing extra profiles/variants. Adding new options to the list will be less disruptive to existing workflow users than changing the input type.

As it is, the "SDXL" option right now tunes for fp16 SDXL 1.0 base VAE, and that mostly (but doesn't exactly) match the Illustrious VAEs. It's easier to leave the space open than to predict against future models with different VAE finetunes. Sounds like Pony V7.1 is going to try for a tuned VAE, but it's unclear if it will remain SDXL architecture or if it will ever be worth officially supporting AuraFlow models anyway.