Better approximation for darkening layers under coat

[AdrienHerubel]

Due to the possibility of total internal reflection at the coat/external medium boundary, layers under the coat should be darker. See https://graphics.cs.yale.edu/sites/default/files/wet.pdf for a detailed model describing the various effect of having a water layer on top of materials. I think that interpenetration of layers is out of scope for OpenPBR surface, but the total internal reflection should be modelled.

In the OpenPBR specification we describe the phenomenon without providing implementation details :
In the full light transport this observed color is further darkened and saturated due to multiple internal reflections from the inside of the coat, including a considerable amount of total internal reflection, which causes light to strike the underlying material multiple times and undergo more absorption. Also the observed tint color should vary away from coat_color as the incidence angle changes, due to the change in path length in the medium. The presence of a rough coat will increase the apparent roughness of the BSDF lobes of the underlying base. We generally assume that in the ground truth appearance, all these effects are accounted for.

In Standard Surface there is a dedicated parameter to simulate the darkening of layers under coat using the following formula :
base_color = pow(base_color, 1.0 + (coat * coat_affect_color)) subsurface_color = pow(subsurface_color, 1.0 + (coat * coat_affect_color))
This approximation is simple and disabled by default in Standard Surface.

In OpenPBR we should evaluate it against ground truth, possibly find a better approximation, and decide whether we want to make the effect user controllable.

[portsmouth]

For the moment, I think it's fine to use the crude Standard Surface model, since according to the spec an implementation can approximate the effect however it wants (including just ignoring it..).

I'd like to leave the effect on since the spirit of the spec is to do this physically correctly, in which case there should be at least some darkening/saturation effect. Even with the rough approximation, it will be good to get feedback about this and because (as Indy argued) it probably is generally more useful for the effect to be on than off.

I have a slight preference for using a lower hard-coded value than 1 though (e.g. 0.5), since the effect is quite strong right now, and that may not be realistic.

As noted in the OpenPBR thread, I think we should for a later release derive a better, more physically correct approximation to get more insight into how big this effect actually is. For example if you restrict to normal incidence, I think the full multi-bounce calculation is pretty tractable.

Whether we put in some control later to modulate the effect, is tricky since we would have to give some concrete physical explanation of it. But once we have a better understanding of a formula for the effect, we could implement it for example like:

this coat_darkening_modulation control works (when set to 1) by altering the base albedo (base_color) according to the following formula (which we derive) which approximately makes the apparent color match the input after interaction with the coat. A value of 0 leaves the base albedo untouched, and values in between blend between these extremes.

In essence that is quite similar to the scheme we have for SSS remapping (i.e. set the internal albedo/MFP to approximately match the desired color/blur, given a detailed understanding of the effect).

[jstone-lucasfilm]

@AdrienHerubel @portsmouth I'd support the idea of providing an artistic control over the darkening effect of the coat, as I would expect the real-world impact of this darkening to be highly variable, and influenced by the composition and thickness of the coating material.

Just to give one example of an artistic use case that OpenPBR ought to handle, here's a link to a printing company talking about the effect of different lamination processes on the colors of printed paper:
https://www.mag-data.net/en/how-lamination-affects-the-colour-rendering-of-a-printed-sheet

In the short term, would it perhaps make sense to add a control along the lines of the coat_darkening_modulation that @portsmouth suggests above, so that artists have some degree of control over the current approximation, which can be extended to a more principled approximation in the future?

[portsmouth]

Just to clarify, I think if we did add that control back, we would need to present a specific formula in the spec (for remapping the base_color to achieve an approximate match). As otherwise it would not be clear how to implement it. It would make sense then to try to derive that formula first, before we consider making any alterations to the parameter set.

[AdrienHerubel]

The setup for the following test is a grey diffuse sphere inside an emissive sphere. The middle images are the same scene with a white coat of roughness 0, IOR 1.5 and three values of coat_affect_color. The image on the right is the same scene with a geometrical coat, fully transmissive with roughness 0.

Base only	Base with coat coat_affect_color 0	Base with coat coat_affect_color 0.5	Base with coat coat_affect_color 1	Base with modeled coat

We can see that the simulated coat is darker, and even much darker than the coat_affect_color image.

This is a similar setup with a textured base.

Base with coat coat_affect_color 0	Base with coat coat_affect_color 0.5	Base with coat coat_affect_color 1	Base with modeled coat

This test shows that the effect on color is much more nuanced that the effect coat_affect_color has.

[jstone-lucasfilm]

@AdrienHerubel What are your thoughts on how universal this model for coat darkening should be considered? Would the effect of the lamination process on paper (e.g. a playing card) show such a strong darkening effect as well, or would we expect the composition, application process, and thickness of the coat to have a strong impact on the degree of coat darkening we would see?

[portsmouth]

We assume that the coat is a dielectric layer directly on top of the base. Then the effect described in 2.1 of the paper Adrien linked is the only one that occurs.

That is the effect that Adrien's tests simulate also, by brute force, i.e. bounces in a thin but finite depth coat.

This is actually independent of the thickness of the coat, if the coat is non-absorbing. (If it absorbs, then it depends on the transmittance, not the depth directly). Even a coat of very small thickness will do this (though if the coat depth is so small that wave optics is important, then things will be more complicated -- we can just assume that is not the case).

As noted in the paper, in real materials there is also the possibility that the substrate is wetted by the coat. Which we could attempt to model, but currently don't (and other CG models don't either I think, currently). Similarly, if the lamination involves some adhesive, we won't/can't model the effect of that. (Presumably the layering involved in e.g. MaterialX implicitly makes such an assumption too).

[jstone-lucasfilm]

The main reason that I bring up the potential variability of the effect is make sure we're giving the artist sufficient freedom to control its visual impact, rather than applying a universal darkening to all base colors under a coat. If there's room for different coats to have different darkening effects, then to my mind that supports the importance of an artistic control to determine where and to what degree it's applied. If, on the other hand, we expect a uniform, universal darkening effect, then this artistic control would be less critical.

[portsmouth]

There's no physical reason why it shouldn't darken/saturate according to the process in Fig 1. above. That just always happens, due to light bouncing around in the layer. (Can you think of a physical reason why it should not?).

What I was suggesting was that, as a convenience, we could give the artist a control that will automatically brighten/de-saturate the base color (internally) to compensate, negating the effect. (So it's nothing physical, it's just purely a remapping to save the artist the bother of manually tweaking the base color if she wants to make the appearance match a particular color). This will complicate the model though (add a slider), so we'd want to be sure it's actually worth it.

[AdrienHerubel]

Would the effect of the lamination process on paper

The effect of lamination on the color of paper looks well documented https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9504082/

[jstone-lucasfilm]

@AdrienHerubel Agreed that there's a visual impact of lamination on printed colors, but I'd expect the impact of this effect to be very different from that of water films on surfaces. From our current Slack discussion, it sounds like we're leaning in the direction of expecting coatings to have a highly variable effect, very different from the uniform darkening/squaring in Standard Surface, and let's see how the conversation evolves.

[portsmouth]

it sounds like we're leaning in the direction of expecting coatings to have a highly variable effect

Assuming the mechanism is just light bouncing in the dielectric layer, you would expect the effect to vary according to the coat IOR, roughness and color (i.e. absorption transmittance), in a way which is completely understood and derivable. (Though not very well modeled by the Standard Surface approximation).

[jstone-lucasfilm]

@portsmouth Absolutely, I'm expecting that we'll work through the derivation and come up with a new, physically motivated behavior for base-under-coat effects, which we'll then integrate into an upcoming reference implementation for OpenPBR. Based on its artistic effectiveness, we'll have the choice to enable the new behavior for all OpenPBR materials or to control it with an artist-facing slider, and I'm open to either approach depending on what artists prefer.

In the meantime, though, my hope is that we can remove the legacy Standard Surface behavior (uniform squaring of base-under-coat colors), since it's artistically unintuitive, and doesn't make a good approximation of the new, physically motivated behavior.

What are your thoughts?

[portsmouth]

As noted above:

I'd like to leave the effect on since the spirit of the spec is to do this physically correctly, in which case there should be at least some darkening/saturation effect. Even with the rough approximation, it will be good to get feedback about this and because it probably is generally more useful for the effect to be on than off.

I have a slight preference for using a lower hard-coded value than 1 though (e.g. 0.5), since the effect is quite strong right now, and that may not be realistic.

[jstone-lucasfilm]

@portsmouth I understand your perspective on this, though I feel that our current workaround for base-under-coat effects reduces the quality of the first impressions that artists and developers will have of the OpenPBR shading model

To my eye, having worked with earlier shading models such as Standard Surface and ILM Unified, simply applying a uniform power to base colors under a coat looks like a bug rather than an intentional, physically motivated effect.

I'll leave the associated pull request active for now, and I'd be interested in additional thoughts from developers and artists on this topic:

#135

[anderslanglands]

FWIW from memory at Weta we had a control (on by default) that automatically compensated so the diffuse albedo remained the same at normal incidence.

[portsmouth]

Here is a specific suggestion for how we define the parametrization of the coat darkening.

We say that by default, the physical effect of darkening due to internal reflections in the coat should be accounted for. We give a suggested approximate formula for implementing this (see notes here for the derivation), which is simply to darken the base lobe by the constant factor (different per channel in general):

(where $\eta$ is the coat IOR, $\rho_d$ is the (assumed known) base albedo, and $E_F$ is the average of the coat Fresnel reflectance).

Though implementations are free to improve on this approximation if they wish (as this assumes a Lambertian base, and perfectly smooth coat). The effect of this darkening varies with IOR and underlying base albedo, as shown below (in the approximation above):

To modulate this effect, we add a new control $\delta$ = coat_darkening, which defaults to 1. In this case, the physical darkening effect above occurs, as described. In the case coat_darkening=0, we say that the base albedo is boosted (in each channel) by just enough to counteract the darkening (in the zero absorption case). This is always doable in principle, since as the base albedo $\rightarrow 1$, the total albedo $\rightarrow 1$. Then intermediate coat_darkening values are defined to linearly interpolate the base albedo between these limits.

(NB, for the glossy-diffuse layer, we will do the same except say that this boosting is always done and is not optional, as noted in #141).

To be precise about what we mean by "counteract the darkening", we have to define it concretely in terms of (say) the physical albedo of the coated base at normal incidence, $R_c(1)$. As for the glossy-diffuse case, to be precise we can say that the "undarkened" coat should be such that $R_c(1)$ is as follows (i.e. the albedo-scaling approximation of the undarkened coat):

$$ R_c(1) = \mathrm{lerp}(F_0, 1, \rho_d) = F_0 + (1 - F_0) \rho_d $$

(This is always achievable physically, since varying the physical base albedo $\rho_d$ will cause the true $R_c(1)$ to vary in the range $R_c(1) \in [F_0, 1]$).

We can then solve for the boosted base albedo $B \rho_d$ (with boost factor $B$), such that after the coat darkening is applied, this required $R_c(1)$ is obtained. This boost factor is just the reciprocal of the darkening factor above, i.e. $B = 1/\Delta$ (where in the Lambertian approximation $\Delta$ is as given above, though if we define it as stated, we don't rely on this approximation).

Thus we can define the base albedo to be boosted by the following variable factor $B(\delta)$ (where $\delta$ = coat_darkening):

$$ B(\delta) = \mathrm{lerp}(\frac{1}{\Delta}, 1, \delta) $$

and $\Delta$ (the coat darkening factor) is implementation dependent, but can be reasonably approximated by the formula above).

As a result, in albedo-scaling approximation the total albedo is

$$ F_0 + (1 - F_0) \rho_d \left[(1-\delta) + \delta \Delta\right] $$

which has the expected limits as $\delta \rightarrow 0, 1$.

Note that in the approximation where no darkening is applied (i.e. $\Delta=1$), the boost factor can just be omitted.

If we give suggested approximations such as the coat darkening factor above, we should probably make the derivation publicly available. Options might be to have a folder of supplemental documents, or e.g. add the notes as a separate technical report to https://arxiv.org.

[portsmouth]

This can be closed now.