Add support for baking positional shadow maps

[jitspoe]

Describe the project you are working on

Retro FPS with realtime per-texel shadows.

Describe the problem or limitation you are having in your project

Hitting a lot of performance issues with dynamic lights, but baked lights don't allow my shader to do the self shadowing effects and some other features I've implemented, and the end result is extremely flat.

Also, single-point lights create very hard details. For example these lights create unnaturally hard shadows on the ground here:

And, finally, baked lighting doesn't let you have cool monster shadows when stuff is lurking around the corner.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Rather than bake the lighting, a shadow for each light would be baked for the static geometry it comes into contact with. A radius could be added to the light to soften the shadows, and real-time shadows of dynamic objects could be combined with the pre-baked shadows. Only the dynamic objects would have to be rendered for the shadow pass, so that should help performance a lot.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

I haven't delved into the details, but it could possibly be implemented something like the shadowmap used for directional lights, except extended to omni/spot lights as a per-light baking option. godotengine/godot#85653

Ideally, it wouldn't be tied to the lightmap GI, though, as I'm not using that and don't want to add any extra data/rendering passes. For my Quake-like aesthetic, directional-only lighting seems to work best for the creepy vibe and some pitch-black shadows.

I'm not sure how feasible this is, as I don't fully understand how the depth maps for shadows and such work and why an atlas is necessary, but my general surface-level idea is this:

• For each light, figure out which polygons of static meshes are impacted by the light and uv unwrap those. Might have to duplicate the mesh data or something if this is to be supported with lightmaps as well since I'm not sure there's something beyond uv2, also several lights might overlap the same polygons.
• Bake line of sight to the light (using a 0-1 range so we can have partial occlusion with a bit of area to the light and also have anti-aliasing) onto either per-light textures or an atlas.
• Dynamic objects rendered to shadow atlas as usual.
• When the shader executes the light() function, it behaves like a normal dynamic light with the baked shadow simply calculated into the ATTENUATION value.

If this enhancement will not be used often, can it be worked around with a few lines of script?

No.

Is there a reason why this should be core and not an add-on in the asset library?

Core rendering feature.

[Calinou]

• Related to Improve shadow rendering performance by rendering static shadows and dynamic shadows separately for each light #4635.

I think you can get most of the performance improvements by separating static and dynamic shadows, without needing to add a lot of complexity to the renderer for something that only some projects will benefit from. In contrast, an automatic solution like #4635 will benefit nearly all projects out there without requiring manual work on the users' end.

PS: Shadows are already cached if no shadow casters within the light's AABB have changed. If you set your dynamic objects' Cast Shadow property to Off, not only the draw call count will be reduced, but omni/spot shadows will also not require any updates unless they shift between quadrants according to the distance between the camera and the light. (You could eliminate quadrant shifts by disabling all quadrants but one, increasing the number of shadows in the first quadrant and increasing the shadow atlas size to compensate.)

And, finally, baked lighting doesn't let you have cool monster shadows when stuff is lurking around the corner.

Remember that baked lighting isn't all or nothing. You can use LightmapGI but keep some lights dynamic when a dramatic effect is needed. This is how most Nightdive re-releases handle lighting.

[jitspoe]

So I went to make some comparison shots to better illustrate my issues with lightmaps:

Realtime:

Baked:

And I noticed a few things:

1. The shadows are so soft they're almost nonexistent. I thought maybe the resolution was too low, so I bumped the texel scale to 4, but it ran out of image space. Not sure if it's a resolution thing or if it is actually using a radius on the light to soften things (and if so, is there a way to tune this?)
2. The light color is different. The baked light is much more yellow. I have 0 bounces, so that shouldn't impact the light color.
3. Since the lightmap isn't exposed to the shader (I don't think, anyway), there's no way to snap to the nearest texel, which looks pretty yucky when paletted:

5. It seems it's still rendering realtime shadows anyway?!

Note that both the book and the static world are casting shadows on the door here. Doesn't this completely defeat the purpose of this, or did I just misconfigure something?

Anyway, as you can see from the first 2 shots, LightmapGI as it currently stands isn't really going to work for this project. Even with the directional option enabled, it doesn't go into the light() shader for self-shadowing and doesn't let me do per-texel snapping.

[jitspoe]

Actually this probably better illustrates the self-shadowing:

And the wall light wasn't casting a shadow because it wasn't set as "static lightmap" (still kind of weird to me that objects must have a light map to cast a shadow on a lightmap). Unfortunately, it didn't respect the transparency in the shader and turned out solid so only the top and a ring on the bottom let light out:

EDIT: Posted the wrong image before.

Interestingly, the lightmap looks plenty high res on that, so I'm not sure why it was so blurry around the bed.

Oh, and I forgot to comment that caching the static geometry shadow depth maps would be great! Being able to add a little softness to the shadows would be cool and all, but avoiding re-rendering the static geo every light update might fix the performance, which is the main concern.

Being able to access the lightmap from the shader might be another viable approach for me.

[Calinou]

The shadows are so soft they're almost nonexistent. I thought maybe the resolution was too low, so I bumped the texel scale to 4, but it ran out of image space. Not sure if it's a resolution thing or if it is actually using a radius on the light to soften things (and if so, is there a way to tune this?)

Bicubic sampling will make shadows softer by design, so I suggest disabling it in the project settings to get sharper shadows.

The light's Size property is used to determine the shadow softness (higher values result in softer shadows), but the shadow Blur property is also used to multiply this value when Size is above 0.0.

The light color is different. The baked light is much more yellow. I have 0 bounces, so that shouldn't impact the light color.

There was an issue in 4.4.beta that resulted in overly saturated static lights, but it should be fixed in 4.4.stable onwards by godotengine/godot#102424. Can you reproduce this in a minimal project?

Since the lightmap isn't exposed to the shader (I don't think, anyway), there's no way to snap to the nearest texel, which looks pretty yucky when paletted:

Shader templates likely address this by letting you override the core shader code without recompiling the engine. It also removes the need to use custom materials for every surface to be affected by texel snapping, so you can keep using BaseMaterial3D and its asset pipeline.

It seems it's still rendering realtime shadows anyway?!

Dynamic objects still receive real-time shadows, since they also receive real-time direct light. This is done because lightmap probes only store indirect light, as direct light is often higher-frequency and benefits from being done in real-time, even when lightmaps are baked on static surfaces. However, this can be too expensive on mobile, which is where #8040 comes handy.

(still kind of weird to me that objects must have a light map to cast a shadow on a lightmap)

• See Meshes without UV2 cannot block light / contribute to LightmapGI godot#96538.

[jitspoe]

Bicubic seemed to make almost no notable difference.

Bicubic on:

Bicubic off:

Turns out it was the denoiser.

Denoiser off, bicubic on:

The light's Size property is used to determine the shadow softness

Ah, perfect! Not sure how I missed that when it was exactly what I was looking for. 😅

There was an issue in 4.4.beta that resulted in overly saturated static lights, but it should be fixed in 4.4.stable onwards by godotengine/godot#102424. Can you reproduce this in a minimal project?

I'll see if I can make a repro case. Pretty sure I synced up right after 4.4 stable, but maybe I'm using something slightly older. v4.5.dev.custom_build [b3942375d] is what the version says.

ffd1a9 is the light color that exhibits the issue.

Edit: Just thinking about this proposal some more, while it would work for static geometry, it probably wouldn't work well for dynamic objects as the light could leak through solid objects, which I guess is already a problem, thus the #8040 proposal. Not sure how this would be addressed. 🤔

Also, looking into the draft PR comments for separating static and dynamic geometry for rendering shadows doesn't look promising. Seems it goes from marginal improvements to making things 30% slower, and the improvements might only be seen on really complex world geometry, which I don't have.
Edit: Draft of the PR is here: godotengine/godot#77683