Add view count support to Viewport

[BastiaanOlij]

This PR adds the ability to configure the view count (layers) on a Viewport. Before this was only possible inside of the renderer itself through an overrule when use_xr is on.

This allows us to create additional viewports that use the layer system and no longer be restricted to the primary XR output. This enabled:

• Creating an additional stereo viewport for foveated inset rendering.
• Creating an additional stereo viewport for mirror effects.
• Creating an additional stereo viewport to use as a composition layer (render hands and pointers separately so they render above UI composition layers).
  For some of the above we also need camera changes but those our outside of the scope of this PR.

It also moves the XR viewport size overrule out of the render server and into the Viewport class. This is currently a "polling" approach just like it was in the rendering server, the logic is already smart enough not to rebuild the viewport if the size has not changed.

This also has the bonus effect that accurate size information will be available on the Viewport node when XR is used. A limitation of the old approach that caused issues in the past.

Triggering output to the HMD and other overrules such as destination buffer overrides and dynamic resolution changes remain internal to the rendering engine and triggered by the use_xr flag. Some of these we may address at some point in the future but marking a Viewport as the primary XR output will remain.

Note

This extracts some core improvements from other PRs that attempted quad view/inset rendering so they can be reviewed in isolation. We're looking into an improved implementation to foveated inset rendering that will rely on this logic.

[dsnopek]

At a high-level, this looks really great to me! And I did some testing to make sure old code still works with these changes, for both XR output and (non-stereo) composition layers. Seems to work great!

However, it's kind of hard to evaluate if this is sufficient to implement the use cases described above without any way to test them (even partially)

For some of the above we also need camera changes but those are outside of the scope of this PR.

How big are the camera changes? Because I think it would be really nice to have them in this PR, so we could test something (if only verifying a "non-XR" stereo viewport renders correctly in RenderDoc), and see more of the whole picture of what changes will be necessary to actually use this

After that, both the foveated inset and projection composition layer changes definitely make sense in their own individual PRs

UPDATE: I did manage to sort of test it? I set a viewport to have 2 views and render always. When using 2D content, nothing turned up in RenderDoc. But when using 3D content (including a camera, of course), I found the render pass where it does output to texture array, but only the clear color. In both cases, the preview in the Godot editor doesn't work with a view count of 2

[BastiaanOlij]

UPDATE: I did manage to sort of test it? I set a viewport to have 2 views and render always. When using 2D content, nothing turned up in RenderDoc. But when using 3D content (including a camera, of course), I found the render pass where it does output to texture array, but only the clear color. In both cases, the preview in the Godot editor doesn't work with a view count of 2

Correct, we have always disabled 2D rendering if there are multiple layers, assuming stereo projection. 2D rendering only works in very limited situations and would require us to introduce a horizontal offset, though in 3D monitor scenarios it would be nice to support it. The big no-no are slanted displays in XR devices, where its impossible to do traditional 2D output even with an offset applied.

I am planning on working on the camera changes next, but I am planning on keeping that a separate PR (one that will be build ontop of this branch) as I suspect there will be a fair bit of debate around the implementation.

[BastiaanOlij]

@dsnopek further on that remark, I will look at why it doesn't render 3D, probably there is a check that if the view count doesn't match the number of projection matrices it will fail. That may be something that will still need to be part of the camera changes but I did want to make it so that it just uses the same projection matrix and you'll need to deal with it in shaders.

[Calinou]

Out of curiosity, does this make it possible to implement godotengine/godot-proposals#2138 later on?

[BastiaanOlij]

Out of curiosity, does this make it possible to implement godotengine/godot-proposals#2138 later on?

I guess it would be able to use a separate layer for that, but that could also be done by just rendering the gizmo's to a separate texture buffer but re-using the depth buffer, and then mixing it in after effects have been processed.

[marcinn]

also be done by just rendering the gizmo's to a separate texture buffer but re-using the depth buffer

This is something what I am thinking of as an alternative to the separate viewport approach.

and then mixing it in after effects have been processed.

Does this require patching the renderer?

[BastiaanOlij]

Does this require patching the renderer?

Yes, but seeing we identify gizmo's as they are on separate rendering layers (as in Godot render layers, not texture layers), you could make something that filters them out of the normal rendering pass and applies them in a separate rendering pass.

But that is a change that will need to be done during rendering.

[BastiaanOlij]

@dsnopek can we safely make an exception for this one? Validate extension JSON: Error: Field 'classes/Viewport/methods/is_using_xr': is_const changed value in new API, from false to true.

That method should have always been defined as const, not sure if it makes sense to have a compatibility function here? Or just add one to be safe?

[m4gr3d]

The code looks good to me.

But when using 3D content (including a camera, of course), I found the render pass where it does output to texture array, but only the clear color. In both cases, the preview in the Godot editor doesn't work with a view count of 2

Is this a blocker to be addressed in this PR?

[BastiaanOlij]

The code looks good to me.

But when using 3D content (including a camera, of course), I found the render pass where it does output to texture array, but only the clear color. In both cases, the preview in the Godot editor doesn't work with a view count of 2

Is this a blocker to be addressed in this PR?

No, the behaviour is expected. When you enable more than one layer 2D rendering is skipped completely, that has been the case for a long time because 2D rendering isn't possible in many stereoscopic applications. So it's correct that nothing was seen.

If you check out #116424, that introduces camera changes that start to provide actual testable use cases for this change.

[AThousandShips]

LGTM once the final detail is resolved

[Repiteo]

Thanks!