Add custom camera scale to rendering server

[huwpascoe]

Partially Implement godotengine/godot-proposals#2713, godotengine/godot-proposals#11436

Effectively allows the user to override the aspect ratio.

Example usage:

extends Camera3D

func _ready() -> void:
	# set oblique projection (camera should be rotated -45 degrees x axis)
	RenderingServer.camera_set_scale(get_camera_rid(), Vector2(1.0, sqrt(2.0))

• Production Edit: This closes Allow for setting the X and Y FOV/frustrum sizes independently, and/or modifying the view transform godot-proposals#1250.

[JNSStudios]

This was a lifesaver in getting the "tile-based" perspective I wanted in my game! Thank you so much for getting this working. I hope that this becomes a main feature in the engine so I can work on the main releases instead of a pull request version.

[JNSStudios]

I've come across a problem with this PR.

I added a custom cursor object to my game, and it uses a camera raycast to determine what it's hovering over.

When in regular 4.4, this is what it looks like. The cursor animation is triggered when the raycast finds a different object.

In this PR-compiled version, the animation is triggered at a point that appears visually incorrect.

Here is the code I wrote for the raycast.

    var mouse_pos = get_viewport().get_mouse_position()
    position = mouse_pos
    var from = camera.project_ray_origin(mouse_pos)
    var to = from + camera.project_ray_normal(mouse_pos) * 1000

    var space_state = get_viewport().world_3d.direct_space_state
    var query = PhysicsRayQueryParameters3D.create(from, to)
    var result = space_state.intersect_ray(query)

I quickly added a Sprite3D that constantly moves with the real mouse cursor's position to illustrate the raycast difference between the CanvasLayer UI element vs the 3D world camera raycast.

[huwpascoe]

Here's a quick reimplementation of the project_ray_origin (as close as I could get it with the exposed interface). Only difference is the y-basis is divided by sqrt(2).

Source

extends Camera3D

func project_ray_origin_modified(p_pos: Vector2) -> Vector3:
	var vp := get_viewport()
	var viewport_size := vp.get_visible_rect().size
	var cpos := (
		(vp.get_stretch_transform() * vp.global_canvas_transform) * p_pos
	)

	var pos := cpos / viewport_size
	var vsize: float
	var hsize: float

	if keep_aspect == Camera3D.KEEP_WIDTH:
		vsize = size / viewport_size.aspect()
		hsize = size
	else:
		hsize = size * viewport_size.aspect()
		vsize = size

	var ray: Vector3
	ray.x = pos.x * (hsize)-hsize / 2;
	ray.y = (1.0 - pos.y) * (vsize)-vsize / 2;
	ray.z = -near

	var tr := global_transform

	tr.basis[1] /= sqrt(2.0)
	tr.origin += tr.basis[1] * v_offset
	tr.origin += tr.basis[0] * h_offset

	ray = tr * ray

	return ray

[JNSStudios]

I think for a long-term solution, raycasting methods like this should operate based on the camera projection matrix in some way. While the method you posted worked perfectly for my project, I think a more future-proofed version of this method would use the custom matrix to determine how to cast the ray through the camera.

[Flarkk]

Many pieces of code across physics (picking) and rendering (culling, effects ...) completely bypass the projection matrix to avoid unnecessary matrix multiplications or inverse computations :

• Sometimes the user-provided camera attributes are used directly instead (typically zfar, or the projection mode PERSPECTIVE / ORTHOGONAL / FRUSTUM).
• Sometimes assumptions are made on the value and/or specific relations among certain components of the matrix and they get completely ignored.

Enforcing a user-defined matrix like this PR does just overlooks this complexity.
This will have material performance and usability impacts.
The issue reported by @JNSStudios in #103553 (comment) is a good illustration.

I see only 2 valid paths towards properly supporting custom matrices :

1. Make sure all pieces of code related to projection across picking and rendering code support custom matrices. In most cases I believe it will require specific code paths, which can add some overhead speaking of shaders. That's not unfeasible but requires a material amount of work
2. Do not expose the raw matrix to users, but instead expose an interface that ensures all the assumptions and constraints are met in the first place when the user updates the camera parameters. It does not offer as much freedom as the other option but I believe yet a good balance can be achieved. This is what Custom Camera3D projection: a practical approach godot-proposals#11436 is all about

So to wrap it up, I'm afraid this PR is not that different from the earlier attempts (#85529, #84454).
It partially implements approach 1 (godotengine/godot-proposals#2713).
It does not implement approach 2 (godotengine/godot-proposals#11436).

[JNSStudios]

Well, this was made by huwpascoe in an attempt to adapt older attempts at this system to the new version of the engine. I was asking about how to make this kind of camera perspective in the Godot discord server and people suggested that I could use one of the PRs you listed, but I expressed concern with the PRs being outdated for the new engine version. Huwpascoe then sent me the link to this PR after I said that.

So, at the very least, we showed that demand for the features proposed in those more thorough PRs does still exist, in a way.

I do hope that this gets added in some capacity. I have tried numerous other methods to create the “Faux Tile” appearance shown in the screenshots, all of them having other problems that made them unviable for my vision. This method has been the most successful attempt yet.

[huwpascoe]

@Flarkk Yeah I agree. Next week or sooner I'll be rebasing with something as close to your proposal as possible.

Just need to round up all the different viewport size functions, as they're kinda all over the place...

[huwpascoe]

Custom aspect is now possible, no projection matrix required.

[Calinou]

Tested locally, it works as expected with all camera modes (perspective, orthogonal, frustum).

Testing project: test_pr_103553.zip

Note that this can only be used from RenderingServer, not the Camera3D node properties. The Camera3D node's scale is not automatically applied to affect the aspect ratio, which is probably a good thing here as we wouldn't want parent node scaling to affect the node's aspect ratio.

Note for reviewers: Comments at the top refer to a previous state of this PR where it allowed using arbitrary custom projection matrices. This PR now only allows scaling the Camera3D node's projection matrix to change the aspect ratio it displays, which can be used for various distortion effects. This can also be used for non-square pixel ratios while still rendering at native resolution. However, it does not suffice on its own to implement portals or planar reflections.

[marleyinbloom]

This is exactly what I need for my game project as well, I'm glad to see it being worked on! Will look into building from the PR to double check that it works with my project but im pretty sure this is what I need.

[acatxnamedvirtue]

Hi folks! This has now been sitting for 3 months. Unless this actively breaks another workflow, I think many of us would agree that this PR, which provides a solution to a problem many of us making retro-inspired games have, is better than a different approach that a contributor has not yet had time to explore. I appreciate that I have been able to custom compile with these changes, but it would sure be nice to just have this in the engine, and be able to use the precompiled, optimized builds as they are released.

Could we move forward with a proper review to vet this approach?

[TheMaydayMan]

I'm excited for this to eventually be merged into release, but the mouse issues probably need to be fixed if they haven't already (I custom built a couple months ago so idk what's changed since then)

[jktrctt]

FWIW, I have also been using this solution for my game because it solves a basic problem with the 3D to fake 2D workflow. Mouse picking is pretty basic to fix on your end as well. If this solution isn’t the exact one, I get it, but the thing this PR addresses is completely necessary in my view. I’m three months into depending on it, assuming that something that accomplishes this goal is somewhere on the horizon.

[TheMaydayMan]

Completely agree and I’m in a similar situation, I don’t want the mouse thing to stop this from being merged but I hope it ends up being addressed eventually

[huwpascoe]

This feature won't make the upcoming version regardless. However I'll provide rebased PR (which also generates optimized builds) so it can still be used like any official release.

I won't let it be forgotten.

[TheMaydayMan]

@huwpascoe Currently all the artifacts of your previous editor and template workflow runs are expired. Could you run those again? I can't get it to work by forking your repo. Thanks!

[AThousandShips]

Running again would require pushing again, it can't be rerun, it'd probably be easier to build directly or clone the branch and run it on your own fork

[TheMaydayMan]

What do you mean by build directly?

[AThousandShips]

See here

[huwpascoe]

I'll do a rebase soon, it'll be for 4.5.

[blueloveTH]

I need this feature! Really hope it can be merged into Godot 4.5.

[QbieShay]

@blueloveTH 4.5 is in feature freeze since a while. No new features will be merged, sorry!

[spoxii]

I think an editor viewport setting for the camera scale would be very helpful for level designers.

I am currently switching to the "Top Orthogonal" view and press the "2" key three times to rotate the viewport by 45°, but without the sqrt(2) scale for the y-axis the graphics are squashed. This makes it hard for level designers, because the view of the level in the editor does not match the view of the level in-game.

Current	Desired

Just a mockup, obviously this should be a Vector2 and not a single number, but you get the idea:

[spoxii]

While implementing a viewport setting as described above, I have noticed that the selection cursor from the GridMap drifts away from the origin because the mouse picking is not corrected for the scale. Maybe we can adapt commit b7a8cb3 to this PR?

godot.windows.editor.dev.x86_64_LHGk9mxCKR_CUT.mp4

Also note that setting the camera scale to 0.0 spams the console with errors each frame:

ERROR: Condition "0.0 == r0[0]" is true.
at: Projection::invert (core\math\projection.cpp:616)

[JNSStudios]

While implementing a viewport setting as described above, I have noticed that the selection cursor from the GridMap drifts away from the origin because the mouse picking is not corrected for the scale. Maybe we can adapt commit b7a8cb3 to this PR?
godot.windows.editor.dev.x86_64_LHGk9mxCKR_CUT.mp4

Also note that setting the camera scale to 0.0 spams the console with errors each frame:

ERROR: Condition "0.0 == r0[0]" is true.
at: Projection::invert (core\math\projection.cpp:616)

I can confirm that this issue applies when running the engine as well, as I noted here:
#103553 (comment)

[spoxii]

I've adjusted Camera3D::project_local_ray_normal and Camera3D::project_ray_origin in my fork to properly account for the camera's scale. With these changes, mouse picking is working again.

However, these changes require a getter for the camera_scale - which is currently missing in this PR. How are you supposed to get the current camera scale? I hacked together a RenderingServer::camera_get_scale(rid), but I feel like this should be an exposed property instead. After all, if we follow the practical approach by Flarkk, we'll have several user-configurable properties for the projection of a Camera3D. The scale could be one of them.

Is there a list of effects we should test when dealing with modified projection matrices?
For instance, I tested this PR with a VisibleOnScreenNotifier3D and a scaled camera to verify that the signals are emitted as expected. A test project with such test cases would be very helpful for future PRs I think.

[eezstreet]

What’s the latest on this?

[huwpascoe]

The current status is I've submitted the more generic version as requested
#110850

It's done afaik.

[JNSStudios]

Just wanted to chime in here real quick: I was able to port this change over to the 4.6-stable branch with no immediate issues that I could tell.

8mb.video-F5K-uauBzLVJ.mp4

I did this because I've been using this PR for my game project to achieve the "top-down" camera view vital to my game's intended style, but I needed to update to the new version to access a new feature. I don't know how easily I could push the change here to catch this up, but it isn't too difficult to cherry-pick this PR into 4.6-stable and compile it that way.

[huwpascoe]

Bastiaan's implementing the feature in a new PR if you want to follow progress. It's primarily focused on VR, but this feature is a part of it. Once it's out of draft, it should be easy enough to try the custom ortho protection.

It'll probably be some months yet before it's ready.
#116424

[blueloveTH]

I like this pr. It is simple and stable, really fits my need. Thanks @huwpascoe