Adding new cameras

[reynoldscem-oculo]

Hi, thanks for the great work. I am curious about adding a new camera model (higher order distortion terms). There are a few parts in the code I'm not particularly sure of. Would you be able to shed some light on the steps needed to add a new model? E.g. using more radial distortion parameters

[sarlinpe]

To add a new camera model, you need to create a new Camera class by subclassing BaseCamera, as we do here:

GeoCalib/geocalib/camera.py

Lines 539 to 547 in a7ef2cf

	class SimpleRadial(BaseCamera):
	"""Implementation of the simple radial camera model.
	Use this model for weakly distorted images.
	The distortion model is 1 + k1 * r^2 where r^2 = x^2 + y^2.
	The undistortion model is 1 - k1 * r^2 estimated as in
	"An Exact Formula for Calculating Inverse Radial Lens Distortions" by Pierre Drap.
	"""

Then you need to register it:

GeoCalib/geocalib/camera.py

Lines 770 to 774 in a7ef2cf

	camera_models = {
	"pinhole": Pinhole,
	"simple_radial": SimpleRadial,
	"simple_divisional": SimpleDivisional,
	}

The camera needs to implement:

• the distortion and undistortion functions as well as their jacobians. We only used invertible camera models, which have an analytical undistortion. If not, this gets tricky - you would need to implement a fixed-point undistortion (with its jacobian, which has an analytical form, as shown in DEQ) or use approximations of Drap & Lefèvre.
• the jacobian of the projection of the up-vector (J_up_projection_offset) - we derived this by hand (sections D & E in the paper) but this could be done automatically with sympy or autodiff.

[reynoldscem-oculo]

@sarlinpe perfect, thank you

[suvigy]

• the distortion and undistortion functions as well as their jacobians. We only used invertible camera models, which have an analytical undistortion. If not, this gets tricky - you would need to implement a fixed-point undistortion (with its jacobian, which has an analytical form, as shown in DEQ) or use approximations of Drap & Lefèvre.
• the jacobian of the projection of the up-vector (J_up_projection_offset) - we derived this by hand (sections D & E in the paper) but this could be done automatically with sympy or autodiff.

Thanks for the great work, I really appreciate it. I also plan to implement a custom camera model with more distorted coeffs (with approximated undistortion).
Question or maybe I would ask for some hints how to technically handle multiple coeffs in the Jacobians so that other codeparts effectively can use it. Is it enough to create a new custom camera model with the mentioned projection, unprojection, jacobians or does other part of the code have to be modified in case of opencv like distortion model (I have more than 4 coeff params)?

If I implement the mentioned jacobians wrt. 'dist', is it sufficient to return the jac vector for all coeffs (k1, .....kn) as vector? Or should we handle them separately?
As I see in the perspective_field.py there is a comment K1 Jacobian Suggesting that maybe this part of the code should also be adapted if we implement the coeffs separately (different wrt values). But I can imagine, this part of code handles also vectors.

In both SimpleRadial and SimpleDivisional there is only one k1 for distortion param, so maybe other parts of the code use this fact, that's why I'm a bit uncertain if it is sufficient to implement a new cameramodel.

[veichta]

The current implementation primarily assumes a single-parameter distortion model. However, most parts of the code should generalize well to higher-order distortion models that follow the form: D([u, v], k) = d(u, v, k) * [u, v] (i.e., higher-order radial distortion).

In this case, the Jacobians should work correctly if you return the Jacobian vector for all coefficients when implementing wrt=dist.

That said, there are a few areas that may require adjustments for higher-order distortions, such as this section. These should be relatively straightforward to identify and modify once the new camera model is in place.