camera imaging model

[dzywater]

I am using webots camera to aquire images for camera calibration. Now, I could not find the code of imaging model.

Besides, I would like create own imaging model, how to do it?

[lukicdarkoo]

Are you looking for a calibration matrix?

https://github.com/cyberbotics/webots_ros2/blob/29913d2e14090684915a262b171f86957ca07fad/webots_ros2_core/webots_ros2_core/devices/camera_device.py#L72-L75
You may find this useful.

[dzywater]

Not yet.
I mean the imaging model formula, that is the mathematical expressions.
Typically, the pinhole model without distortions:
u=fx * (x / z) + cx
v=fx * (y / z) + cy
x=R11X + R12Y + R13 * Z + tx
y=R21X + R22Y + R23 * Z + ty
z=R31X + R32Y + R33 * Z + tz

(u, v): pixel coordinates
(x, y, z): coordinate in camera frame
(X, Y, Z): coordinate in world frame
(R11~R33, tx, ty, tz): camera pose in world frame.

I'd like get the implementation like above formulas in webots source codes.
Besides, I'd like create my own imaging model for academic research.
For latest imaging model summary, refer to: "The Double Sphere Camera Model".
Now, I would like implement the imaging models of "The Double Sphere Camera Model" in webots for some tests and verifications.
If possible, I also would like integrate these models into webots.

[omichel]

Webots basically use the a standard OpenGL camera projection for the main 3D view.
The same projection model is also used by default for robot cameras in Webots.
However, it is possible for robot cameras to enable spherical projection to be more realistic. This is especially useful with large field-of-view cameras.
The code for this spherical projection is spread in different classes, but you will find a good starting point here.
Note: we recently found some issue with spherical projections which need to be addressed.

You are very welcome to improve the current model or integrate a better model for camera projections in Webots.
We will be happy to review your contribution.

[dzywater]

Webots basically use the a standard OpenGL camera projection for the main 3D view.
The same projection model is also used by default for robot cameras in Webots.
However, it is possible for robot cameras to enable spherical projection to be more realistic. This is especially useful with large field-of-view cameras.
The code for this spherical projection is spread in different classes, but you will find a good starting point here.
Note: we recently found some issue with spherical projections which need to be addressed.

You are very welcome to improve the current model or integrate a better model for camera projections in Webots.
We will be happy to review your contribution.

I cannot find the mathematical model.
Could you show me the mathematical model in webots sources or in webots documents. Only after I get webots model, I could analyse and locate the problem.
It indeed returns unreasonable intrinsic parameters when I calibrate webots cameras, especially for spherical projection. It confuses me. I don't think there is something wrong with my model. So I cannot wait to know webots models.
Besides, I think that webots may integrate more imaging models which are well-known. These models are useful for computer vision researchers to stimulate real lens and study more reasonable models. I am happy to do that. But I am now stuck.

[omichel]

I am sorry I cannot further help on this. You will have to dive into the code until you find the mathematical model. This takes time, but I we cannot afford to do it for you free-of-charge.

[dzywater]

Thanks, anyway. I'll do it myself.