Should CameraInfo msg be synced with depth and RGB?

[gavanderhoorn]

The current implementation appears to use a separate subscriber for CameraInfo msgs:

industrial_reconstruction/industrial_reconstruction/industrial_reconstruction/industrial_reconstruction.py

Line 131 in 2cf137d

self.info_sub = self.create_subscription(CameraInfo, self.camera_info_topic, self.cameraInfoCallback, 10)

incoming msgs are stored in a member variable in the callback, with a keep-last policy:

industrial_reconstruction/industrial_reconstruction/industrial_reconstruction/industrial_reconstruction.py

Lines 350 to 351 in 2cf137d

	def cameraInfoCallback(self, camera_info):
	self.intrinsics = getIntrinsicsFromMsg(camera_info)

If the assumption is those msgs are all identical, this can work, but technically, it's legal for ROS camera drivers to publish new CameraInfo msgs for every image they publish. Synchronisation would happen based on the header.stamp in both. This allows for settings to change between captures, and consuming nodes to be aware of such changes.

It looks like this should not be too difficult to implement, as message_filters is already used to sync depth and RGB streams. Adding CameraInfo would just require an additional message_filters.Subscriber instance.

[gavanderhoorn]

technically, it's legal for ROS camera drivers to publish new CameraInfo msgs for every image they publish

actually, it'd also be legal for drivers to publish different CameraInfo messages for depth, RGB and other images they publish. A separate CameraInfo topic for each topic carrying Images. That would complicate things a bit more.

It'd be OK if that would not be supported, but I just wanted to point it out.

[marrts]

I like the idea of syncing CameraInfo with the Images based off what you said and also this could also lead to supporting multiple cameras simultaneously.

I also think the TSDF algorithm relies on both depth and color images having the same intrinsic parameters, so a single CameraInfo per camera would be necessary.