disabling hardware depth registration leads to error

[130s]

Issue by bit-pirate
Monday Sep 23, 2013 at 02:59 GMT
Originally opened as ros-drivers/openni2_launch#9

---

When I deactivated depth registration via rqt_reconfigure, I get the following error:

[ERROR] [1379904975.770528069]: Depth image frame id [sensor_3d_depth_optical_frame] doesn't match RGB image frame id [sensor_3d_rgb_optical_frame]

Can anyone confirm this? What's different to the old version (openni1)?

[130s]

Comment by bit-pirate
Monday Sep 23, 2013 at 03:05 GMT

---

Maybe @piyushk knows what's going wrong.

[130s]

Comment by piyushk
Monday Sep 23, 2013 at 16:56 GMT

---

@bit-pirate I don't have anything other than a kinect at hand at the moment, so I can't test this directly. Can you try and locate which nodelet is generating this error?

[130s]

Comment by bit-pirate
Tuesday Sep 24, 2013 at 04:15 GMT

---

It's the registration nodelet (PointCloudXyzrgbNodelet) complaining:

Depth image frame id [sensor_3d_depth_optical_frame] doesn't match RGB image frame id [sensor_3d_rgb_optical_frame]

Location:
/tmp/buildd/ros-hydro-depth-image-proc-1.11.2-0precise-20130909-1058/src/nodelets/point_cloud_xyzrgb.cpp:PointCloudXyzrgbNodelet::imageCb:167

Looks to me like the driver is outputting images or point clouds with different reference frames, when not using hardware registration.

[130s]

Comment by piyushk
Tuesday Sep 24, 2013 at 15:32 GMT

---

I suspect the s/w registration pipeline is broken somehow. Once s/w
registered, the depth image should have the RGB image frame id. I'm not
sure how the driver could make this mistake.

On Mon, Sep 23, 2013 at 11:15 PM, Marcus Liebhardt <[email protected]

wrote:

It's the registration nodelet (PointCloudXyzrgbNodelet) complaining:

Depth image frame id [sensor_3d_depth_optical_frame] doesn't match RGB image frame id [sensor_3d_rgb_optical_frame]

Location:
/tmp/buildd/ros-hydro-depth-image-proc-1.11.2-0precise-20130909-1058/src/nodelets/point_cloud_xyzrgb.cpp:PointCloudXyzrgbNodelet::imageCb:167

Looks to me like the driver is outputting images or point clouds with
different reference frames, when not using hardware registration.

—
Reply to this email directly or view it on GitHubhttps://github.com/ros-drivers/openni2_launch/issues/9#issuecomment-24973657
.

[130s]

Comment by mikeferguson
Wednesday Sep 25, 2013 at 18:35 GMT

---

@bit-pirate you mention "disable via rqt" -- does software registration work correctly if you start up in that mode?

[130s]

Comment by liborw
Tuesday Oct 01, 2013 at 08:07 GMT

---

I have the same problem when started without the hardware registration (not using the dynamic reconfigure).

[130s]

Comment by liborw
Tuesday Oct 01, 2013 at 08:46 GMT

---

The problem is, that one must also set hw_registered_processing to false and sw_registered_processing to true along with the depth_registration to false. That way everything works fine.

[130s]

Comment by mikeferguson
Tuesday Oct 01, 2013 at 09:00 GMT

---

@liborw if you leave both pipelines running, do you get a massive number of errors?

If so, I think we can fix that, around here https://github.com/ros-drivers/openni2_camera/blob/hydro-devel/src/openni2_driver.cpp#L451 by moving the publishers into the if/else block. Since pub_depth_ is being remapped to depth_registered right now by device.launch.xml, and pub_depth_raw_ is used by the sw pipeline which apparently includes a convert metric nodelet.

[130s]

Comment by liborw
Tuesday Oct 01, 2013 at 09:23 GMT

---

@mikeferguson I get some errors same are mentioned above when the depth_registration is false, no errors when true. But that way both pipelines are publishing to points topic.

[130s]

Comment by mikeferguson
Tuesday Oct 01, 2013 at 09:51 GMT

---

I've merged #12 as a temporary fix, I'm pretty certain that with a few updates to the drivers we can leave both sw and hw registration running, as only one of depth/image_raw or depth_registered/image_raw will be published. I should be able to find a few minutes to crank that out when I get to the office and have sensors to test with.

[130s]

Comment by achim-k
Thursday May 11, 2017 at 14:54 GMT

---

I think we need an extra publisher to depth_registered/image_raw to solve this problem. Depending whether depth_registration is enabled, the appropriate publisher is chosen:

if (depth_registration_)
{
  // Publish on registered topic: depth_registered/image_raw
  image->header.frame_id = color_frame_id_;
  // ...
} else {
  // Publish on non-registered depth topic: depth/image[_raw]
  image->header.frame_id = depth_frame_id_;
  // ...
}

The remapping in openni2_launch/launch/includes/device.launch.xml would then also not be necessary.

<remap from="$(arg depth)/image" to="$(arg depth_registered)/image_raw" />

At least that worked for me locally. Am I missing something?