Suggestion regarding using a model for object detection and pose estimation

[ArghyaChatterjee]

Hello @TontonTremblay ,

Wanted to have suggestion regarding using a model for my own case. I have been using DOPE and Centerpose for sometime now. Though both of them are good (one for instance and one for category), they need N trained models for N objects. Can you suggest some recent works which have 1 trained model for N objects to detect and to estimate pose of (which has a decent performance)?? I will be using that for manipulation task. Also, an interesting thing would be to see if region segmentation based methods work better than keypoint prediction based methods. What's your take on that ?

Thanks in advance.

[TontonTremblay]

Sadly there is no single solution with everything together. You are going to need to work on connecting detection to pose estimation.

Detection, this is not perfect, but to my knowledge there is not much work in that space. https://github.com/nv-nguyen/cnos

Pose estimation, I think that you can use today, I would go for megapose, https://github.com/megapose6d/megapose6d but there are other solutions available and/or coming up:

• https://github.com/nv-nguyen/gigaPose/
• https://nvlabs.github.io/FoundationPose/
• https://arxiv.org/abs/2311.18809

As for segmentation, I agree with you, specially when mixing diff rendering. https://diffdope.github.io/ it uses segmentation (and other modalities) to do pose refinement. But this assumes you already have a coarse pose estimation.

Overall we are still in need of a solution that does both detection and pose estimation. DOPE is a quite old way to do pose estimation, newer methods are more flexible, but at the same time not as well integrated together -- except if you want to run on BOP. Anyway hope this helps. Please keep me posted on your work.