Using LoFTR keypoints ?

[tcourat]

Would it be possible to use lightglue in order to match keypoints obtained from LoFTR (or other semi-dense/dense matchers, not only sparse) ?

I know they already provided the matches and not only the detection, but there are still wrongly matched points in most cases and maybe LightGlue could filter them.

[Phil26AT]

Hey, we have never tried this but it might be possible. Given the keypoints from both images, You would need to interpolate descriptors from a dense descriptormap (e.g. the one from SuperPoint). Then you can run LightGlue. Please let me know how well this works, I am also curious :)

[tcourat]

I have tried on a pair of images to interpolate the SP descriptors at LoFTR keypoint location. Seems to work decently
I adapted the demo notebook.

From SP+Lightglue :

From LoFTR + Lightglue (interpolation) :

Can be downloaded there :
demo_loftr_sp.zip

I guess re-training with LoFTR kpts would lead to even better results, but if we can use lightglue to filter the outliers from SPpoint its already a good thing (I noticed that loftr can often produce some wrong matches).