XFeat + GIM

[leleleoo123]

Hi,

Thanks for your amazing work!

Have you noticed the this GIM work: https://xuelunshen.com/gim/ ? It seems using internet videos to train the feature & matching networks is very powerful.

Do you have any plan to retrain the xfeat using their approach?

Best regards.

[Asherchi]

do you compare the results with GIM and xfeat? which one is better?

[leleleoo123]

do you compare the results with GIM and xfeat? which one is better?

Only compared them on several image pairs, xfeat can't handle large in-plane rotations while GIM can.

[Asherchi]

thanks

[Asherchi]

"We train XFeat in a supervised manner with pixel-level ground truth correspondences", this is from the paper, do you konw how to get the ground truth, and there is no GT from coco datasets.

[guipotje]

Hi @leleleoo123 @Asherchi, thanks for the suggestion! This method looks interesting and could potentially improve XFeat in specific domain targets, especially since it appears you can extract supervision from raw videos without needing to run expensive SfM pipelines. While I do not plan to retrain XFeat using their approach at the moment, any efforts from others would be greatly appreciated!

However, for large rotations, GIM seem to be learning rotation invariance through brute force, which I wouldn't recommend. I suggest considering a more elegant approach by @georg-bn; please check this issue where he integrates steerers with XFeat.

[Asherchi]

"We train XFeat in a supervised manner with pixel-level ground truth correspondences", this is from the paper, do you konw how to get the ground truth, and there is no GT from coco datasets.

solved, the original code is clear.

[noone-code]

@Asherchi

Mismatched: Evaluating the Limits of Image Matching Approaches and Benchmarks

[Asherchi]

got it, tks.