Regarding estimating Camera matrix K, R and T

[Ram-198]

Hi,

Thank you and the matching results are very good. :)

Earlier i used Sift for feature extraction and cv.detail_BestOf2NearestMatcher() for matching. Then i use cv.detail_HomographyBasedEstimator() to estimate the K, R and T.

How can i achieve this using LightGlue results? Could you please help me with calculating K, R and T from lightglue results?

Thank you

[Phil26AT]

Hi @Ram-198

I am not very familiar with the functions you used, but you could do something similar with pycolmap:

import pycolmap
import cv2

image0 = load_image('path/to/image0.jpg')
image1 = load_image('path/to/image1.jpg')

# run lightglue to obtain m_kpts0, m_kpts1 (see demo notebook)

K0 = pycolmap.infer_camera_from_image('path/to/image0.jpg').calibration_matrix()
K1 = pycolmap.infer_camera_from_image('path/to/image1.jpg').calibration_matrix()

H, inliers = cv2.findHomography(m_kpts0, m_kpts1, cv2.USAC_MAGSAC, 0.5, 0.999, 100000)

ret = pycolmap.homography_decomposition(H, K0, K1, m_kpts0, m_kpts1)

R, t = ret['R'], ret['t']

[aelsaer]

Thank you @Phil26AT for providing this code snippet. I think cv2.findHomography takes 4 arguments while cv2.FindFundamentalMat takes 6. And a minor comment, considering that m_kpts0, m_kpts1 are tensors, they need to be transformed to numpy thus, m_kpts0.numpy() , m_kpts1.numpy() will work. Thanks again @Phil26AT! LightGlue is outstanding.

[yayaYsmile]

Thank you @Phil26AT for providing this code snippet. I think cv2.findHomography takes 4 arguments while cv2.FindFundamentalMat takes 6. And a minor comment, considering that m_kpts0, m_kpts1 are tensors, they need to be transformed to numpy thus, m_kpts0.numpy() , m_kpts1.numpy() will work. Thanks again @Phil26AT! LightGlue is outstanding.

Hi！
When calculating the camera matrix and converting the tensor m_kpts0, m_kpts1 to numpy it seems that the tensor on the GPU can't be converted directly with numpy, it has to be converted to a tensor on the CPU and then to numpy: torch.cpu().numpy(). However this seems to make the computation slow, is there any other way to solve this problem?

[aelsaer]

Hello @yayaYsmile. As far as I'm aware, if you're working with PyTorch tensors, Kornia is a good option since it allows for operations directly on tensors. However, in terms of outlier removal, I believe Kornia primarily offers RANSAC. It's possible they may have added more methods in newer versions, so it might be worth checking their documentation or source code for any updates.

[xl232lx]

Hi @Ram-198

I am not very familiar with the functions you used, but you could do something similar with pycolmap:

import pycolmap
import cv2

image0 = load_image('path/to/image0.jpg')
image1 = load_image('path/to/image1.jpg')

# run lightglue to obtain m_kpts0, m_kpts1 (see demo notebook)

K0 = pycolmap.infer_camera_from_image('path/to/image0.jpg').calibration_matrix()
K1 = pycolmap.infer_camera_from_image('path/to/image1.jpg').calibration_matrix()

H, inliers = cv2.findHomography(m_kpts0, m_kpts1, cv2.USAC_MAGSAC, 0.5, 0.999, 100000)

ret = pycolmap.homography_decomposition(H, K0, K1, m_kpts0, m_kpts1)

R, t = ret['R'], ret['t']

Hello, may I ask, after I install pycolmap, the following error will appear:
AttributeError: module 'pycolmap' has no attribute 'homography_decomposition'
Do you write the homography_decomposition function in ret = pycolmap.homography_decomposition(H, K0, K1, m_kpts0, m_kpts1) yourself?