Incorrect 2D key point projection when using the original scale of the model

[sriram487]

Describe the issue

Hi,

I am trying to generate dataset using my custom 3D model. when I try to generate dataset with the scaled 3D model which is equivalent to the scale of YCB model, I am able to correctly calculate the 2D projections using 3D key points, rotation matrix, translation vector and intrinsic parameters for each image (Note: I am setting the scale of the model to set_scale(Vector((0.001, 0.001, 0.001))) ) .

But when I use the original scale of the 3D model to generate dataset using the same approach and calculated projected 2D key points have same pixel co-ordinates. I am using 8 key points for my model (Note: Here I am setting the scale to set_scale(Vector((1, 1, 1))).

what might be the issue in the data generation part when the scale of the 3D model changes? what is the unit of the translation vector in both cases ?

Minimal code example

No response

Files required to run the code

No response

Expected behavior

Even when using the original scale of the model, we should get correct projected 2D key points.

BlenderProc version

Github Main Branch

[cornerfarmer]

Hey @sriram487,

could you please provide a minimal code example which reproduces your issue?

[sriram487]

Hi @cornerfarmer ,

Here the code example I used to generated projected 2D key points

def project_keypoints_2D(R, T, K, keypoints):

 
    '''
    K - np.ndarray (3,3) intrinsic parametres
    R - np.ndarray (3,3) rotation_matrix 
    Keypoints - np.array (8,3) Key points 3D
    T - np.array (3,1) Translation vector
    
    o/p : 2D list contaning x and y 2D co-ordinates
    '''

    rep = np.matmul(K, np.matmul(R, keypoints.T) + T)

    x = np.int32(rep[0]/rep[2] + 0.5)
    y = np.int32(rep[1]/rep[2] + 0.5)

    x = x.tolist()
    y = y.tolist()

    points2D = [x, y]

    return points2D

The above method of generating 2D key points works well when we are using bigger scale of the model. But when we are using the original scale of the model all the projected 2D key points have the same pixel co-ordinates.

[cornerfarmer]

The code looks correct. Could you please provide a complete example of how you use this function?