Implementation of Zhang's Camera Calibration algorithm to estimate a camera's extrinsic parameter matrices "R", "t" and intrinsic parameter matrix "K". It uses a Calibration pattern of checkerboard to estimate these parameters
0. Generate dataset of images by using the Calibration Pattern in different views
1. Generate Hough lines and separate horizontal from vertical
2. From the hough lines, get rid of duplicate lines and then find intersection of horizontal and vertical lines to get corners corrdinates on image plane
3. Generate world coordinates of the corresponding corners in the image plane
4. Find Homography H between world plane Z=0 and image plane
5. Using this homography and the image of absolute conic, solve equations to estimate the image (omega) of the absolute conic.
6. Using the relationship between omega and K, obtain the intrinsic parameter matrix K. This is a estimate and will be refined further
7. Now, build an initial estimate of R and t matrices using K and H
8. Condition matrix R to be orthonormal
9. Convert the matrix R to a vector R_vec using Rodrigues formula
10. Build initial estimate of all the parameters, 5 for K, 3*no_of_images in dataset for R and 3*no_of_images_in_dataset for t
11. If incorporating radial distortion, the also include parameters k1 and k2
12. Optimize the geometric error between the actual corners and projected coordinates of world coordinates in the image plane using Levenberg - Marquardt algorithm
13. Rebuild the matrices R, t for each image from the 1D vector obtained after optimization. Note that the R_vect should be converted to R matrix using Rodrigues formula
14. Rebuild estimated K matrix from 1D vector after optimization
Original Paper: Zhang_technical_report.pdf
Refer here for the mathematical formulation and technical explanation
- OpenCV
- NumPy
- SciPy (for Levenberg-Marquardt implementation)
- calibrate_camera.py: MAIN file to run. Pass the correct parent folder and image list in the script. Result folder with all intermediate images are generated in /<parent_folder>/results_<fixed_img_no>
python calibrate_camera.py- compute_corners.py: Script to get coordinates of corners by intersection of lines obtained using Hough transform
- estimate_homography.py: Helper functions which help in bilinear interpolation and projecting images to a canvas using Homography matrix
Pic_28.jpg
Pic_40.jpg
The world coordinates are taken as [0, 0, 0, 0] for the point marked "0" in the above image and increased by 25 and 25 in x, y directions for subsequent labelled points in the above image. That is, "0": [0, 0, 0, 0], "1": [25, 0, 0, 0], "8": [0, 25, 0, 0] and so on.
Camera Intrinsic Parameters (K) and Extrinsic Parameters ([R|t]) before and after Levenberg Marquardt optimization - Without Radial Distortion incorportated
Camera Intrinsic Parameters (K) and Extrinsic Parameters ([R|t]) before and after Levenberg Marquardt optimization - WITH Radial Distortion incorportated
Projecting the world coordinates of Pic_28.jpg using the calculated camera matrix. The left image is before the LM optimization for refinement of parameters.
The right image is after the refinement of parameters. Blue are the estimated projection of world coordinates in image plane using camera matrix.
Green are the ground truth image plane projections of world coordinates.
Projecting image coordinate estimation of world coordinates using Pic_40.jpg camera matrix onto Pic_28.jpg
Projecting the world coordinates of Pic_28.jpg using the calculated camera matrix. The left image is before the LM optimization for refinement of parameters.
The right image is after the refinement of parameters. Blue are the estimated projection of world coordinates in image plane using camera matrix.
Green are the ground truth image plane projections of world coordinates.
Projecting image coordinate estimation of world coordinates using Pic_40.jpg camera matrix onto Pic_28.jpg
