The bad calibration results really frustrate me ;)

[demul]

Hi oliver!
I'm using multical for intrinsic/extrinsic calibration for my camera system.
It consist of 5 cameras.
The rig of my camera system looks like this picture.

When visualizing my result with multical-vis, The relative poses between cameras looks reasonable(looks similar with mechanical design), at least with the naked eye.

But when i try to validate my calibration with the method below, it always give me bad results.
Let me explain my validation method.

1. I triangulate the marker corners with cam1, cam2 stereo pair, and cam3, cam4 stereo pair.
2. and transform them into cam0's coordinate.
3. As a result, I can get aligned point clouds. The point clouds from left stereo pair(cam1, cam2) are marked with red. and The point clouds from right stereo pair are marked with blue.
4. If the calibration was successful, The red and blud point clouds must be aligned well, and must look like that they are on the same plane.

But my result, the point clouds from each stereo pair are not aligned well. they do not look like on the same plane!

The Triangulation Result.

cam0

cam1

cam2

cam3

cam4

And It was noted that the point clouds triangulated from center area of image look clear and planar, but the point clouds triangulated from edge area of image look very very noisy.

You can checkout my triangulation script and the sample data on my github repo
It simply just have dependencies such that "apriltags_eth", "numpy" and "pptk(Point Cloud Visualizer)". I tested it on python3.6

and I'll send you the full calibration data(images) and target configuration to your e-mail.
you can just reproduce it with

1. "multical calibrate --cameras cam0 cam1 cam2 cam3 cam4 --image_path MY_FULL_DATA_PATH --board MY_TARGET_PATH --limit_intrinsic 1000 --limit_images 1000"
2. and try my own validation process i describe above using my github repo
3. you can observe bad calibration results

I carefully researched about this calibration fail case about more than 2 month. and couldn't get any improvement at all. It really frustrate me and It have been driving me mad....! please help me! oliver senpai!