Simultaneous Calibration of Multiple Cameras and Generation of Omnidirectional Images

Abstract. Omnidirectional images are increasingly being used in various areas, such as urban mapping, virtual reality, agriculture, and robotics. These images can be generated by different acquisition systems, including multi-camera systems, which can acquire higher-resolution images. Stitching techniques are often used and can be suitable for non-metric applications, but rigorous photogrammetric processing is recommended when having more accurate requirements. The main challenges related to this kind of product are the system calibration and the generation of the final omnidirectional images. When using multi-camera systems, the displacement of the cameras' perspective centres can affect the generation of the omnidirectional images and the resulting accuracy. A common approach to minimising the resulting parallax error is to establish a value for the projection cylinder radius as close as possible to the object's depth. This work proposes a highly accurate simultaneous calibration technique for multiple camera systems using self-calibrating bundle adjustment with constraints of stability of the relative orientation parameters. These parameters are later used to generate a projecting cylindrical surface, maintaining the original camera perspective centres and relative orientation angles. The experiments show that using constraints improved both the calibration results and the final omnidirectional images. Residual mismatches between points in overlapping areas are subpixel.