Non-frontal camera calibration using focal stack imagery

A non-frontal camera has its lens and sensor plane misaligned either due to manufacturing limitations or an intentional tilting as in tilt-shift cameras. Under ideal perspective imaging, a geometric calibration of tilt is impossible as tilt parameters are correlated with the principal point location parameter. In other words, there are infinite combinations of principal point and sensor tilt parameters such that the perspective imaging equations are satisfied equally well. Previously, the non-frontal calibration problem (including sensor tilt estimation) has been solved by introducing constraints to align the principal point with the center of radial distortion. In this paper, we propose an additional constraint which incorporates image blur/defocus present in non-frontal camera images into the calibration framework. Specifically, it has earlier been shown that a non-frontal camera rotating about its center of projection captures images with varying focus. This stack of images is referred to as a focal stack. Given a focal stack of a known checkerboard (CB) pattern captured from a non-frontal camera, we combine geometric re-projection error and image bur error computed from current estimate of sensor tilt as the calibration optimization criteria. We show that the combined technique outperforms geometry-only methods while also additionally yielding blur kernel estimates at CB corners.