Restoration and recognition in a loop

In this paper we present a novel learning based method for restoring and recognizing images of digits that have been blurred using an unknown kernel. The novelty of our work is an iterative loop that alternates between recognition and restoration stages. In the restoration stage we model the image as an undirected graphical model over the image patches with the compatibility functions represented as non-parametric kernel densities. Compatibility functions are initially learned using uniform random samples from the training data. We solve the inference problem by an extended version of the non-parametric belief propagation algorithm in which we introduce the notion of partial messages. We close the loop by using the confidence scores of the recognition to non-uniformly sample from the training set in order to retrain the compatibility functions. We show experimental results on synthetic and license plate images.