Image nonlocal self-similarity has been widely adopted as natural image prior in various low-level vision tasks such as image restoration, while the low-rank matrix recovery theory has been drawing much attention to describe and utilize the image nonlocal self-similarities. However, whether the low-rank prior models exist to characterize the nonlocal self-similarity for a wide range of natural images is not clear yet. In this paper we investigate this issue by evaluating the heavy-tailed distributions of singular values of the matrices of nonlocal similar patches collected from natural images. A novel image prior model, namely nonlocal spectral prior (NSP) model, is then proposed to characterize the singular values of nonlocal similar patches. We consequently apply the NSP model to typical image restoration tasks, including denoising, super-resolution and deblurring, and the experimental results demonstrated the highly competitive performance of NSP in solving these low-level vision problems.