Motivated by the fact that age information can often be observed from local evidence on the human face, we contribute to the age estimation problem in two aspects. On the one hand, we present a new feature descriptor, called spatially flexible patch (SFP), which encodes the local appearance and position information simultaneously. SFP has the potential to alleviate the problem of insufficient samples owing to that SFPs similar in appearance yet slightly different in position can still provide similar confidence for age estimation. One the other hand, the SFP associated with age label is modeled with Gaussian Mixture Model, and then age estimation is conducted by maximizing the sum of likelihoods from all the SFPs associated with the hypothetic age. Experiments are conducted on the YAMAHA database with 8,000 face images and ages ranging from 0 to 93. Compared with the latest reported results, our new algorithm brings encouraging reduction in mean absolute error for age estimation.