We present a novel scale adaptive, nonparametric approach to clustering point patterns. Clusters are detected by moving all points to their cluster cores using shift vectors. First, we propose a novel scale selection criterion based on local density isotropy which determines the neighborhoods over which the shift vectors are computed. We then construct a directed graph induced by these shift vectors. Clustering is obtained by simulating random walks on this digraph. We also examine the spectral properties of a similarity matrix obtained from the directed graph to obtain a K-way partitioning of the data. Additionally, we use the eigenvector alignment algorithm of  to automatically determine the number of clusters in the dataset. We also compare our approach with supervised and completely unsupervised spectral clustering, normalized cuts, K-Means, and adaptive bandwidth meanshift on MNIST digits, USPS digits and UCI machine learning data.