Low-level multiscale image segmentation and a benchmark for its evaluation

In this paper, we present a segmentation algorithm to detect low-level structure present in images. The algorithm is designed to partition a given image into regions, corresponding to image structures, regardless of their shapes, sizes, and levels of interior homogeneity. We model a region as a connected set of pixels that is surrounded by ramp edge discontinuities where the magnitude of these discontinuities is large compared to the variation inside the region. Each region is associated with a scale that depends upon the fraction of the strong and weak parts of its boundary. Traversing through the range of all possible scales, we obtain all regions present in the image. Regions strictly merge as the scale increases; hence a tree is formed where the root node corresponds to the whole image, nodes closer to the root along a path correspond to larger regions and those further from the root capture smaller regions representing embedded details. To evaluate the accuracy and precision of our algorithm, as well as to compare it to the existing algorithms, we present a new benchmark dataset for low-level image segmentation. We provide evaluation methods for both boundary-based and region-based performance of algorithms. We also annotate different parts of the images that are difficult to segment with the types of segmentation challenges they pose. This enables our benchmark to give an account of which algorithm fails where. We show that our proposed algorithm performs better than the widely used low-level segmentation algorithms on this benchmark.