Feature guided pixel matching and segmentation in motion image sequences

The problem of feature correspondences and trajectory finding for a long image sequence has received considerable attention. Most attempts involve small numbers of features and make restrictive assumptions such as the visibility of features in all the frames. In this paper, a coarse-to-fine algorithm is described to obtain pixel trajectories through the sequence and to segment into subsets corresponding to distinctly moving objects. The algorithm uses a coarse scale point feature detector to form a 3-D dot pattern in the spatio-temporal space. The trajectories are extracted as 3-D curves formed by the points using perceptual grouping. Increasingly dense correspondences are obtained iteratively from the sparse feature trajectories. At the finest level, matching of all pixels is done using intensity correlation and the finest boundaries of the moving objects are obtained.