On-chip normal flow computation with aperture problem compensation circuitry

Extracting relevant visual information about the operating environment of a roving robot at the focal plane is a challenging problem. We present two image processing architectures for motion computation at the focal plane. The first imaging architecture, composed of 250 × 250 active pixel sensors, has spatiotemporal difference computation capabilities at the focal plane. This spatiotemporal difference imager, fabricated in a 0.35μ process, contains in-pixel storage elements for previous and current frames and difference computational units outside the imaging array. A novel scan out technique allows for parallel computation of spatial and temporal 1-D derivatives on the read out. The final motion estimation based on the image brightness constancy equation, which is approximated as the ratio of the temporal and spatial derivates, is computed off chip, but it can be easily implemented on-chip. This approach can only determine the motion component normal to the spatial gradient. In order to address this ill-posed problem, known as the aperture problem, we propose a second imaging architecture, which is an extension of the first imaging architecture. The latter imaging architecture contains in-pixel memory for storing partial computational results. Hence, the motion vectors are stored back into the pixel memories after an initial scan; iterative algorithms can be applied to the stored computational results to solve the aperture problem. Experimental data from both imaging architectures are presented to validate the accuracy of the magnitude and angle of the computed target velocities.