An algorithm and its implementation to compute the surface map of a scene from a stereo pair of mages are described. This algorithm differs from other stereo algorithms in that it uses smoothness of the resulting surface as a criterion for matching. Planar and quadratic patches are used as local models of the suface. A multiresolution hierarchy of surface maps is generated, starting from the coarse and progressing towards the fine. At each stage the surface interpolation process takes into account the detected occluding and ridge contours in the scene, which are places where depth and orientation change abruptly. Interpolation is performed within regions enclosed by these contours. Occluded regions in the image are identified and are not used for matching and interpolation. The algorithm assumes that surfaces in the real world are smooth and continuous except across relatively rare occluding and ridge contours.