Single-image footstep prediction for versatile legged locomotion

Walking and climbing robots need to plan longterm routes on both horizontal and vertical terrain, but onboard sensors take images from vantage points that provide strongly foreshortened images that cause the appearance of terrain features to vary greatly by distance and viewing angle. This paper presents a convolutional neural network (CNN) method for predicting valid handhold and foothold locations from single RGB+D images taken at arbitrary tilt angles. Experiments show that the method predicts holds more accurately than comparable learning techniques, and that a route planner based on these predictions generates plausible plans for flat ground, stairs, and walls in rock climbing gyms.