Lyapunov Perception Contracts for Operating Design Domains

There are two barriers to assessing the reliability of visual control systems that use machine learning (ML) models for perception and state estimation. First, the reasoning has to include the image rendering process, which is affected by environmental factors such as lighting and weather in complex ways. Second, we lack meaningful specifications for ML models like deep neural networks (DNNs). In this paper, we introduce Lyapunov Perception Contracts (LPC) as a method to address these challenges. We show how these contracts can be used as specifications for DNN-based state estimators, which assure closed-loop stability. We propose a method for synthesizing LPC from data and the models for the controller and plant dynamics. We also show how LPCs can be used to find operating design domains for visual controllers that operate in finitely parameterized environments. We illustrate applications of this method in a visual automated landing system using image data from both rendered simulations and Google Earth.