Perception Contracts for Safety of ML-Enabled Systems

Angello Astorga, Chiao Hsieh, P. Madhusudan, Sayan Mitra

Research output: Contribution to journalArticlepeer-review


We introduce a novel notion of perception contracts to reason about the safety of controllers that interact with an environment using neural perception. Perception contracts capture errors in ground-truth estimations that preserve invariants when systems act upon them. We develop a theory of perception contracts and design symbolic learning algorithms for synthesizing them from a finite set of images. We implement our algorithms and evaluate synthesized perception contracts for two realistic vision-based control systems, a lane tracking system for an electric vehicle and an agricultural robot that follows crop rows. Our evaluation shows that our approach is effective in synthesizing perception contracts and generalizes well when evaluated over test images obtained during runtime monitoring of the systems.

Original languageEnglish (US)
Article number299
JournalProceedings of the ACM on Programming Languages
Issue numberOOPSLA2
StatePublished - Oct 16 2023


  • neural perception
  • perception contracts
  • safety

ASJC Scopus subject areas

  • Software
  • Safety, Risk, Reliability and Quality


Dive into the research topics of 'Perception Contracts for Safety of ML-Enabled Systems'. Together they form a unique fingerprint.

Cite this