Towards trustworthy AI: Safe-visor architecture for uncertified controllers in stochastic cyber-physical systems

Abolfazl Lavaei, Bingzhuo Zhong, Marco Caccamo, Majid Zamani

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Artificial intelligence-based (a.k.a. AI-based) controllers have received significant attentions in the past few years due to their broad applications in cyber-physical systems (CPSs) to accomplish complex control missions. However, guaranteeing safety and reliability of CPSs equipped with this kind of (uncertified) controllers is currently very challenging, which is of vital importance in many real-life safety-critical applications. To cope with this difficulty, we propose a Safe-visor architecture for sandboxing AI-based controllers in stochastic CPSs. The proposed framework contains (i) a history-based supervisor which checks inputs from the AI-based controller and makes compromise between functionality and safety of the system, and (ii) a safety advisor that provides fallback when the AI-based controller endangers the safety of the system. By employing this architecture, we provide formal probabilistic guarantees on the satisfaction of those classes of safety specifications which can be represented by the accepting languages of deterministic finite automata (DFA), while AI-based controllers can still be employed in the control loop even though they are not reliable.

Original languageEnglish (US)
Title of host publicationProceedings of 2021 Workshop on Computation-Aware Algorithmic Design for Cyber-Physical Systems, CAADCPS 2021
PublisherAssociation for Computing Machinery, Inc
Pages7-8
Number of pages2
ISBN (Electronic)9781450383998
DOIs
StatePublished - May 19 2021
Event021 Workshop on Computation-Aware Algorithmic Design for Cyber-Physical Systems, CAADCPS 2021 - Nashville, United States
Duration: May 18 2021May 21 2021

Publication series

NameProceedings of 2021 Workshop on Computation-Aware Algorithmic Design for Cyber-Physical Systems, CAADCPS 2021

Conference

Conference021 Workshop on Computation-Aware Algorithmic Design for Cyber-Physical Systems, CAADCPS 2021
Country/TerritoryUnited States
CityNashville
Period5/18/215/21/21

Keywords

  • AI-based controllers
  • Artificial intelligence
  • Safe-visor architecture
  • Stochastic cyber-physical systems
  • Trustworthy AI

ASJC Scopus subject areas

  • Computer Networks and Communications

Fingerprint

Dive into the research topics of 'Towards trustworthy AI: Safe-visor architecture for uncertified controllers in stochastic cyber-physical systems'. Together they form a unique fingerprint.

Cite this