ML-Based Fault Injection for Autonomous Vehicles: A Case for Bayesian Fault Injection

Saurabh Jha, Subho Banerjee, Timothy Tsai, Siva K.S. Hari, Michael B. Sullivan, Zbigniew T Kalbarczyk, Stephen W. Keckler, Ravishankar K Iyer

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The safety and resilience of fully autonomous vehicles (AVs) are of significant concern, as exemplified by several headline-making accidents. While AV development today involves verification, validation, and testing, end-To-end assessment of AV systems under accidental faults in realistic driving scenarios has been largely unexplored. This paper presents DriveFI, a machine learning-based fault injection engine, which can mine situations and faults that maximally impact AV safety, as demonstrated on two industry-grade AV technology stacks (from NVIDIA and Baidu). For example, DriveFI found 561 safety-critical faults in less than 4 hours. In comparison, random injection experiments executed over several weeks could not find any safety-critical faults.

Original languageEnglish (US)
Title of host publicationProceedings - 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages112-124
Number of pages13
ISBN (Electronic)9781728100562
DOIs
StatePublished - Jun 2019
Event49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2019 - Portland, United States
Duration: Jun 24 2019Jun 27 2019

Publication series

NameProceedings - 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2019

Conference

Conference49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2019
CountryUnited States
CityPortland
Period6/24/196/27/19

Fingerprint

Learning systems
Accidents
Engines
Testing
Industry
Experiments

Keywords

  • Autonomous Vehicles
  • Fault Injection
  • Machine Learning

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Safety, Risk, Reliability and Quality
  • Hardware and Architecture

Cite this

Jha, S., Banerjee, S., Tsai, T., Hari, S. K. S., Sullivan, M. B., Kalbarczyk, Z. T., ... Iyer, R. K. (2019). ML-Based Fault Injection for Autonomous Vehicles: A Case for Bayesian Fault Injection. In Proceedings - 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2019 (pp. 112-124). [8809495] (Proceedings - 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/DSN.2019.00025

ML-Based Fault Injection for Autonomous Vehicles : A Case for Bayesian Fault Injection. / Jha, Saurabh; Banerjee, Subho; Tsai, Timothy; Hari, Siva K.S.; Sullivan, Michael B.; Kalbarczyk, Zbigniew T; Keckler, Stephen W.; Iyer, Ravishankar K.

Proceedings - 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 112-124 8809495 (Proceedings - 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Jha, S, Banerjee, S, Tsai, T, Hari, SKS, Sullivan, MB, Kalbarczyk, ZT, Keckler, SW & Iyer, RK 2019, ML-Based Fault Injection for Autonomous Vehicles: A Case for Bayesian Fault Injection. in Proceedings - 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2019., 8809495, Proceedings - 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2019, Institute of Electrical and Electronics Engineers Inc., pp. 112-124, 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2019, Portland, United States, 6/24/19. https://doi.org/10.1109/DSN.2019.00025
Jha S, Banerjee S, Tsai T, Hari SKS, Sullivan MB, Kalbarczyk ZT et al. ML-Based Fault Injection for Autonomous Vehicles: A Case for Bayesian Fault Injection. In Proceedings - 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 112-124. 8809495. (Proceedings - 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2019). https://doi.org/10.1109/DSN.2019.00025
Jha, Saurabh ; Banerjee, Subho ; Tsai, Timothy ; Hari, Siva K.S. ; Sullivan, Michael B. ; Kalbarczyk, Zbigniew T ; Keckler, Stephen W. ; Iyer, Ravishankar K. / ML-Based Fault Injection for Autonomous Vehicles : A Case for Bayesian Fault Injection. Proceedings - 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 112-124 (Proceedings - 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2019).
@inproceedings{c77d677797494e9a88e82f4dc920ef81,
title = "ML-Based Fault Injection for Autonomous Vehicles: A Case for Bayesian Fault Injection",
abstract = "The safety and resilience of fully autonomous vehicles (AVs) are of significant concern, as exemplified by several headline-making accidents. While AV development today involves verification, validation, and testing, end-To-end assessment of AV systems under accidental faults in realistic driving scenarios has been largely unexplored. This paper presents DriveFI, a machine learning-based fault injection engine, which can mine situations and faults that maximally impact AV safety, as demonstrated on two industry-grade AV technology stacks (from NVIDIA and Baidu). For example, DriveFI found 561 safety-critical faults in less than 4 hours. In comparison, random injection experiments executed over several weeks could not find any safety-critical faults.",
keywords = "Autonomous Vehicles, Fault Injection, Machine Learning",
author = "Saurabh Jha and Subho Banerjee and Timothy Tsai and Hari, {Siva K.S.} and Sullivan, {Michael B.} and Kalbarczyk, {Zbigniew T} and Keckler, {Stephen W.} and Iyer, {Ravishankar K}",
year = "2019",
month = "6",
doi = "10.1109/DSN.2019.00025",
language = "English (US)",
series = "Proceedings - 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2019",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "112--124",
booktitle = "Proceedings - 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2019",
address = "United States",

}

TY - GEN

T1 - ML-Based Fault Injection for Autonomous Vehicles

T2 - A Case for Bayesian Fault Injection

AU - Jha, Saurabh

AU - Banerjee, Subho

AU - Tsai, Timothy

AU - Hari, Siva K.S.

AU - Sullivan, Michael B.

AU - Kalbarczyk, Zbigniew T

AU - Keckler, Stephen W.

AU - Iyer, Ravishankar K

PY - 2019/6

Y1 - 2019/6

N2 - The safety and resilience of fully autonomous vehicles (AVs) are of significant concern, as exemplified by several headline-making accidents. While AV development today involves verification, validation, and testing, end-To-end assessment of AV systems under accidental faults in realistic driving scenarios has been largely unexplored. This paper presents DriveFI, a machine learning-based fault injection engine, which can mine situations and faults that maximally impact AV safety, as demonstrated on two industry-grade AV technology stacks (from NVIDIA and Baidu). For example, DriveFI found 561 safety-critical faults in less than 4 hours. In comparison, random injection experiments executed over several weeks could not find any safety-critical faults.

AB - The safety and resilience of fully autonomous vehicles (AVs) are of significant concern, as exemplified by several headline-making accidents. While AV development today involves verification, validation, and testing, end-To-end assessment of AV systems under accidental faults in realistic driving scenarios has been largely unexplored. This paper presents DriveFI, a machine learning-based fault injection engine, which can mine situations and faults that maximally impact AV safety, as demonstrated on two industry-grade AV technology stacks (from NVIDIA and Baidu). For example, DriveFI found 561 safety-critical faults in less than 4 hours. In comparison, random injection experiments executed over several weeks could not find any safety-critical faults.

KW - Autonomous Vehicles

KW - Fault Injection

KW - Machine Learning

UR - http://www.scopus.com/inward/record.url?scp=85072099788&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85072099788&partnerID=8YFLogxK

U2 - 10.1109/DSN.2019.00025

DO - 10.1109/DSN.2019.00025

M3 - Conference contribution

AN - SCOPUS:85072099788

T3 - Proceedings - 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2019

SP - 112

EP - 124

BT - Proceedings - 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -