Failure-adverse closed-loop statistical verification

John F. Quindlen, Ufuk Topcu, Girish Chowdhary, Jonathan P. How

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The increasing complexity of control systems highlights the importance of verification analysis to ensure the resulting closed-loop system robustly satisfies necessary performance requirements. Safety-critical performance requirements present further challenges as failure to satisfy these requirements have severe real-world consequences. For physical systems such as cars or aircraft, simulation-based analysis provides a good indication of the real-world system’s performance without risking physical property or people, but cannot completely replace experimental testing due to discrepancies between simulation models and the real world. Given these considerations, this paper develops new statistical verification frameworks for failure-adverse testing of safety-critical performance requirements in experimental domains. Extensions of recently-developed closed-loop statistical verification algorithms form the core of this failure-adverse approach. These new failure-adverse algorithms break verification into a simulation stage and a hardware stage and carefully select experimental tests so as to maximize the information gained with each experiment while minimizing the likelihood of encountering failures. Two case studies compare the failure-adverse closed-loop verification approach against existing verification techniques and demonstrate its effectiveness in substantially decreasing the number of failures without sacrifices in prediction accuracy.

Original languageEnglish (US)
Title of host publicationAIAA Scitech 2019 Forum
PublisherAmerican Institute of Aeronautics and Astronautics Inc, AIAA
ISBN (Print)9781624105784
DOIs
StatePublished - Jan 1 2019
EventAIAA Scitech Forum, 2019 - San Diego, United States
Duration: Jan 7 2019Jan 11 2019

Publication series

NameAIAA Scitech 2019 Forum

Conference

ConferenceAIAA Scitech Forum, 2019
CountryUnited States
CitySan Diego
Period1/7/191/11/19

Fingerprint

Testing
Closed loop systems
Railroad cars
Physical properties
Aircraft
Hardware
Control systems
Experiments

ASJC Scopus subject areas

  • Aerospace Engineering

Cite this

Quindlen, J. F., Topcu, U., Chowdhary, G., & How, J. P. (2019). Failure-adverse closed-loop statistical verification. In AIAA Scitech 2019 Forum (AIAA Scitech 2019 Forum). American Institute of Aeronautics and Astronautics Inc, AIAA. https://doi.org/10.2514/6.2019-0648

Failure-adverse closed-loop statistical verification. / Quindlen, John F.; Topcu, Ufuk; Chowdhary, Girish; How, Jonathan P.

AIAA Scitech 2019 Forum. American Institute of Aeronautics and Astronautics Inc, AIAA, 2019. (AIAA Scitech 2019 Forum).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Quindlen, JF, Topcu, U, Chowdhary, G & How, JP 2019, Failure-adverse closed-loop statistical verification. in AIAA Scitech 2019 Forum. AIAA Scitech 2019 Forum, American Institute of Aeronautics and Astronautics Inc, AIAA, AIAA Scitech Forum, 2019, San Diego, United States, 1/7/19. https://doi.org/10.2514/6.2019-0648
Quindlen JF, Topcu U, Chowdhary G, How JP. Failure-adverse closed-loop statistical verification. In AIAA Scitech 2019 Forum. American Institute of Aeronautics and Astronautics Inc, AIAA. 2019. (AIAA Scitech 2019 Forum). https://doi.org/10.2514/6.2019-0648
Quindlen, John F. ; Topcu, Ufuk ; Chowdhary, Girish ; How, Jonathan P. / Failure-adverse closed-loop statistical verification. AIAA Scitech 2019 Forum. American Institute of Aeronautics and Astronautics Inc, AIAA, 2019. (AIAA Scitech 2019 Forum).
@inproceedings{de48d8d77ee6496f91c2a6ca0ab69694,
title = "Failure-adverse closed-loop statistical verification",
abstract = "The increasing complexity of control systems highlights the importance of verification analysis to ensure the resulting closed-loop system robustly satisfies necessary performance requirements. Safety-critical performance requirements present further challenges as failure to satisfy these requirements have severe real-world consequences. For physical systems such as cars or aircraft, simulation-based analysis provides a good indication of the real-world system’s performance without risking physical property or people, but cannot completely replace experimental testing due to discrepancies between simulation models and the real world. Given these considerations, this paper develops new statistical verification frameworks for failure-adverse testing of safety-critical performance requirements in experimental domains. Extensions of recently-developed closed-loop statistical verification algorithms form the core of this failure-adverse approach. These new failure-adverse algorithms break verification into a simulation stage and a hardware stage and carefully select experimental tests so as to maximize the information gained with each experiment while minimizing the likelihood of encountering failures. Two case studies compare the failure-adverse closed-loop verification approach against existing verification techniques and demonstrate its effectiveness in substantially decreasing the number of failures without sacrifices in prediction accuracy.",
author = "Quindlen, {John F.} and Ufuk Topcu and Girish Chowdhary and How, {Jonathan P.}",
year = "2019",
month = "1",
day = "1",
doi = "10.2514/6.2019-0648",
language = "English (US)",
isbn = "9781624105784",
series = "AIAA Scitech 2019 Forum",
publisher = "American Institute of Aeronautics and Astronautics Inc, AIAA",
booktitle = "AIAA Scitech 2019 Forum",

}

TY - GEN

T1 - Failure-adverse closed-loop statistical verification

AU - Quindlen, John F.

AU - Topcu, Ufuk

AU - Chowdhary, Girish

AU - How, Jonathan P.

PY - 2019/1/1

Y1 - 2019/1/1

N2 - The increasing complexity of control systems highlights the importance of verification analysis to ensure the resulting closed-loop system robustly satisfies necessary performance requirements. Safety-critical performance requirements present further challenges as failure to satisfy these requirements have severe real-world consequences. For physical systems such as cars or aircraft, simulation-based analysis provides a good indication of the real-world system’s performance without risking physical property or people, but cannot completely replace experimental testing due to discrepancies between simulation models and the real world. Given these considerations, this paper develops new statistical verification frameworks for failure-adverse testing of safety-critical performance requirements in experimental domains. Extensions of recently-developed closed-loop statistical verification algorithms form the core of this failure-adverse approach. These new failure-adverse algorithms break verification into a simulation stage and a hardware stage and carefully select experimental tests so as to maximize the information gained with each experiment while minimizing the likelihood of encountering failures. Two case studies compare the failure-adverse closed-loop verification approach against existing verification techniques and demonstrate its effectiveness in substantially decreasing the number of failures without sacrifices in prediction accuracy.

AB - The increasing complexity of control systems highlights the importance of verification analysis to ensure the resulting closed-loop system robustly satisfies necessary performance requirements. Safety-critical performance requirements present further challenges as failure to satisfy these requirements have severe real-world consequences. For physical systems such as cars or aircraft, simulation-based analysis provides a good indication of the real-world system’s performance without risking physical property or people, but cannot completely replace experimental testing due to discrepancies between simulation models and the real world. Given these considerations, this paper develops new statistical verification frameworks for failure-adverse testing of safety-critical performance requirements in experimental domains. Extensions of recently-developed closed-loop statistical verification algorithms form the core of this failure-adverse approach. These new failure-adverse algorithms break verification into a simulation stage and a hardware stage and carefully select experimental tests so as to maximize the information gained with each experiment while minimizing the likelihood of encountering failures. Two case studies compare the failure-adverse closed-loop verification approach against existing verification techniques and demonstrate its effectiveness in substantially decreasing the number of failures without sacrifices in prediction accuracy.

UR - http://www.scopus.com/inward/record.url?scp=85068976781&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85068976781&partnerID=8YFLogxK

U2 - 10.2514/6.2019-0648

DO - 10.2514/6.2019-0648

M3 - Conference contribution

AN - SCOPUS:85068976781

SN - 9781624105784

T3 - AIAA Scitech 2019 Forum

BT - AIAA Scitech 2019 Forum

PB - American Institute of Aeronautics and Astronautics Inc, AIAA

ER -