TY - JOUR
T1 - ARCH-COMP24 Category Report
T2 - 11th International Workshop on Applied Verification for Continuous and Hybrid Systems, ARCH-COMP 2024
AU - Lopez, Diego Manzanas
AU - Althoff, Matthias
AU - Benet, Luis
AU - Blab, Clemens
AU - Forets, Marcelo
AU - Jia, Yuhao
AU - Johnson, Taylor T.
AU - Kranzl, Manuel
AU - Ladner, Tobias
AU - Linauer, Lukas
AU - Neubauer, Philipp
AU - Neubauer, Sophie A.
AU - Schilling, Christian
AU - Zhang, Huan
AU - Zhong, Xiangru
N1 - Publisher Copyright:
© 2024, EasyChair. All rights reserved.
PY - 2024
Y1 - 2024
N2 - This report presents the results of a friendly competition for formal verification of continuous and hybrid systems with artificial intelligence (AI) components. Specifically, machine learning (ML) components in cyber-physical systems (CPS), such as feedforward neural networks used as feedback controllers in closed-loop systems, are considered, which is a class of systems classically known as intelligent control systems, or in more modern and specific terms, neural network control systems (NNCS). We broadly refer to this category as AI and NNCS (AINNCS). The friendly competition took place as part of the workshop Applied Verification for Continuous and Hybrid Systems (ARCH) in 2024. In the 8th edition of this AINNCS category at ARCH-COMP, five tools have been applied to solve 12 benchmarks, which are CORA, CROWN-Reach, GoTube, JuliaReach, and NNV. This is the year with the largest interest in the community, with two new, and three previous participants. Following last year’s trend, despite the additional challenges presented, the verification results have improved year-over-year. In terms of computation time, we can observe that the previous participants have improved as well, showing speed-ups of up to one order of magnitude, such as JuliaReach on the TORA benchmark with ReLU controller, and NNV on the TORA benchmark with both heterogeneous controllers.
AB - This report presents the results of a friendly competition for formal verification of continuous and hybrid systems with artificial intelligence (AI) components. Specifically, machine learning (ML) components in cyber-physical systems (CPS), such as feedforward neural networks used as feedback controllers in closed-loop systems, are considered, which is a class of systems classically known as intelligent control systems, or in more modern and specific terms, neural network control systems (NNCS). We broadly refer to this category as AI and NNCS (AINNCS). The friendly competition took place as part of the workshop Applied Verification for Continuous and Hybrid Systems (ARCH) in 2024. In the 8th edition of this AINNCS category at ARCH-COMP, five tools have been applied to solve 12 benchmarks, which are CORA, CROWN-Reach, GoTube, JuliaReach, and NNV. This is the year with the largest interest in the community, with two new, and three previous participants. Following last year’s trend, despite the additional challenges presented, the verification results have improved year-over-year. In terms of computation time, we can observe that the previous participants have improved as well, showing speed-ups of up to one order of magnitude, such as JuliaReach on the TORA benchmark with ReLU controller, and NNV on the TORA benchmark with both heterogeneous controllers.
UR - http://www.scopus.com/inward/record.url?scp=85207855191&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85207855191&partnerID=8YFLogxK
U2 - 10.29007/mxld
DO - 10.29007/mxld
M3 - Conference article
AN - SCOPUS:85207855191
SN - 2398-7340
VL - 103
SP - 64
EP - 121
JO - EPiC Series in Computing
JF - EPiC Series in Computing
Y2 - 3 July 2024 through 3 July 2024
ER -