On Network Quickest Change Detection with Uncertain Models: An Experimental Study

We study the problem of Quickest Change Detection (QCD) in a complex networked system consisting of a set of heterogeneous agents that sequentially feed information to a central fusion center. At any unknown deterministic time, a persistent anomaly occurs, causing the distribution of observations from an unknown distinguishable subset of agents to simultaneously change from a nominal (pre-change) distribution to an anomalous (post-change) distribution, and the goal of the fusion center is to detect the change as quickly as possible subject to a false alarm constraint. Traditionally, various fusion rules have been proposed that assume that the distributions at each agent are either completely known or unknown and are locally solved using the Cumulative Sum (CuSum) and Generalized Likelihood Ratio (GLR) statistics, respectively. When an agent has access to training data, the Uncertain Likelihood Ratio (ULR) test generalizes distributional assumptions using uncertain distributions. However, the ULR has not been implemented for network change detection. This paper empirically studies incorporating the ULR statistics into the existing fusion rules for QCD and compares the average detection delay. Our results show that the ULR test can improve the average detection delay over the GLR tests using certain fusion techniques, while approaching the detection delay of the CuSum tests as the training data increases. Our results provide insights into future theoretical analysis to improve network QCD with imprecise knowledge of the distributions.