Data-Driven Prediction of Confidence for EVAR in Time-Varying Datasets

The key challenge for Dynamic Data Driven Applications Systems (DDDAS) operating in time-varying environments is to predict when the learned model may lose relevance. If the learned model loses relevance, then the autonomous system is at risk of making wrong decisions. The Entropic Value at Risk (EVAR) is a computationally efficient and coherent risk measure that can be utilized to quantify this model relevance. The value of EVAR is calculated with respect to an assumed confidence value; e.g., a 90% confidence may be desired for robust decision-making. Without a model on the confidence value directly, there is no guarantee that EVAR calculations will reflect the uncertainty present in dynamic real-world environments. In this paper, we present a Bayesian model and learning algorithms to predict the state-dependent confidence necessary for calculating the EVAR in time-varying datasets. We discuss applications of the data-driven EVAR to a monitoring problem, in which a DDDAS agent has to chose a set of sensing locations in order to maximize the expected EVAR of the acquired data. In this way, the DDDAS agent can learn a model on an underlying phenomenon of interest by prioritizing the areas where the model is most likely incorrect but highly valued. We empirically demonstrate the efficacy of the presented model and learning algorithms on five real-world datasets. We show that, overall, the EVAR-Real-time Adative Prediction of Time-varying and Obscure Rewards (EVAR-RAPTOR) algorithm outperforms EVAR-Predicted Information Gain* (EVAR-PIG*) as well as naive searches such as random and sequential search across these five real-world datasets.