Anomaly detection in streaming environmental sensor data: A data-driven modeling approach

David J. Hill, Barbara S. Minsker

Research output: Contribution to journalArticlepeer-review

Abstract

The deployment of environmental sensors has generated an interest in real-time applications of the data they collect. This research develops a real-time anomaly detection method for environmental data streams that can be used to identify data that deviate from historical patterns. The method is based on an autoregressive data-driven model of the data stream and its corresponding prediction interval. It performs fast, incremental evaluation of data as it becomes available, scales to large quantities of data, and requires no pre-classification of anomalies. Furthermore, this method can be easily deployed on a large heterogeneous sensor network. Sixteen instantiations of this method are compared based on their ability to identify measurement errors in a windspeed data stream from Corpus Christi, Texas. The results indicate that a multilayer perceptron model of the data stream, coupled with replacement of anomalous data points, performs well at identifying erroneous data in this data stream.

Original languageEnglish (US)
Pages (from-to)1014-1022
Number of pages9
JournalEnvironmental Modelling and Software
Volume25
Issue number9
DOIs
StatePublished - Sep 2010

Keywords

  • Anomaly detection
  • Artificial intelligence
  • Coastal environment
  • Data quality control
  • Data-driven modeling
  • Machine learning
  • Real-time data
  • Sensor networks

ASJC Scopus subject areas

  • Software
  • Environmental Engineering
  • Ecological Modeling

Fingerprint Dive into the research topics of 'Anomaly detection in streaming environmental sensor data: A data-driven modeling approach'. Together they form a unique fingerprint.

Cite this