Abstract

Although Markov models are widely used and researched, improving their capability to guarantee optimal performance in real world processes relies on perfect state inference amidst non-stationarity. This paper develops a novel estimation technique to capture non-stationarity in Markov sequences induced by switching transition probability matrices (TPMs). We introduce the concept of likelihood rate to establish existence of non-stationarity and to detect and estimate multiple TPMs. We layer another Markov chain to model switches between the estimated transition probability matrices resulting in Layered Non-stationary Markov Models (LNMM). We present a novel non-parametric estimation process that evaluates multiple priors and performs Bayesian update of a prior with highest likelihood rate. Our experiments on synthetic and honey bee dance dataset shows that the inference using LNMM is two times more accurate than the existing unsupervised learning methods while being computationally efficient, validating it as a highly expressive model for non-stationary Markov sequences.

Original languageEnglish (US)
Title of host publication2018 IEEE Conference on Decision and Control, CDC 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages4759-4766
Number of pages8
ISBN (Electronic)9781538613955
DOIs
StatePublished - Jan 18 2019
Event57th IEEE Conference on Decision and Control, CDC 2018 - Miami, United States
Duration: Dec 17 2018Dec 19 2018

Publication series

NameProceedings of the IEEE Conference on Decision and Control
Volume2018-December
ISSN (Print)0743-1546

Conference

Conference57th IEEE Conference on Decision and Control, CDC 2018
CountryUnited States
CityMiami
Period12/17/1812/19/18

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Modeling and Simulation
  • Control and Optimization

Fingerprint Dive into the research topics of 'Likelihood Rate Based Estimation of Nonstationary Markov Models'. Together they form a unique fingerprint.

  • Cite this

    Maske, H., & Chowdhary, G. (2019). Likelihood Rate Based Estimation of Nonstationary Markov Models. In 2018 IEEE Conference on Decision and Control, CDC 2018 (pp. 4759-4766). [8619316] (Proceedings of the IEEE Conference on Decision and Control; Vol. 2018-December). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CDC.2018.8619316