Although Markov models are widely used and researched, improving their capability to guarantee optimal performance in real world processes relies on perfect state inference amidst non-stationarity. This paper develops a novel estimation technique to capture non-stationarity in Markov sequences induced by switching transition probability matrices (TPMs). We introduce the concept of likelihood rate to establish existence of non-stationarity and to detect and estimate multiple TPMs. We layer another Markov chain to model switches between the estimated transition probability matrices resulting in Layered Non-stationary Markov Models (LNMM). We present a novel non-parametric estimation process that evaluates multiple priors and performs Bayesian update of a prior with highest likelihood rate. Our experiments on synthetic and honey bee dance dataset shows that the inference using LNMM is two times more accurate than the existing unsupervised learning methods while being computationally efficient, validating it as a highly expressive model for non-stationary Markov sequences.

Original languageEnglish (US)
Title of host publication2018 IEEE Conference on Decision and Control, CDC 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages8
ISBN (Electronic)9781538613955
StatePublished - Jul 2 2018
Event57th IEEE Conference on Decision and Control, CDC 2018 - Miami, United States
Duration: Dec 17 2018Dec 19 2018

Publication series

NameProceedings of the IEEE Conference on Decision and Control
ISSN (Print)0743-1546
ISSN (Electronic)2576-2370


Conference57th IEEE Conference on Decision and Control, CDC 2018
Country/TerritoryUnited States

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Modeling and Simulation
  • Control and Optimization


Dive into the research topics of 'Likelihood Rate Based Estimation of Nonstationary Markov Models'. Together they form a unique fingerprint.

Cite this