Likelihood Rate Based Estimation of Nonstationary Markov Models

Harshal Maske, Girish Chowdhary

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Although Markov models are widely used and researched, improving their capability to guarantee optimal performance in real world processes relies on perfect state inference amidst non-stationarity. This paper develops a novel estimation technique to capture non-stationarity in Markov sequences induced by switching transition probability matrices (TPMs). We introduce the concept of likelihood rate to establish existence of non-stationarity and to detect and estimate multiple TPMs. We layer another Markov chain to model switches between the estimated transition probability matrices resulting in Layered Non-stationary Markov Models (LNMM). We present a novel non-parametric estimation process that evaluates multiple priors and performs Bayesian update of a prior with highest likelihood rate. Our experiments on synthetic and honey bee dance dataset shows that the inference using LNMM is two times more accurate than the existing unsupervised learning methods while being computationally efficient, validating it as a highly expressive model for non-stationary Markov sequences.

Original languageEnglish (US)
Title of host publication2018 IEEE Conference on Decision and Control, CDC 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages4759-4766
Number of pages8
ISBN (Electronic)9781538613955
DOIs
StatePublished - Jan 18 2019
Event57th IEEE Conference on Decision and Control, CDC 2018 - Miami, United States
Duration: Dec 17 2018Dec 19 2018

Publication series

NameProceedings of the IEEE Conference on Decision and Control
Volume2018-December
ISSN (Print)0743-1546

Conference

Conference57th IEEE Conference on Decision and Control, CDC 2018
CountryUnited States
CityMiami
Period12/17/1812/19/18

Fingerprint

Transition Probability Matrix
Nonstationarity
Markov Model
Likelihood
Unsupervised Learning
Nonparametric Estimation
Markov chain
Switch
Update
Unsupervised learning
Evaluate
Markov processes
Model
Estimate
Experiment
Switches
Experiments

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Modeling and Simulation
  • Control and Optimization

Cite this

Maske, H., & Chowdhary, G. (2019). Likelihood Rate Based Estimation of Nonstationary Markov Models. In 2018 IEEE Conference on Decision and Control, CDC 2018 (pp. 4759-4766). [8619316] (Proceedings of the IEEE Conference on Decision and Control; Vol. 2018-December). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CDC.2018.8619316

Likelihood Rate Based Estimation of Nonstationary Markov Models. / Maske, Harshal; Chowdhary, Girish.

2018 IEEE Conference on Decision and Control, CDC 2018. Institute of Electrical and Electronics Engineers Inc., 2019. p. 4759-4766 8619316 (Proceedings of the IEEE Conference on Decision and Control; Vol. 2018-December).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Maske, H & Chowdhary, G 2019, Likelihood Rate Based Estimation of Nonstationary Markov Models. in 2018 IEEE Conference on Decision and Control, CDC 2018., 8619316, Proceedings of the IEEE Conference on Decision and Control, vol. 2018-December, Institute of Electrical and Electronics Engineers Inc., pp. 4759-4766, 57th IEEE Conference on Decision and Control, CDC 2018, Miami, United States, 12/17/18. https://doi.org/10.1109/CDC.2018.8619316
Maske H, Chowdhary G. Likelihood Rate Based Estimation of Nonstationary Markov Models. In 2018 IEEE Conference on Decision and Control, CDC 2018. Institute of Electrical and Electronics Engineers Inc. 2019. p. 4759-4766. 8619316. (Proceedings of the IEEE Conference on Decision and Control). https://doi.org/10.1109/CDC.2018.8619316
Maske, Harshal ; Chowdhary, Girish. / Likelihood Rate Based Estimation of Nonstationary Markov Models. 2018 IEEE Conference on Decision and Control, CDC 2018. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 4759-4766 (Proceedings of the IEEE Conference on Decision and Control).
@inproceedings{0882949628804bcf8a5faa014e47819a,
title = "Likelihood Rate Based Estimation of Nonstationary Markov Models",
abstract = "Although Markov models are widely used and researched, improving their capability to guarantee optimal performance in real world processes relies on perfect state inference amidst non-stationarity. This paper develops a novel estimation technique to capture non-stationarity in Markov sequences induced by switching transition probability matrices (TPMs). We introduce the concept of likelihood rate to establish existence of non-stationarity and to detect and estimate multiple TPMs. We layer another Markov chain to model switches between the estimated transition probability matrices resulting in Layered Non-stationary Markov Models (LNMM). We present a novel non-parametric estimation process that evaluates multiple priors and performs Bayesian update of a prior with highest likelihood rate. Our experiments on synthetic and honey bee dance dataset shows that the inference using LNMM is two times more accurate than the existing unsupervised learning methods while being computationally efficient, validating it as a highly expressive model for non-stationary Markov sequences.",
author = "Harshal Maske and Girish Chowdhary",
year = "2019",
month = "1",
day = "18",
doi = "10.1109/CDC.2018.8619316",
language = "English (US)",
series = "Proceedings of the IEEE Conference on Decision and Control",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "4759--4766",
booktitle = "2018 IEEE Conference on Decision and Control, CDC 2018",
address = "United States",

}

TY - GEN

T1 - Likelihood Rate Based Estimation of Nonstationary Markov Models

AU - Maske, Harshal

AU - Chowdhary, Girish

PY - 2019/1/18

Y1 - 2019/1/18

N2 - Although Markov models are widely used and researched, improving their capability to guarantee optimal performance in real world processes relies on perfect state inference amidst non-stationarity. This paper develops a novel estimation technique to capture non-stationarity in Markov sequences induced by switching transition probability matrices (TPMs). We introduce the concept of likelihood rate to establish existence of non-stationarity and to detect and estimate multiple TPMs. We layer another Markov chain to model switches between the estimated transition probability matrices resulting in Layered Non-stationary Markov Models (LNMM). We present a novel non-parametric estimation process that evaluates multiple priors and performs Bayesian update of a prior with highest likelihood rate. Our experiments on synthetic and honey bee dance dataset shows that the inference using LNMM is two times more accurate than the existing unsupervised learning methods while being computationally efficient, validating it as a highly expressive model for non-stationary Markov sequences.

AB - Although Markov models are widely used and researched, improving their capability to guarantee optimal performance in real world processes relies on perfect state inference amidst non-stationarity. This paper develops a novel estimation technique to capture non-stationarity in Markov sequences induced by switching transition probability matrices (TPMs). We introduce the concept of likelihood rate to establish existence of non-stationarity and to detect and estimate multiple TPMs. We layer another Markov chain to model switches between the estimated transition probability matrices resulting in Layered Non-stationary Markov Models (LNMM). We present a novel non-parametric estimation process that evaluates multiple priors and performs Bayesian update of a prior with highest likelihood rate. Our experiments on synthetic and honey bee dance dataset shows that the inference using LNMM is two times more accurate than the existing unsupervised learning methods while being computationally efficient, validating it as a highly expressive model for non-stationary Markov sequences.

UR - http://www.scopus.com/inward/record.url?scp=85062182195&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85062182195&partnerID=8YFLogxK

U2 - 10.1109/CDC.2018.8619316

DO - 10.1109/CDC.2018.8619316

M3 - Conference contribution

AN - SCOPUS:85062182195

T3 - Proceedings of the IEEE Conference on Decision and Control

SP - 4759

EP - 4766

BT - 2018 IEEE Conference on Decision and Control, CDC 2018

PB - Institute of Electrical and Electronics Engineers Inc.

ER -