Random Markov decision processes for sustainable infrastructure systems

Hadi Meidani, Roger Ghanem

Research output: Contribution to journalArticlepeer-review


The quantitative assessment of the life-cycle performance of infrastructure systems has seen rapid progress using methods from systems dynamics. Markov chains and Markov decision processes are some of the more useful modelling tools that permit the integration of stochasticity in decisions related to sustainable infrastructure management. In this paper, we explore the effect of uncertainties in the characterisation of these decision models. In particular, we consider the uncertainty in the transition parameters of these models and quantitatively discuss the implication of this kind of uncertainty on the optimal decisions as the solution of these models. We also make the case for the development of rationales that can differentiate between these decision models. In addition, a computationally efficient solution algorithm is introduced for these systems under transition uncertainties. Finally, we demonstrate the sensitivity of the optimal decisions to transition uncertainties and also the computational advantage of the discussed solution algorithm using the problem of optimal maintenance of pavement.

Original languageEnglish (US)
Pages (from-to)655-667
Number of pages13
JournalStructure and Infrastructure Engineering
Issue number5
StatePublished - May 4 2015
Externally publishedYes


  • Markov decision processes
  • decision systems
  • optimal maintenance strategy
  • sustainability of infrastructure systems
  • uncertainty quantification

ASJC Scopus subject areas

  • Civil and Structural Engineering
  • Building and Construction
  • Safety, Risk, Reliability and Quality
  • Geotechnical Engineering and Engineering Geology
  • Ocean Engineering
  • Mechanical Engineering


Dive into the research topics of 'Random Markov decision processes for sustainable infrastructure systems'. Together they form a unique fingerprint.

Cite this