Artificial intelligence framework for simulating clinical decision-making: A Markov decision process approach

Casey C. Bennett, Kris Hauser

Research output: Contribution to journalArticle

Abstract

Objective: In the modern healthcare system, rapidly expanding costs/complexity, the growing myriad of treatment options, and exploding information streams that often do not effectively reach the front lines hinder the ability to choose optimal treatment decisions over time. The goal in this paper is to develop a general purpose (non-disease-specific) computational/artificial intelligence (AI) framework to address these challenges. This framework serves two potential functions: (1) a simulation environment for exploring various healthcare policies, payment methodologies, etc., and (2) the basis for clinical artificial intelligence - an AI that can " think like a doctor" Methods: This approach combines Markov decision processes and dynamic decision networks to learn from clinical data and develop complex plans via simulation of alternative sequential decision paths while capturing the sometimes conflicting, sometimes synergistic interactions of various components in the healthcare system. It can operate in partially observable environments (in the case of missing observations or data) by maintaining belief states about patient health status and functions as an online agent that plans and re-plans as actions are performed and new observations are obtained. This framework was evaluated using real patient data from an electronic health record. Results: The results demonstrate the feasibility of this approach; such an AI framework easily outperforms the current treatment-as-usual (TAU) case-rate/fee-for-service models of healthcare. The cost per unit of outcome change (CPUC) was $189 vs $497 for AI vs. TAU (where lower is considered optimal) - while at the same time the AI approach could obtain a 30-35% increase in patient outcomes. Tweaking certain AI model parameters could further enhance this advantage, obtaining approximately 50% more improvement (outcome change) for roughly half the costs. Conclusion: Given careful design and problem formulation, an AI simulation framework can approximate optimal decisions even in complex and uncertain environments. Future work is described that outlines potential lines of research and integration of machine learning algorithms for personalized medicine.

Original languageEnglish (US)
Pages (from-to)9-19
Number of pages11
JournalArtificial Intelligence in Medicine
Volume57
Issue number1
DOIs
StatePublished - Jan 1 2013
Externally publishedYes

Keywords

  • Chronic illness
  • Clinical artificial intelligence
  • Dynamic decision network
  • Markov decision process
  • Medical decision making
  • Multi-agent system

ASJC Scopus subject areas

  • Medicine (miscellaneous)
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Artificial intelligence framework for simulating clinical decision-making: A Markov decision process approach'. Together they form a unique fingerprint.

  • Cite this