Adaptive planning for markov decision processes with uncertain transition models via incremental feature dependency discovery

N. Kemal Ure, Alborz Geramifard, Girish Chowdhary, Jonathan P. How

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Solving large scale sequential decision making problems without prior knowledge of the state transition model is a key problem in the planning literature. One approach to tackle this problem is to learn the state transition model online using limited observed measurements. We present an adaptive function approximator (incremental Feature Dependency Discovery (iFDD)) that grows the set of features online to approximately represent the transition model. The approach leverages existing feature-dependencies to build a sparse representation of the state transition model. Theoretical analysis and numerical simulations in domains with state space sizes varying from thousands to millions are used to illustrate the benefit of using iFDD for incrementally building transition models in a planning framework.

Original languageEnglish (US)
Title of host publicationMachine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2012, Proceedings
Pages99-115
Number of pages17
EditionPART 2
DOIs
StatePublished - 2012
Externally publishedYes
Event2012 European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML-PKDD 2012 - Bristol, United Kingdom
Duration: Sep 24 2012Sep 28 2012

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
NumberPART 2
Volume7524 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other2012 European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML-PKDD 2012
CountryUnited Kingdom
CityBristol
Period9/24/129/28/12

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Adaptive planning for markov decision processes with uncertain transition models via incremental feature dependency discovery'. Together they form a unique fingerprint.

Cite this