TY - GEN
T1 - Adaptive planning for markov decision processes with uncertain transition models via incremental feature dependency discovery
AU - Ure, N. Kemal
AU - Geramifard, Alborz
AU - Chowdhary, Girish
AU - How, Jonathan P.
N1 - Copyright:
Copyright 2012 Elsevier B.V., All rights reserved.
PY - 2012
Y1 - 2012
N2 - Solving large scale sequential decision making problems without prior knowledge of the state transition model is a key problem in the planning literature. One approach to tackle this problem is to learn the state transition model online using limited observed measurements. We present an adaptive function approximator (incremental Feature Dependency Discovery (iFDD)) that grows the set of features online to approximately represent the transition model. The approach leverages existing feature-dependencies to build a sparse representation of the state transition model. Theoretical analysis and numerical simulations in domains with state space sizes varying from thousands to millions are used to illustrate the benefit of using iFDD for incrementally building transition models in a planning framework.
AB - Solving large scale sequential decision making problems without prior knowledge of the state transition model is a key problem in the planning literature. One approach to tackle this problem is to learn the state transition model online using limited observed measurements. We present an adaptive function approximator (incremental Feature Dependency Discovery (iFDD)) that grows the set of features online to approximately represent the transition model. The approach leverages existing feature-dependencies to build a sparse representation of the state transition model. Theoretical analysis and numerical simulations in domains with state space sizes varying from thousands to millions are used to illustrate the benefit of using iFDD for incrementally building transition models in a planning framework.
UR - http://www.scopus.com/inward/record.url?scp=84866860255&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84866860255&partnerID=8YFLogxK
U2 - 10.1007/978-3-642-33486-3_7
DO - 10.1007/978-3-642-33486-3_7
M3 - Conference contribution
AN - SCOPUS:84866860255
SN - 9783642334856
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 99
EP - 115
BT - Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2012, Proceedings
T2 - 2012 European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML-PKDD 2012
Y2 - 24 September 2012 through 28 September 2012
ER -