Adaptive planning for markov decision processes with uncertain transition models via incremental feature dependency discovery

N. Kemal Ure; Alborz Geramifard; Girish Chowdhary; Jonathan P. How

doi:10.1007/978-3-642-33486-3_7

Adaptive planning for markov decision processes with uncertain transition models via incremental feature dependency discovery

N. Kemal Ure, Alborz Geramifard, Girish Chowdhary, Jonathan P. How

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Solving large scale sequential decision making problems without prior knowledge of the state transition model is a key problem in the planning literature. One approach to tackle this problem is to learn the state transition model online using limited observed measurements. We present an adaptive function approximator (incremental Feature Dependency Discovery (iFDD)) that grows the set of features online to approximately represent the transition model. The approach leverages existing feature-dependencies to build a sparse representation of the state transition model. Theoretical analysis and numerical simulations in domains with state space sizes varying from thousands to millions are used to illustrate the benefit of using iFDD for incrementally building transition models in a planning framework.

Original language	English (US)
Title of host publication	Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2012, Proceedings
Pages	99-115
Number of pages	17
Edition	PART 2
DOIs	https://doi.org/10.1007/978-3-642-33486-3_7
State	Published - 2012
Externally published	Yes
Event	2012 European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML-PKDD 2012 - Bristol, United Kingdom Duration: Sep 24 2012 → Sep 28 2012

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Number	PART 2
Volume	7524 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Other

Other	2012 European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML-PKDD 2012
Country	United Kingdom
City	Bristol
Period	9/24/12 → 9/28/12

ASJC Scopus subject areas

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-642-33486-3_7

Fingerprint Dive into the research topics of 'Adaptive planning for markov decision processes with uncertain transition models via incremental feature dependency discovery'. Together they form a unique fingerprint.

Cite this

Ure, N. K., Geramifard, A., Chowdhary, G., & How, J. P. (2012). Adaptive planning for markov decision processes with uncertain transition models via incremental feature dependency discovery. In Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2012, Proceedings (PART 2 ed., pp. 99-115). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7524 LNAI, No. PART 2). https://doi.org/10.1007/978-3-642-33486-3_7

Adaptive planning for markov decision processes with uncertain transition models via incremental feature dependency discovery. / Ure, N. Kemal; Geramifard, Alborz; Chowdhary, Girish; How, Jonathan P.

Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2012, Proceedings. PART 2. ed. 2012. p. 99-115 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7524 LNAI, No. PART 2).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Ure, NK, Geramifard, A, Chowdhary, G & How, JP 2012, Adaptive planning for markov decision processes with uncertain transition models via incremental feature dependency discovery. in Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2012, Proceedings. PART 2 edn, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), no. PART 2, vol. 7524 LNAI, pp. 99-115, 2012 European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML-PKDD 2012, Bristol, United Kingdom, 9/24/12. https://doi.org/10.1007/978-3-642-33486-3_7

Ure NK, Geramifard A, Chowdhary G, How JP. Adaptive planning for markov decision processes with uncertain transition models via incremental feature dependency discovery. In Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2012, Proceedings. PART 2 ed. 2012. p. 99-115. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 2). https://doi.org/10.1007/978-3-642-33486-3_7

Ure, N. Kemal ; Geramifard, Alborz ; Chowdhary, Girish ; How, Jonathan P. / Adaptive planning for markov decision processes with uncertain transition models via incremental feature dependency discovery. Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2012, Proceedings. PART 2. ed. 2012. pp. 99-115 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 2).

@inproceedings{bd6b79d176464a25a7a8605382fd1ca4,

title = "Adaptive planning for markov decision processes with uncertain transition models via incremental feature dependency discovery",

abstract = "Solving large scale sequential decision making problems without prior knowledge of the state transition model is a key problem in the planning literature. One approach to tackle this problem is to learn the state transition model online using limited observed measurements. We present an adaptive function approximator (incremental Feature Dependency Discovery (iFDD)) that grows the set of features online to approximately represent the transition model. The approach leverages existing feature-dependencies to build a sparse representation of the state transition model. Theoretical analysis and numerical simulations in domains with state space sizes varying from thousands to millions are used to illustrate the benefit of using iFDD for incrementally building transition models in a planning framework.",

author = "Ure, {N. Kemal} and Alborz Geramifard and Girish Chowdhary and How, {Jonathan P.}",

note = "Copyright: Copyright 2012 Elsevier B.V., All rights reserved.; 2012 European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML-PKDD 2012 ; Conference date: 24-09-2012 Through 28-09-2012",

year = "2012",

doi = "10.1007/978-3-642-33486-3_7",

language = "English (US)",

isbn = "9783642334856",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

number = "PART 2",

pages = "99--115",

booktitle = "Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2012, Proceedings",

edition = "PART 2",

}

TY - GEN

T1 - Adaptive planning for markov decision processes with uncertain transition models via incremental feature dependency discovery

AU - Ure, N. Kemal

AU - Geramifard, Alborz

AU - Chowdhary, Girish

AU - How, Jonathan P.

PY - 2012

Y1 - 2012

N2 - Solving large scale sequential decision making problems without prior knowledge of the state transition model is a key problem in the planning literature. One approach to tackle this problem is to learn the state transition model online using limited observed measurements. We present an adaptive function approximator (incremental Feature Dependency Discovery (iFDD)) that grows the set of features online to approximately represent the transition model. The approach leverages existing feature-dependencies to build a sparse representation of the state transition model. Theoretical analysis and numerical simulations in domains with state space sizes varying from thousands to millions are used to illustrate the benefit of using iFDD for incrementally building transition models in a planning framework.

AB - Solving large scale sequential decision making problems without prior knowledge of the state transition model is a key problem in the planning literature. One approach to tackle this problem is to learn the state transition model online using limited observed measurements. We present an adaptive function approximator (incremental Feature Dependency Discovery (iFDD)) that grows the set of features online to approximately represent the transition model. The approach leverages existing feature-dependencies to build a sparse representation of the state transition model. Theoretical analysis and numerical simulations in domains with state space sizes varying from thousands to millions are used to illustrate the benefit of using iFDD for incrementally building transition models in a planning framework.

UR - http://www.scopus.com/inward/record.url?scp=84866860255&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84866860255&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-33486-3_7

DO - 10.1007/978-3-642-33486-3_7

M3 - Conference contribution

AN - SCOPUS:84866860255

SN - 9783642334856

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 99

EP - 115

BT - Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2012, Proceedings

T2 - 2012 European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML-PKDD 2012

Y2 - 24 September 2012 through 28 September 2012

ER -