Learning to execute instructions in a minecraft dialogue

Prashant Jayannavar; Anjali Narayan-Chen; Julia Hockenmaier

Learning to execute instructions in a minecraft dialogue

Prashant Jayannavar, Anjali Narayan-Chen, Julia Hockenmaier

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

The Minecraft Collaborative Building Task is a two-player game in which an Architect A instructs a Builder B to construct a target structure out of 3D blocks. We consider the task of predicting B's action sequences (block placements and removals) in a given game context, and show that capturing B's past actions as well as B's perspective leads to a significant improvement in performance on this challenging language understanding problem.

Original language	English (US)
Title of host publication	ACL 2020 - 58th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference
Publisher	Association for Computational Linguistics (ACL)
Pages	2589-2602
Number of pages	14
ISBN (Electronic)	9781952148255
State	Published - 2020
Event	58th Annual Meeting of the Association for Computational Linguistics, ACL 2020 - Virtual, Online, United States Duration: Jul 5 2020 → Jul 10 2020

Publication series

Name	Proceedings of the Annual Meeting of the Association for Computational Linguistics
ISSN (Print)	0736-587X

Conference

Conference	58th Annual Meeting of the Association for Computational Linguistics, ACL 2020
Country/Territory	United States
City	Virtual, Online
Period	7/5/20 → 7/10/20

ASJC Scopus subject areas

Computer Science Applications
Linguistics and Language
Language and Linguistics

Library availability

Discover UIUC Full Text

Cite this

Jayannavar, P., Narayan-Chen, A., & Hockenmaier, J. (2020). Learning to execute instructions in a minecraft dialogue. In ACL 2020 - 58th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (pp. 2589-2602). (Proceedings of the Annual Meeting of the Association for Computational Linguistics). Association for Computational Linguistics (ACL).

Learning to execute instructions in a minecraft dialogue. / Jayannavar, Prashant; Narayan-Chen, Anjali; Hockenmaier, Julia.

ACL 2020 - 58th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference. Association for Computational Linguistics (ACL), 2020. p. 2589-2602 (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Jayannavar, P, Narayan-Chen, A & Hockenmaier, J 2020, Learning to execute instructions in a minecraft dialogue. in ACL 2020 - 58th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference. Proceedings of the Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics (ACL), pp. 2589-2602, 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Virtual, Online, United States, 7/5/20.

Jayannavar P, Narayan-Chen A, Hockenmaier J. Learning to execute instructions in a minecraft dialogue. In ACL 2020 - 58th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference. Association for Computational Linguistics (ACL). 2020. p. 2589-2602. (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

Jayannavar, Prashant ; Narayan-Chen, Anjali ; Hockenmaier, Julia. / Learning to execute instructions in a minecraft dialogue. ACL 2020 - 58th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference. Association for Computational Linguistics (ACL), 2020. pp. 2589-2602 (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

@inproceedings{588ab133bd274aa48cd954b98e63fab1,

title = "Learning to execute instructions in a minecraft dialogue",

abstract = "The Minecraft Collaborative Building Task is a two-player game in which an Architect A instructs a Builder B to construct a target structure out of 3D blocks. We consider the task of predicting B's action sequences (block placements and removals) in a given game context, and show that capturing B's past actions as well as B's perspective leads to a significant improvement in performance on this challenging language understanding problem.",

author = "Prashant Jayannavar and Anjali Narayan-Chen and Julia Hockenmaier",

note = "Funding Information: We would like to thank the reviewers for their valuable comments. This work was supported by Contract W911NF-15-1-0461 with the US Defense Advanced Research Projects Agency (DARPA) Communicating with Computers Program and the Army Research Office (ARO). Approved for Public Release, Distribution Unlimited. The views expressed are those of the authors and do not reflect the official policy or position of the Department of Defense or the U.S. Government. Publisher Copyright: {\textcopyright} 2020 Association for Computational Linguistics; 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020 ; Conference date: 05-07-2020 Through 10-07-2020",

year = "2020",

language = "English (US)",

series = "Proceedings of the Annual Meeting of the Association for Computational Linguistics",

publisher = "Association for Computational Linguistics (ACL)",

pages = "2589--2602",

booktitle = "ACL 2020 - 58th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference",

}

TY - GEN

T1 - Learning to execute instructions in a minecraft dialogue

AU - Jayannavar, Prashant

AU - Narayan-Chen, Anjali

AU - Hockenmaier, Julia

N1 - Funding Information: We would like to thank the reviewers for their valuable comments. This work was supported by Contract W911NF-15-1-0461 with the US Defense Advanced Research Projects Agency (DARPA) Communicating with Computers Program and the Army Research Office (ARO). Approved for Public Release, Distribution Unlimited. The views expressed are those of the authors and do not reflect the official policy or position of the Department of Defense or the U.S. Government. Publisher Copyright: © 2020 Association for Computational Linguistics

PY - 2020

Y1 - 2020

N2 - The Minecraft Collaborative Building Task is a two-player game in which an Architect A instructs a Builder B to construct a target structure out of 3D blocks. We consider the task of predicting B's action sequences (block placements and removals) in a given game context, and show that capturing B's past actions as well as B's perspective leads to a significant improvement in performance on this challenging language understanding problem.

AB - The Minecraft Collaborative Building Task is a two-player game in which an Architect A instructs a Builder B to construct a target structure out of 3D blocks. We consider the task of predicting B's action sequences (block placements and removals) in a given game context, and show that capturing B's past actions as well as B's perspective leads to a significant improvement in performance on this challenging language understanding problem.

UR - http://www.scopus.com/inward/record.url?scp=85106400397&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85106400397&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85106400397

T3 - Proceedings of the Annual Meeting of the Association for Computational Linguistics

SP - 2589

EP - 2602

BT - ACL 2020 - 58th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference

PB - Association for Computational Linguistics (ACL)

T2 - 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020

Y2 - 5 July 2020 through 10 July 2020

ER -

Learning to execute instructions in a minecraft dialogue

Abstract

Publication series

Conference

ASJC Scopus subject areas

Library availability

Related links

Fingerprint

Cite this