Learning to communicate: A machine learning framework for heterogeneous multi-agent robotic systems

Hyung Jin Yoon, Huaiyu Chen, Kehan Long, Heling Zhang, Aditya Gahlawat, Donghwan Lee, Naira Hovakimyan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We present a machine learning framework for multi-agent systems to learn both the optimal policy for maximizing the rewards and the encoding of the high dimensional visual observation. The encoding is useful for sharing local visual observations with other agents under communication resource constraints. The actor-encoder encodes the raw images and chooses an action based on local observations and messages sent by the other agents. The machine learning agent generates not only an actuator command to the physical device, but also a communication message to the other agents. We formulate a reinforcement learning problem, which extends the action space to consider the communication action as well. The feasibility of the reinforcement learning framework is demonstrated using a 3D simulation environment with two collaborating agents. The environment provides realistic visual observations to be used and shared between the two agents.

Original languageEnglish (US)
Title of host publicationAIAA Scitech 2019 Forum
PublisherAmerican Institute of Aeronautics and Astronautics Inc, AIAA
ISBN (Print)9781624105784
DOIs
StatePublished - Jan 1 2019
EventAIAA Scitech Forum, 2019 - San Diego, United States
Duration: Jan 7 2019Jan 11 2019

Publication series

NameAIAA Scitech 2019 Forum

Conference

ConferenceAIAA Scitech Forum, 2019
CountryUnited States
CitySan Diego
Period1/7/191/11/19

Fingerprint

Learning systems
Robotics
Reinforcement learning
Communication
Multi agent systems
Actuators

ASJC Scopus subject areas

  • Aerospace Engineering

Cite this

Yoon, H. J., Chen, H., Long, K., Zhang, H., Gahlawat, A., Lee, D., & Hovakimyan, N. (2019). Learning to communicate: A machine learning framework for heterogeneous multi-agent robotic systems. In AIAA Scitech 2019 Forum (AIAA Scitech 2019 Forum). American Institute of Aeronautics and Astronautics Inc, AIAA. https://doi.org/10.2514/6.2019-1456

Learning to communicate : A machine learning framework for heterogeneous multi-agent robotic systems. / Yoon, Hyung Jin; Chen, Huaiyu; Long, Kehan; Zhang, Heling; Gahlawat, Aditya; Lee, Donghwan; Hovakimyan, Naira.

AIAA Scitech 2019 Forum. American Institute of Aeronautics and Astronautics Inc, AIAA, 2019. (AIAA Scitech 2019 Forum).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yoon, HJ, Chen, H, Long, K, Zhang, H, Gahlawat, A, Lee, D & Hovakimyan, N 2019, Learning to communicate: A machine learning framework for heterogeneous multi-agent robotic systems. in AIAA Scitech 2019 Forum. AIAA Scitech 2019 Forum, American Institute of Aeronautics and Astronautics Inc, AIAA, AIAA Scitech Forum, 2019, San Diego, United States, 1/7/19. https://doi.org/10.2514/6.2019-1456
Yoon HJ, Chen H, Long K, Zhang H, Gahlawat A, Lee D et al. Learning to communicate: A machine learning framework for heterogeneous multi-agent robotic systems. In AIAA Scitech 2019 Forum. American Institute of Aeronautics and Astronautics Inc, AIAA. 2019. (AIAA Scitech 2019 Forum). https://doi.org/10.2514/6.2019-1456
Yoon, Hyung Jin ; Chen, Huaiyu ; Long, Kehan ; Zhang, Heling ; Gahlawat, Aditya ; Lee, Donghwan ; Hovakimyan, Naira. / Learning to communicate : A machine learning framework for heterogeneous multi-agent robotic systems. AIAA Scitech 2019 Forum. American Institute of Aeronautics and Astronautics Inc, AIAA, 2019. (AIAA Scitech 2019 Forum).
@inproceedings{36e95d145d3b47f69a28ebe31a3c1f2c,
title = "Learning to communicate: A machine learning framework for heterogeneous multi-agent robotic systems",
abstract = "We present a machine learning framework for multi-agent systems to learn both the optimal policy for maximizing the rewards and the encoding of the high dimensional visual observation. The encoding is useful for sharing local visual observations with other agents under communication resource constraints. The actor-encoder encodes the raw images and chooses an action based on local observations and messages sent by the other agents. The machine learning agent generates not only an actuator command to the physical device, but also a communication message to the other agents. We formulate a reinforcement learning problem, which extends the action space to consider the communication action as well. The feasibility of the reinforcement learning framework is demonstrated using a 3D simulation environment with two collaborating agents. The environment provides realistic visual observations to be used and shared between the two agents.",
author = "Yoon, {Hyung Jin} and Huaiyu Chen and Kehan Long and Heling Zhang and Aditya Gahlawat and Donghwan Lee and Naira Hovakimyan",
year = "2019",
month = "1",
day = "1",
doi = "10.2514/6.2019-1456",
language = "English (US)",
isbn = "9781624105784",
series = "AIAA Scitech 2019 Forum",
publisher = "American Institute of Aeronautics and Astronautics Inc, AIAA",
booktitle = "AIAA Scitech 2019 Forum",

}

TY - GEN

T1 - Learning to communicate

T2 - A machine learning framework for heterogeneous multi-agent robotic systems

AU - Yoon, Hyung Jin

AU - Chen, Huaiyu

AU - Long, Kehan

AU - Zhang, Heling

AU - Gahlawat, Aditya

AU - Lee, Donghwan

AU - Hovakimyan, Naira

PY - 2019/1/1

Y1 - 2019/1/1

N2 - We present a machine learning framework for multi-agent systems to learn both the optimal policy for maximizing the rewards and the encoding of the high dimensional visual observation. The encoding is useful for sharing local visual observations with other agents under communication resource constraints. The actor-encoder encodes the raw images and chooses an action based on local observations and messages sent by the other agents. The machine learning agent generates not only an actuator command to the physical device, but also a communication message to the other agents. We formulate a reinforcement learning problem, which extends the action space to consider the communication action as well. The feasibility of the reinforcement learning framework is demonstrated using a 3D simulation environment with two collaborating agents. The environment provides realistic visual observations to be used and shared between the two agents.

AB - We present a machine learning framework for multi-agent systems to learn both the optimal policy for maximizing the rewards and the encoding of the high dimensional visual observation. The encoding is useful for sharing local visual observations with other agents under communication resource constraints. The actor-encoder encodes the raw images and chooses an action based on local observations and messages sent by the other agents. The machine learning agent generates not only an actuator command to the physical device, but also a communication message to the other agents. We formulate a reinforcement learning problem, which extends the action space to consider the communication action as well. The feasibility of the reinforcement learning framework is demonstrated using a 3D simulation environment with two collaborating agents. The environment provides realistic visual observations to be used and shared between the two agents.

UR - http://www.scopus.com/inward/record.url?scp=85068990018&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85068990018&partnerID=8YFLogxK

U2 - 10.2514/6.2019-1456

DO - 10.2514/6.2019-1456

M3 - Conference contribution

AN - SCOPUS:85068990018

SN - 9781624105784

T3 - AIAA Scitech 2019 Forum

BT - AIAA Scitech 2019 Forum

PB - American Institute of Aeronautics and Astronautics Inc, AIAA

ER -