TY - GEN
T1 - Disentangling Controllable Object Through Video Prediction Improves Visual Reinforcement Learning
AU - Zhong, Yuanyi
AU - Schwing, Alexander
AU - Peng, Jian
N1 - Publisher Copyright:
© 2020 IEEE.
PY - 2020/5
Y1 - 2020/5
N2 - In many vision-based reinforcement learning (RL) problems, the agent controls a movable object in its visual field, e.g., the player's avatar in video games and the robotic arm in visual grasping and manipulation. Leveraging action-conditioned video prediction, we propose an end-to-end learning frame-work to disentangle the controllable object from the observation signal. The disentangled representation is shown to be useful for RL as additional observation channels to the agent. Experiments on a set of Atari games with the popular Double DQN algorithm demonstrate improved sample efficiency and game performance (from 222.8% to 261.4% measured in normalized game scores, with prediction bonus reward).
AB - In many vision-based reinforcement learning (RL) problems, the agent controls a movable object in its visual field, e.g., the player's avatar in video games and the robotic arm in visual grasping and manipulation. Leveraging action-conditioned video prediction, we propose an end-to-end learning frame-work to disentangle the controllable object from the observation signal. The disentangled representation is shown to be useful for RL as additional observation channels to the agent. Experiments on a set of Atari games with the popular Double DQN algorithm demonstrate improved sample efficiency and game performance (from 222.8% to 261.4% measured in normalized game scores, with prediction bonus reward).
KW - reinforcement learning
KW - representation learning
KW - sample efficiency
KW - video prediction
UR - http://www.scopus.com/inward/record.url?scp=85089239006&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85089239006&partnerID=8YFLogxK
U2 - 10.1109/ICASSP40776.2020.9053819
DO - 10.1109/ICASSP40776.2020.9053819
M3 - Conference contribution
AN - SCOPUS:85089239006
T3 - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
SP - 3672
EP - 3676
BT - 2020 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2020 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020
Y2 - 4 May 2020 through 8 May 2020
ER -