TY - JOUR
T1 - Dynamic Neural Relational Inference
AU - Graber, Colin
AU - Schwing, Alexander G.
N1 - Funding Information:
We introduced Dynamic Neural Relational Inference, extending the NRI framework to systems where the relations between entities are expected to change across time. We demonstrated that modeling dynamic entity relations leads to better performance across various tasks. In the future, we will investigate whether we can adapt additional methods used by recent sequential latent variable models, such as auxiliary loss functions, to further improve performance. Acknowledgements. This work is supported in part by NSF under Grant No. 1718221 and MRI #1725729, UIUC, Samsung, 3M, Cisco Systems Inc. (Gift Award CG 1377144) and Adobe. We thank Raymond Yeh for visualization code, Yurii Vlasov for the helpful discussions, and Cisco for access to the Arcetri cluster.
PY - 2020
Y1 - 2020
N2 - Understanding interactions between entities, e.g., joints of the human body, team sports players, etc., is crucial for tasks like forecasting. However, interactions between entities are commonly not observed and often hard to quantify. To address this challenge, recently, 'Neural Relational Inference' was introduced. It predicts static relations between entities in a system and provides an interpretable representation of the underlying system dynamics that are used for better trajectory forecasting. However, generally, relations between entities change as time progresses. Hence, static relations improperly model the data. In response to this, we develop Dynamic Neural Relational Inference (dNRI), which incorporates insights from sequential latent variable models to predict separate relation graphs for every time-step. We demonstrate on several real-world datasets that modeling dynamic relations improves forecasting of complex trajectories.
AB - Understanding interactions between entities, e.g., joints of the human body, team sports players, etc., is crucial for tasks like forecasting. However, interactions between entities are commonly not observed and often hard to quantify. To address this challenge, recently, 'Neural Relational Inference' was introduced. It predicts static relations between entities in a system and provides an interpretable representation of the underlying system dynamics that are used for better trajectory forecasting. However, generally, relations between entities change as time progresses. Hence, static relations improperly model the data. In response to this, we develop Dynamic Neural Relational Inference (dNRI), which incorporates insights from sequential latent variable models to predict separate relation graphs for every time-step. We demonstrate on several real-world datasets that modeling dynamic relations improves forecasting of complex trajectories.
UR - http://www.scopus.com/inward/record.url?scp=85094857369&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85094857369&partnerID=8YFLogxK
U2 - 10.1109/CVPR42600.2020.00854
DO - 10.1109/CVPR42600.2020.00854
M3 - Conference article
AN - SCOPUS:85094857369
SP - 8510
EP - 8519
JO - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
JF - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
SN - 1063-6919
M1 - 9157727
T2 - 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020
Y2 - 14 June 2020 through 19 June 2020
ER -