TY - GEN
T1 - Where to look
T2 - 29th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016
AU - Shih, Kevin J.
AU - Singh, Saurabh
AU - Hoiem, Derek
N1 - Funding Information:
This work is supported by NSF CAREER award 1053768, NSF Award IIS-1029035, and ONR MURI Awards N00014-10-1-0934 and N00014-16-1-2007. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Tesla K40 GPUs used for this research
Publisher Copyright:
© 2016 IEEE.
PY - 2016/12/9
Y1 - 2016/12/9
N2 - We present a method that learns to answer visual questions by selecting image regions relevant to the text-based query. Our method maps textual queries and visual features from various regions into a shared space where they are compared for relevance with an inner product. Our method exhibits significant improvements in answering questions such as 'what color,' where it is necessary to evaluate a specific location, and 'what room,' where it selectively identifies informative image regions. Our model is tested on the recently released VQA [1] dataset, which features free-form human-annotated questions and answers.
AB - We present a method that learns to answer visual questions by selecting image regions relevant to the text-based query. Our method maps textual queries and visual features from various regions into a shared space where they are compared for relevance with an inner product. Our method exhibits significant improvements in answering questions such as 'what color,' where it is necessary to evaluate a specific location, and 'what room,' where it selectively identifies informative image regions. Our model is tested on the recently released VQA [1] dataset, which features free-form human-annotated questions and answers.
UR - http://www.scopus.com/inward/record.url?scp=84986327457&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84986327457&partnerID=8YFLogxK
U2 - 10.1109/CVPR.2016.499
DO - 10.1109/CVPR.2016.499
M3 - Conference contribution
AN - SCOPUS:84986327457
T3 - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
SP - 4613
EP - 4621
BT - Proceedings - 29th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016
PB - IEEE Computer Society
Y2 - 26 June 2016 through 1 July 2016
ER -