TY - GEN
T1 - Phrase Localization and Visual Relationship Detection with Comprehensive Image-Language Cues
AU - Plummer, Bryan A.
AU - Mallya, Arun
AU - Cervantes, Christopher M.
AU - Hockenmaier, Julia
AU - Lazebnik, Svetlana
N1 - Publisher Copyright:
© 2017 IEEE.
PY - 2017/12/22
Y1 - 2017/12/22
N2 - This paper presents a framework for localization or grounding of phrases in images using a large collection of linguistic and visual cues. We model the appearance, size, and position of entity bounding boxes, adjectives that contain attribute information, and spatial relationships between pairs of entities connected by verbs or prepositions. Special attention is given to relationships between people and clothing or body part mentions, as they are useful for distinguishing individuals. We automatically learn weights for combining these cues and at test time, perform joint inference over all phrases in a caption. The resulting system produces state of the art performance on phrase localization on the Flickr30k Entities dataset [33] and visual relationship detection on the Stanford VRD dataset [27].
AB - This paper presents a framework for localization or grounding of phrases in images using a large collection of linguistic and visual cues. We model the appearance, size, and position of entity bounding boxes, adjectives that contain attribute information, and spatial relationships between pairs of entities connected by verbs or prepositions. Special attention is given to relationships between people and clothing or body part mentions, as they are useful for distinguishing individuals. We automatically learn weights for combining these cues and at test time, perform joint inference over all phrases in a caption. The resulting system produces state of the art performance on phrase localization on the Flickr30k Entities dataset [33] and visual relationship detection on the Stanford VRD dataset [27].
UR - http://www.scopus.com/inward/record.url?scp=85041911235&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85041911235&partnerID=8YFLogxK
U2 - 10.1109/ICCV.2017.213
DO - 10.1109/ICCV.2017.213
M3 - Conference contribution
AN - SCOPUS:85041911235
T3 - Proceedings of the IEEE International Conference on Computer Vision
SP - 1946
EP - 1955
BT - Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 16th IEEE International Conference on Computer Vision, ICCV 2017
Y2 - 22 October 2017 through 29 October 2017
ER -