Joint Estimation of Human Pose and Conversational Groups from Social Scenes

Jagannadan Varadarajan, Ramanathan Subramanian, Samuel Rota Bulò, Narendra Ahuja, Oswald Lanz, Elisa Ricci

Research output: Contribution to journalArticlepeer-review


Despite many attempts in the last few years, automatic analysis of social scenes captured by wide-angle camera networks remains a very challenging task due to the low resolution of targets, background clutter and frequent and persistent occlusions. In this paper, we present a novel framework for jointly estimating (i) head, body orientations of targets and (ii) conversational groups called F-formations from social scenes. In contrast to prior works that have (a) exploited the limited range of head and body orientations to jointly learn both, or (b) employed the mutual head (but not body) pose of interactors for deducing F-formations, we propose a weakly-supervised learning algorithm for joint inference. Our algorithm employs body pose as the primary cue for F-formation estimation, and an alternating optimization strategy is proposed to iteratively refine F-formation and pose estimates. We demonstrate the increased efficacy of joint inference over the state-of-the-art via extensive experiments on three social datasets.

Original languageEnglish (US)
Pages (from-to)410-429
Number of pages20
JournalInternational Journal of Computer Vision
Issue number2-4
StatePublished - Apr 1 2018


  • Conversational groups
  • Convex optimization
  • F-formation estimation
  • Head and body pose estimation
  • Semi-supervised learning
  • Video surveillance

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence


Dive into the research topics of 'Joint Estimation of Human Pose and Conversational Groups from Social Scenes'. Together they form a unique fingerprint.

Cite this