Multiple granularity analysis for fine-grained action detection

Bingbing Ni, Vignesh R. Paramathayalan, Pierre Moulin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We propose to decompose the fine-grained human activ- ity analysis problem into two sequential tasks with increas- ing granularity. Firstly, we infer the coarse interaction sta- tus, i.e., which object is being manipulated and where it is. Knowing that the major challenge is frequent mutual oc- clusions during manipulation, we propose an 'interaction tracking' framework in which hand/object position and in- teraction status are jointly tracked by explicitly modeling the contextual information between mutual occlusion and interaction status. Secondly, the inferred hand/object posi- tion and interaction status are utilized to provide 1) more compact feature pooling by effectively pruning large num- ber of motion features from irrelevant spatio-temporal po- sitions and 2) discriminative action detection by a granu- larity fusion strategy. Comprehensive experiments on two challenging fine-grained activity datasets (i.e., cooking ac- tion) show that the proposed framework achieves high ac- curacy/robustness in tracking multiple mutually occluded hands/objects during manipulation as well as significant performance improvement on fine-grained action detection over state-of-the-art methods.

Original languageEnglish (US)
Title of host publicationProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
PublisherIEEE Computer Society
Pages756-763
Number of pages8
ISBN (Electronic)9781479951178, 9781479951178
DOIs
StatePublished - Sep 24 2014
Event27th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014 - Columbus, United States
Duration: Jun 23 2014Jun 28 2014

Publication series

NameProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
ISSN (Print)1063-6919

Other

Other27th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014
CountryUnited States
CityColumbus
Period6/23/146/28/14

Keywords

  • action detection
  • interaction tracking
  • multiple granularity

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition

Fingerprint Dive into the research topics of 'Multiple granularity analysis for fine-grained action detection'. Together they form a unique fingerprint.

Cite this