Action detection in complex scenes with spatial and temporal ambiguities

Yuxiao Hu, Liangliang Cao, Fengjun Lv, Shuicheng Yan, Yihong Gong, Thomas S. Huang

Research output: Chapter in Book/Report/Conference proceedingConference contribution


In this paper, we investigate the detection of semantic human actions in complex scenes. Unlike conventional action recognition in well-controlled environments, action detection in complex scenes suffers from cluttered backgrounds, heavy crowds, occluded bodies, and spatial-temporal boundary ambiguities caused by imperfect human detection and tracking. Conventional algorithms are likely to fail with such spatial-temporal ambiguities. In this work, the candidate regions of an action are treated as a bag of instances. Then a novel multiple-instance learning framework, named SMILE-SVM (Simulated annealingMultiple Instance LEarning Support Vector Machines), is presented for learning human action detector based on imprecise action locations. SMILE-SVM is extensively evaluated with satisfactory performances on two tasks: 1) human action detection on a public video action database with cluttered backgrounds, and 2) a real world problem of detecting whether the customers in a shopping mall show an intention to purchase the merchandise on shelf (even if they didn't buy it eventually). In addition, the complementary nature of motion and appearance features in action detection are also validated, demonstrating a boosted performance in our experiments.

Original languageEnglish (US)
Title of host publication2009 IEEE 12th International Conference on Computer Vision, ICCV 2009
Number of pages8
StatePublished - 2009
Event12th International Conference on Computer Vision, ICCV 2009 - Kyoto, Japan
Duration: Sep 29 2009Oct 2 2009

Publication series

NameProceedings of the IEEE International Conference on Computer Vision


Other12th International Conference on Computer Vision, ICCV 2009

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition


Dive into the research topics of 'Action detection in complex scenes with spatial and temporal ambiguities'. Together they form a unique fingerprint.

Cite this