Multicamera summarization of rehabilitation sessions in home environment

Tarek Elgamal, Klara Nahrstedt

Research output: Chapter in Book/Report/Conference proceedingConference contribution


In this paper we present a cyber-physiotherapy system (CyPhy) that brings daily rehabilitation to patient's home with supervision from trained therapist. CyPhy is able to capture and record RGB-D, skeleton, and physiotherapy-related medical sensing data streams from patient's exercises using multiple cameras and body sensors. With hours of exercises from every patient, that are captured every day from multiple cameras, therapists spend huge amount of their time watching videos to monitor the correctness of patients' moves. This becomes even more challenging in the presence of multiple cameras where the therapist might not know which camera stream shows the incorrect motion. In this paper, we explore the multicamera summarization problem from various aspects: (1) We first explore the types of exercises that benefit the most from using multiple cameras; (2) We propose a method to detect incorrect motion from multiple cameras in rehabilitation exercises; (3) We show how the analysis of incorrect motion is used to summarize the video and recommend the camera view that best visualizes the mistake. Our method for detecting incorrect motion achieves more than 92% accuracy at wide range of thresholds with significant improvement of 20% over single camera and 10% over the closest approach that uses multiple cameras.

Original languageEnglish (US)
Title of host publicationMM 2017 - Proceedings of the 2017 ACM Multimedia Conference
PublisherAssociation for Computing Machinery, Inc
Number of pages9
ISBN (Electronic)9781450349062
StatePublished - Oct 23 2017
Event25th ACM International Conference on Multimedia, MM 2017 - Mountain View, United States
Duration: Oct 23 2017Oct 27 2017

Publication series

NameMM 2017 - Proceedings of the 2017 ACM Multimedia Conference


Other25th ACM International Conference on Multimedia, MM 2017
CountryUnited States
CityMountain View

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Media Technology
  • Computer Vision and Pattern Recognition
  • Software

Fingerprint Dive into the research topics of 'Multicamera summarization of rehabilitation sessions in home environment'. Together they form a unique fingerprint.

Cite this