A Unified Framework for Video Summarization, Browsing, and Retrieval

This chapter presents techniques for constructing the video “table of contents” (ToC), video highlights, and video index as well as the ways to integrate them into a unified framework. It reviews video analysis, representation, summarization, and retrieval. Further, a unified framework for video summarization and retrieval is described, followed by algorithms and experimental results on real-world video clips. Video content can be accessed by using either a top-down approach or a bottom-up approach. The top-down approach—that is, video browsing—is useful when one needs to get an “essence” of the content. The bottom-up approach—that is, video retrieval—is useful when one knows exactly what is needed from the content. In video summarization, the “essence” a summary should capture depends on whether or not the content is scripted. Because scripted content, such as news, drama, and movies, is carefully structured as a sequence of semantic units, one can get its essence by enabling a traversal through representative items from these semantic units. Hence, ToC-based video browsing caters to summarization of scripted content.