Scene change detection based on multimodal integration

Yingying Zhu; Dongru Zhou

doi:10.1117/12.539934

25 September 2003 Scene change detection based on multimodal integration

Yingying Zhu, Dongru Zhou

Proceedings Volume 5286, Third International Symposium on Multispectral Image Processing and Pattern Recognition; (2003) https://doi.org/10.1117/12.539934
Event: Third International Symposium on Multispectral Image Processing and Pattern Recognition, 2003, Beijing, China

Abstract

Scene change detection is an essential step to automatic and content-based video indexing, retrieval and browsing. In this paper, a robust scene change detection and classification approach is presented, which analyzes audio, visual and textual sources and accounts for their inter-relations and coincidence to semantically identify and classify video scenes. Audio analysis focuses on the segmentation of audio stream into four types of semantic data such as silence, speech, music and environmental sound. Further processing on speech segments aims at locating speaker changes. Video analysis partitions visual stream into shots. Text analysis can provide a supplemental source of clues for scene classification and indexing information. We integrate the video and audio analysis results to identify video scenes and use the text information detected by the video OCR technology or derived from transcripts available to refine scene classification. Results from single source segmentation are in some cases suboptimal. By combining visual, aural features adn the accessorial text information, the scence extraction accuracy is enhanced, and more semantic segmentations are developed. Experimental results are proven to rather promising.

Citation Download Citation

Yingying Zhu and Dongru Zhou "Scene change detection based on multimodal integration", Proc. SPIE 5286, Third International Symposium on Multispectral Image Processing and Pattern Recognition, (25 September 2003); https://doi.org/10.1117/12.539934

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available