Video action detection based on spatio-temporal contrastive learning

Yimin Lin; Qian He; Biao Guo; Qinghe Dong

doi:10.1117/12.3048411

3 October 2024 Video action detection based on spatio-temporal contrastive learning

Yimin Lin, Qian He, Biao Guo, Qinghe Dong

Proceedings Volume 13272, Fifth International Conference on Computer Vision and Data Mining (ICCVDM 2024); 132721H (2024) https://doi.org/10.1117/12.3048411
Event: 5th International Conference on Computer Vision and Data Mining (ICCVDM 2024), 2024, Changchun, China

Abstract

Human action detection in videos is crucial across various fields, yet manually annotating video data is time-consuming and results in limited datasets, weakening model generalization. To address this, this paper introduces a semi-supervised dual-channel spatio-temporal contrastive learning method-STCL, for accurate human action detection in videos. By constructing a dual-channel model to extract and compare spatio-temporal features in videos and adopting a semisupervised learning strategy that enhances unlabeled samples through forward and reverse playback, using a contrastive model to compare features, the method learns model parameters to distinguish between different video samples. This approach leverages the spatio-temporal information of unlabeled videos, avoiding extensive manual annotation. Experimental results on the Something-V2 dataset show that, even with only 5% labeled data, our method significantly outperforms existing techniques. This demonstrates the effectiveness of the proposed STCL model in utilizing video spatio-temporal information, achieving good results even with limited labeled data.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Yimin Lin, Qian He, Biao Guo, and Qinghe Dong "Video action detection based on spatio-temporal contrastive learning", Proc. SPIE 13272, Fifth International Conference on Computer Vision and Data Mining (ICCVDM 2024), 132721H (3 October 2024); https://doi.org/10.1117/12.3048411

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available