Weighing classes and streams: toward better methods for two-stream convolutional networks

Hoseong Kim; Youngjung Uh; Seunghyeon Ko; Hyeran Byun

doi:10.1117/1.OE.55.5.053108

23 May 2016 Weighing classes and streams: toward better methods for two-stream convolutional networks

Hoseong Kim, Youngjung Uh, Seunghyeon Ko, Hyeran Byun

Author Affiliations +

Optical Engineering, Vol. 55, Issue 5, 053108 (May 2016). https://doi.org/10.1117/1.OE.55.5.053108

Abstract

The emergence of two-stream convolutional networks has boosted the performance of action recognition by concurrently extracting appearance and motion features from videos. However, most existing approaches simply combine the features by averaging the prediction scores from each recognition stream without realizing that some classes favor greater weight for appearance than motion. We propose a fusion method of two-stream convolutional networks for action recognition by introducing objective functions of weights with two assumptions: (1) the scores from streams do not weigh the same and (2) the weights vary across different classes. We evaluate our method by extensive experiments on UCF101, HMDB51, and Hollywood2 datasets in the context of action recognition. The results show that the proposed approach outperforms the standard two-stream convolutional networks by a large margin (5.7%, 4.8%, and 3.6%) on UCF101, HMDB51, and Hollywood2 datasets, respectively.

Citation Download Citation

Hoseong Kim, Youngjung Uh, Seunghyeon Ko, and Hyeran Byun "Weighing classes and streams: toward better methods for two-stream convolutional networks," Optical Engineering 55(5), 053108 (23 May 2016). https://doi.org/10.1117/1.OE.55.5.053108

Published: 23 May 2016

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available