Imaging Components, Systems, and Processing

Weighing classes and streams: toward better methods for two-stream convolutional networks

[+] Author Affiliations
Hoseong Kim, Youngjung Uh, Seunghyeon Ko, Hyeran Byun

Yonsei University, Department of Computer Science, Seoul 120-749, Republic of Korea

Opt. Eng. 55(5), 053108 (May 23, 2016). doi:10.1117/1.OE.55.5.053108
History: Received January 28, 2016; Accepted April 11, 2016
Text Size: A A A

Abstract.  The emergence of two-stream convolutional networks has boosted the performance of action recognition by concurrently extracting appearance and motion features from videos. However, most existing approaches simply combine the features by averaging the prediction scores from each recognition stream without realizing that some classes favor greater weight for appearance than motion. We propose a fusion method of two-stream convolutional networks for action recognition by introducing objective functions of weights with two assumptions: (1) the scores from streams do not weigh the same and (2) the weights vary across different classes. We evaluate our method by extensive experiments on UCF101, HMDB51, and Hollywood2 datasets in the context of action recognition. The results show that the proposed approach outperforms the standard two-stream convolutional networks by a large margin (5.7%, 4.8%, and 3.6%) on UCF101, HMDB51, and Hollywood2 datasets, respectively.

Figures in this Article
© 2016 Society of Photo-Optical Instrumentation Engineers

Topics

Networks ; Video

Citation

Hoseong Kim ; Youngjung Uh ; Seunghyeon Ko and Hyeran Byun
"Weighing classes and streams: toward better methods for two-stream convolutional networks", Opt. Eng. 55(5), 053108 (May 23, 2016). ; http://dx.doi.org/10.1117/1.OE.55.5.053108


Access This Article
Sign in or Create a personal account to Buy this article ($20 for members, $25 for non-members).

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Related Book Chapters

Topic Collections

PubMed Articles
Advertisement
  • Don't have an account?
  • Subscribe to the SPIE Digital Library
  • Create a FREE account to sign up for Digital Library content alerts and gain access to institutional subscriptions remotely.
Access This Article
Sign in or Create a personal account to Buy this article ($20 for members, $25 for non-members).
Access This Proceeding
Sign in or Create a personal account to Buy this article ($15 for members, $18 for non-members).
Access This Chapter

Access to SPIE eBooks is limited to subscribing institutions and is not available as part of a personal subscription. Print or electronic versions of individual SPIE books may be purchased via SPIE.org.