On stochastic optimization for deep learning

S. Volkova

doi:10.1117/12.3024943

20 February 2024 On stochastic optimization for deep learning

S. Volkova

Proceedings Volume 13065, Third International Conference on Optics, Computer Applications, and Materials Science (CMSD-III 2023); 130650H (2024) https://doi.org/10.1117/12.3024943
Event: Third International Conference on Optics, Computer Applications, and Materials Science (CMSD-III 2023), 2023, Dushanbe, Tajikistan

Abstract

Various stochastic optimization methods are utilized for training neural networks. The objective of this research is to provide a comprehensive overview of stochastic optimization methods proposed to enhance and expedite the convergence of neural networks. The article presents a highlighting of stochastic optimization methods' advantages and drawbacks, while also analyzing the constraints of their applicability. It commences with an introduction to the problem formulation, followed by sections dedicated to various algorithmic modifications: SGD-based stochastic optimization methods, adaptive gradient methods, and methods of adaptive moment estimation. In conclusion, the article underscores the importance of a judicious selection of a method, contingent upon its characteristics, applicability constraints, specific task, model architecture, and data quality.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

S. Volkova "On stochastic optimization for deep learning", Proc. SPIE 13065, Third International Conference on Optics, Computer Applications, and Materials Science (CMSD-III 2023), 130650H (20 February 2024); https://doi.org/10.1117/12.3024943

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available