High-speed train automatic stopping control method based on deep reinforcement learning

Shengquan Ma; Wei Shan; Hai Deng; Xianyi Xie; Wentao Zhu; Lisheng Jin

doi:10.1117/12.3030703

16 May 2024 High-speed train automatic stopping control method based on deep reinforcement learning

Shengquan Ma, Wei Shan, Hai Deng, Xianyi Xie, Wentao Zhu, Lisheng Jin

Proceedings Volume 13160, Fourth International Conference on Smart City Engineering and Public Transportation (SCEPT 2024); 131601C (2024) https://doi.org/10.1117/12.3030703
Event: 4th International Conference on Smart City Engineering and Public Transportation (SCEPT 2024), 2024, Beijin, China

Abstract

Train Automatic Stop Control (TASC) system based on reinforcement learning(RL) is a crucial technology to improve self-driving performance for automatic train operation(ATO). Although significant progress has been achieved, existing RL-TASC systems still face challenges related to sparse rewards and lower control precision. In this paper, we design a train automatic stop strategy based on the Deep Q-Network (DQN) algorithm with both global rewards and single-step rewards to enhance the precision and training stability of RL-TASC systems. Initially, a single-point motion equation for the train was established and the state-action space was defined. Subsequently, a global reward function was designed based on stopping error. To address the challenges posed by sparse global rewards, which led to poor training effects, and imprecise stopping performance, a method for setting a single-step reward function based on expected velocity was proposed. Finally, through simulation experiments, the results showed that the improved control method could maintain the train stopping error within 0.05 meters. Comparing to the pre-improved model, the average value of stopping error decreased by 76.56% and the standard deviation of stopping error decreased by 67.74%, confirming the effectiveness of the improved method. This paper provides a high-precision and highly robust method for automatic stopping control in high-speed train ATO systems.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Shengquan Ma, Wei Shan, Hai Deng, Xianyi Xie, Wentao Zhu, and Lisheng Jin "High-speed train automatic stopping control method based on deep reinforcement learning", Proc. SPIE 13160, Fourth International Conference on Smart City Engineering and Public Transportation (SCEPT 2024), 131601C (16 May 2024); https://doi.org/10.1117/12.3030703

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available