TCP-QNCC: congestion control algorithm based on deep Q-network

Xiang Huang; Hua Zhu; Li Yang; Chunlin Yin; Jie Li; Hang Li

doi:10.1117/12.3009403

16 October 2023 TCP-QNCC: congestion control algorithm based on deep Q-network

Xiang Huang, Hua Zhu, Li Yang, Chunlin Yin, Jie Li, Hang Li

Proceedings Volume 12803, Fifth International Conference on Artificial Intelligence and Computer Science (AICS 2023); 1280308 (2023) https://doi.org/10.1117/12.3009403
Event: 2023 5th International Conference on Artificial Intelligence and Computer Science (AICS 2023), 2023, Wuhan, China

Abstract

When the demand for network resources exceeds the available capacity of the network, network congestion occurs on the Internet. Network congestion can cause transmission delays, packet loss, and even congestion collapse. When this happens, all communication across the network comes to a halt. Therefore, congestion has long been a concern. Although traditional congestion control algorithms have performed well in early simple networks, their limitations prevent them from being effective before network congestion occurs, making them difficult to cope with the challenges of network complexity and service diversification. In this paper, we model network congestion control as a Markov decision process and optimize congestion control strategies using a deep Q network in deep reinforcement learning, proposing a congestion control algorithm, QNCC, that is purely data-driven and does not rely on any assumptions. QNCC uses a fully connected neural network to approximate the value function, enabling it to automatically learn features and have good generalization ability. Experiments show that QNCC performs better overall than traditional congestion control algorithms such as TCP-Cubic and TCP-Vegas in multiple network scenarios.

(2023) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Xiang Huang, Hua Zhu, Li Yang, Chunlin Yin, Jie Li, and Hang Li "TCP-QNCC: congestion control algorithm based on deep Q-network", Proc. SPIE 12803, Fifth International Conference on Artificial Intelligence and Computer Science (AICS 2023), 1280308 (16 October 2023); https://doi.org/10.1117/12.3009403

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available