Multi-agent autonomous battle management using deep neuroevolution

Sean Soleyman; Fan Hung; Deepak Khosla; Yang Chen; Joshua G. Fadaie; Navid Naderi

doi:10.1117/12.2585530

12 April 2021 Multi-agent autonomous battle management using deep neuroevolution

Sean Soleyman, Fan Hung, Deepak Khosla, Yang Chen, Joshua G. Fadaie, Navid Naderi

Proceedings Volume 11758, Unmanned Systems Technology XXIII; 117580C (2021) https://doi.org/10.1117/12.2585530
Event: SPIE Defense + Commercial Sensing, 2021, Online Only

Abstract

Next-generation autonomous vehicles will require a level of team coordination that cannot be achieved using traditional Artificial Intelligence (AI) planning algorithms or Machine Learning (ML) algorithms alone. We present a method for controlling teams of military aircraft in air battle applications by using a novel combination of deep neuroevolution with an allocation-based task assignment algorithm. We describe the neuroevolution techniques that enable a deep neural network to evolve an effective policy, including a novel mutation operator that enhances the stability of the evolution process. We also compare this new method to policy gradient Reinforcement Learning (RL) techniques that we have utilized in previous work, and explain why neuroevolution presents several benefits in this particular application domain. The key analytical result is that neuroevolution makes it easier to select long sequences of actions following a consistent pattern, such as a continuous turning maneuver that occurs frequently in air engagements. We additionally describe multiple ways in which this neuroevolution approach can be integrated with allocation algorithms such as the Kuhn- Munkres Hungarian algorithm. We explain why gradient-free methods are particularly amenable to this hybrid approach and open up exciting new algorithmic possibilities. Since neuroevolution requires thousands of training episodes, we also describe an asynchronous parallelization scheme that yields order of magnitude speedup by evaluating multiple individuals from the evolving population simultaneously. Our deep neuroevolution approach out-performs human-programmed AI opponents with a win rate greater than 80% in multi-agent Beyond Visual Range air engagement simulations developed using AFSIM.^†

Conference Presentation

Citation Download Citation

Sean Soleyman, Fan Hung, Deepak Khosla, Yang Chen, Joshua G. Fadaie, and Navid Naderi "Multi-agent autonomous battle management using deep neuroevolution", Proc. SPIE 11758, Unmanned Systems Technology XXIII, 117580C (12 April 2021); https://doi.org/10.1117/12.2585530

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available