Ambrose Kam, Applying Multi-Agent Reinforcement Learning (MARL) in a Cyber Wargame Engine

Cybersecurity is inherently complicated due to the dynamic nature of the threats andever-expanding attack surfaces.  Ironically,this challenge is exacerbated by the rapid advancement of many new technologieslike Internet of Things (IoT) devices, 5G infrastructure, cloud-basedcomputing, etc.  This is where artificialintelligence (AI) and machine learning (ML) techniques can be called intoservice, and provide potential solutions in terms of threat detection andmitigation responses in a rapidly changing environment.  On contrary, humans are often limited by theirinnate inability to process information and fail to recognize/respond to attackpatterns in the multi-dimensional, multi-faceted world.  The recent DARPA AlphaDogFight has proven AIpilots can defeat even the best human pilot in air-to-air combat.  This prompted our engineers to develop aminimum viable product (MVP) that demonstrates the value of a multi-agent reinforcementlearning (MARL) architecture in a simulated cyber wargaming environment.   By using our simulation framework, we essentially"trained" the learning agents to produce the optimum combination/permutation ofcyber attack vectors in a given scenario. This cyber wargaming engine allows our analysts to examine tactics,techniques and procedures (TTPs) potentially employed by our adversaries.  Once these vulnerabilities are analyzed, ourcyber protection team (CPT) can close security gaps in the system. About the speaker: Ambrose Kam is a Lockheed Martin Fellow with over 25 years of experience in the Department of Defense (DoD) industry. He is one of the earliest pioneers at applying modeling, simulation, and operations analysis techniques to threat modeling and cyber resiliency assessment. He regularly gives lectures at MIT, Georgia Tech, and industry consortiums like the Military Operations Research Society (MORS) and National Defense Industry Association (NDIA). Ambrose has been quoted in major publications including Forbes, The Economist, etc, and has co-authored a book in Simulation and Wargames. As a subject matter expert, he represents Lockheed Martin in industry standards organizations like ISO, LOTAR, and INCITS. His most recent efforts in wargaming, Machine Learning/Deep Learning, Cyber Digital Twin, and Blockchain earned him patents and trade secret awards. In 2017, Ambrose won the prestigious Asian American Engineer of the Year (AAEOY) award for his technical leadership and innovations. He holds several advanced degrees from MIT and Cornell University as well as a Bachelor of Science degree from the University at Buffalo.