A Curriculum Framework for Autonomous Network Defense using Multi-agent Reinforcement Learning
Publication Date
1-1-2023
Document Type
Conference Proceeding
Publication Title
2023 Silicon Valley Cybersecurity Conference, SVCC 2023
DOI
10.1109/SVCC56964.2023.10165310
Abstract
Early threat detection is an increasing part of the cybersecurity landscape given the growing scale and scope of cyberattacks in the recent years. Increasing exploitation of software vulnerabilities, especially in the manufacturing sector, demonstrates the ongoing need for autonomous network defense. In this work, we model the problem as a zero-sum Markov game between an attacker and defender reinforcement learning agents. Previous methods test their approach on a single topology or limit the agents to a subset of the network. However, real world networks are rarely fixed and often add or remove hosts based on demand, link failures, outages, or other factors. We consider two types of topologies: static topologies that remain fixed throughout training and a dynamic topology curriculum. The proposed robust training curriculum incorporates network topologies to build more general, capable agents. We also use Proximal Policy optimization (PPO) which offers a good balance of computational complexity and convergence speed. We evaluate various threat scenarios in terms of the exploitability and impact and conclude that the curriculum improves the defender's win rate over training on a static topology by exposing the agent to more challenging environments over time.
Keywords
autonomous network defense, curriculum learning, cybersecurity, MARL, Multi-agent reinforcement learning, self-play
Department
Computer Engineering
Recommended Citation
Robert G. Campbell, Magdalini Eirinaki, and Younghee Park. "A Curriculum Framework for Autonomous Network Defense using Multi-agent Reinforcement Learning" 2023 Silicon Valley Cybersecurity Conference, SVCC 2023 (2023). https://doi.org/10.1109/SVCC56964.2023.10165310
