Communication Relaying Networks Using Autonomous Drones: Solving Cooperative Markov Games Using Multi-Agent Reinforcement Learning

Abstract

Training several autonomous actors to solve cooperative tasks allows for systems with decentralized control and without single points of failure. One example is the use of swarms of drones to build quick and adaptable communication networks, allowing for efficient communication in hard-to-reach areas or dynamic settings. To ensure that these drone swarms operate effectively, robust training methods like reinforcement learning are essential. The problem can be modeled as a Markov game, allowing for multi-agent reinforcement learning (MARL). Multi-agent proximal policy optimization (MAPPO) was used as the training algorithm as it can effectively utilize shared data between all actors during training, and still allow for decentralized decision making during execution. It was found that building these communication relaying networks was possible for a small number of drones in fully observable environments. However, this ability decreased as the number of agents or the complexity of the environment increased.