Understanding Linear Quadratic Drone Games through Simulation: Linear Quadratic Games as a Baseline for Evaluating Multi- Agent Reinforcement Learning Algorithms and Simulation as a Tool for Understanding and Innovation

Abstract

A relevant method of generating control policies for autonomous drones is through multi-agent reinforcement learning (MARL) algorithms. Novel MARL algorithms do not always have theoretical guarantees of convergence which motivates the need for robust and reliable baselines to benchmark these algorithms against. This thesis investigates the efficacy of derived Nash equilibrium (NE) solutions to the family of linear quadratic (LQ) games as one such possible baseline. The first part of the thesis derives the Nash equilibrium solutions for LQ games, which serve as the baseline policies. Based on these results, two experimental scenarios are designed to benchmark MARL algorithms against the baseline. The experimental results indicate that the MARL policies perform on par with the baseline in the two-player scenario, while outperforming the baseline in the cooperative five-player scenario. The second part of the thesis explores to what extent computer simulation can be used as an effective knowledge sharing method at an engineering company. An exploratory pilot study was conducted comparing two learning sessions, one session employing an online simulation tool developed for this purpose and the other being a traditional lecture. Responses collected after each learning session indicate that the visual and interactive elements of the simulation tool were conducive to generating engagement and curiosity among participants. Furthermore, providing necessary context and examples of applicability were deemed important aspects when sharing information about a novel topic among engineers.