Multi-Agent Deep Reinforcement Learning in a Three-Species Predator-Prey Ecosystem

Abstract

In computational biology, population dynamics in simulated ecosystems is one im portant topic. Standard mathematical tools of population dynamics such as systems of differential equations are typically incapable of accounting for a variety of impor tant attributes, such as the intelligent and adaptive behavior of individual agents in complex environments. Thus, they are often insufficient to simulate dynamics in real-world ecosystems. In this thesis, a three-species predator-prey simulated ecosystem was implemented in the Unity game engine. Agents in the ecosystem were trained through multi-agent reinforcement learning. The population dynam ics were then analysed with respect to the Lotka-Volterra predator-prey equations which are described by several parameters and assumptions regarding the responses of the parameters to changing population densities. The responses of the parameters were estimated through simulation experiments. It was found that the population dynamics of an ecosystem with trained agents exhibited Lotka-Volterra cycles where a random policy agent ecosystem failed to do so. Further, it was shown that the observed responses of the parameters did not fulfill the Lotka-Volterra assumptions, but rather showed properties that could be argued to be more realistic. For the reinforcement learning, a reward system was introduced as the happiness network, which incorporated both the external and internal state of the animat, inspired by behavioral science of real-world animals. This reward system was shown to perform better than a simple reward system with a positive reward for eating food and a negative for dying, in some environments and was argued to have benefits in more complex ecosystems.