Multi-Agent Deep Reinforcement Learning in a Three-Species Predator-Prey Ecosystem
Ladda ner
Publicerad
Författare
Typ
Examensarbete för masterexamen
Program
Modellbyggare
Tidskriftstitel
ISSN
Volymtitel
Utgivare
Sammanfattning
In computational biology, population dynamics in simulated ecosystems is one im portant topic. Standard mathematical tools of population dynamics such as systems
of differential equations are typically incapable of accounting for a variety of impor tant attributes, such as the intelligent and adaptive behavior of individual agents
in complex environments. Thus, they are often insufficient to simulate dynamics
in real-world ecosystems. In this thesis, a three-species predator-prey simulated
ecosystem was implemented in the Unity game engine. Agents in the ecosystem
were trained through multi-agent reinforcement learning. The population dynam ics were then analysed with respect to the Lotka-Volterra predator-prey equations
which are described by several parameters and assumptions regarding the responses
of the parameters to changing population densities. The responses of the parameters
were estimated through simulation experiments. It was found that the population
dynamics of an ecosystem with trained agents exhibited Lotka-Volterra cycles where
a random policy agent ecosystem failed to do so. Further, it was shown that the
observed responses of the parameters did not fulfill the Lotka-Volterra assumptions,
but rather showed properties that could be argued to be more realistic.
For the reinforcement learning, a reward system was introduced as the happiness
network, which incorporated both the external and internal state of the animat,
inspired by behavioral science of real-world animals. This reward system was shown
to perform better than a simple reward system with a positive reward for eating food
and a negative for dying, in some environments and was argued to have benefits in
more complex ecosystems.
Beskrivning
Ämne/nyckelord
animats, multi-agent reinforcement learning, ecosystems, lotka-volterra, predator-prey