Reinforcement learning A comparison of learning agents in environments with large discrete state spaces

Abstract

For some real-world optimization problems where the best behavior is sought, it is infeasible to search for a solution by making a model of the problem and performing calculations on it. When this is the case, good solutions can sometimes be found by trial and error. Reinforcement learning is a way of finding optimal behavior by systematic trial and error. This thesis aims to compare different reinforcement learning techniques and evaluate them. Model-based interval estimation (MBIE) and Explicit Explore or Exploit using dynamic bayesian networks (DBN-E3) are two algorithms that are evaluated. To evaluate the techniques, learning agents were constructed using the algorithms and then simulated in the environment Invasive Species from the Reinforcement Learning Competition. The results of the study show that an optimized version of DBN-E3 is better than MBIE at finding an optimal or near optimal behavior policy in Invasive Species for a selection of environment parameters. Using a factored model like a DBN shows certain advantages operating in Invasive Species, which is a factored environment. For example it achieves a near optimal policy within fewer episodes than MBIE.