A World Model Reinforcement Learning Approach for Vehicle Control: A vehicle controller based on DreamerV3 and a research platform for training and evaluating machine learning based controllers

Abstract

The purpose of this thesis was to investigate whether a world-model-based reinforcement learning approach could be used for path following on a small off-road unmanned ground vehicle. The method involved building a 1/10 scale RC car platform equipped with a binocular camera and developing a ROS 2 graph to manage interprocess communication, including pose estimation and vehicle control. A worldmodel reinforcement learning controller was developed by adapting the DreamerV3 implementation for the path-following task. This controller was evaluated across various trajectories, including sine waves and clothoid turns on uneven grass, as well as backtracking on sand. A Pure Pursuit controller was used as a baseline. However, it was not tuned to individual paths and did not represent state-of-the-art controllers. The results indicate that the controller tracked all paths effectively and mostly smoothly, outperforming Pure Pursuit on all tasks in terms of cross-track error, successfully bridging the sim-to-real gap using a limited amount of training data. While it exhibited minor over-adjustments on certain paths and underutilisation of the steering range, the controller demonstrated an emergent behaviour of reversing to adjust its alignment at sharp corners. Consequently, this validated the successful development of the underlying research platform.