Adaptive Model-Free Control Applied to Truck Front Wheel Drive: Real time control with reinforcement learning utilising recurrent deterministic policy gradient

Abstract

This thesis investigates how reinforcement learning methods can be used to achieve adaptive and model-free control of a hydraulic front-wheel drive system. First, an existing sub-optimal controller is emulated by an artificial neural network using supervised learning. The continuous action- and state space reinforcement learning method “Recurrent Deterministic Policy Gradients” (RDPG) is then modified to work continuously in real time and implemented to improve the performance of the network and make it adaptive. The emulating network performed similarly, albeit somewhat worse, compared to the original controller. Using RDPG with the emulating network against a simple model of the hydraulic system showed that the network adapted and further improved the performance from the sub-optimal starting point. However, applying the RDPG algorithm against a real system was infeasible with the selected hyper-parameters and would require further investigation for the algorithm to converge. The conclusion is that using RDPG for adaptive model-free control can be feasible for non-linear dynamic system that exhibit slow, gradual changes and that first emulating a sub-optimal controller can enable learning on a system where learning from the beginning is not desired or possible.