Reinforcement Learning-based Model Predictive Control for Autonomous Truck Driving

Abstract

Autonomous vehicles have gained significant attention in recent years due to their potential to improve transportation safety, efficiency, and operational flexibility. In addition to increasing driver comfort and reducing human error, autonomous driving systems can enable continuous operation and improve fuel efficiency, which is particularly important for heavy-duty vehicles such as trucks. However, autonomous truck control remains a challenging problem due to complex traffic interactions, strict safety requirements, and the need for long-term decision-making under computational constraints. This thesis investigates a hybrid control framework that combines Model Predictive Control (MPC) with Reinforcement Learning (RL) for autonomous truck control in highway driving scenarios. The proposed approach aims to exploit the safety, interpretability, and constraint-handling capabilities of MPC while benefiting from the long-term prediction and computational efficiency provided by RL-based value approximation methods. We implement an MPC controller with various safety and operational constraints for driving in a simulated highway environment. A Deep Q-Network (DQN) based RL model is trained to estimate the value function, providing a faster and more accurate approximation of terminal cost. We also compare the performance of proposed framework with various variants such as rollout-based approximations. Furthermore, uncertainties and disturbances in the traffic environment are incorporated to investigate the robustness of the proposed framework under noisy and unpredictable conditions. The results demonstrate that the proposed RL-MPC framework can successfully satisfy safety and operational constraints in different driving scenarios while improving long-term prediction and reducing overall control cost. The study further highlights the importance of terminal cost approximation and uncertainty handling in achieving robust and computationally efficient autonomous truck control.