Falsification of Safety-Critical Decision and Control Software

Abstract

Safety testing of autonomous driving systems requires methods that can expose rare but safety-critical failures beyond nominal driving behavior. Since exhaustive realworld testing is infeasible, simulation-based falsification provides a practical way to actively search for scenarios in which a system under test violates desired safety requirements. This thesis investigates deep reinforcement learning for falsification of safety-critical decision and control software in closed-loop simulation. The falsification task is formulated as an adversarial control problem in a simplified longitudinal two-vehicle scenario, where a learning agent controls the target vehicle and attempts to induce collisions with an ego vehicle controlled by the system under test. A TD3- based falsifier is first evaluated against Monte Carlo Tree Search and uniform random sampling to study collision discovery performance across randomly sampled initial conditions. The thesis then extends the framework towards diversity-oriented falsification using a DTW-based(dynamic time wrapping) Series RL method. In this method, previously discovered collision trajectories are stored in a repository and used to reward later agents for finding trajectories with different gap and relativespeed evolution. A collision replay buffer is also introduced to increase the reuse of rare collision-related experiences during training. The results show that the TD3- based falsifier discovers collisions more effectively than the non-learning baselines in the tested setting. They also show that the DTW-based diversity reward leads to a broader and less repetitive set of collision trajectories than the baseline Series RL method. These findings suggest that reinforcement learning can support both efficient failure discovery and diversity-oriented exploration of failure modes, although the conclusions are limited to the simplified simulation setting considered in this thesis.