Fusion of Human Context for Vehicle Trajectory Prediction - Incorporating Driver Monitoring Data for Short-Horizon ADAS Applications

Abstract

Short-horizon ego-vehicle trajectory prediction is a key component of Advanced Driver Assistance Systems, particularly in situations where driver behavior directly influences vehicle motion. Classical kinematic models are limited in such cases, as they rely solely on observed motion and do not account for anticipatory cues related to driver awareness. This thesis investigates whether gaze information from a Driver Monitoring System can improve short-term trajectory prediction without relying on environmental perception. A learning-based model, termed Gaze-Aware Dual-Stream Trajectory Prediction (GA-DSTP), is proposed. The model predicts 3.5 seconds future ego-vehicle trajectories by jointly leveraging temporal histories of vehicle dynamics, actuator inputs, and driver gaze data. Vehicle-centric and driver-centric information are processed in separate streams and fused to capture both physical motion tendencies and driver-based indicators of upcoming maneuvers. A multi-modal prediction formulation is used to represent uncertainty in future motion. The model is trained and evaluated on a self-collected real-world driving dataset and compared against classical kinematic baselines as well as the same model trained without DMS data. GA-DSTP achieves a 37% reduction in Average Displacement Error (ADE) compared to the Constant Turn Rate and Acceleration (CTRA) model and over 60% compared to the Constant Velocity (CV) model. It also achieves a 2.1% lower ADE than the model not using DMS data, indicating that gaze information provides complementary predictive value beyond vehicle dynamics alone. The results demonstrate that driver gaze provides complementary information to vehicle dynamics and can enhance the robustness of ego-vehicle trajectory prediction for ADAS applications.