Latent Space Control in Autoencoders for Synthetic Face Generation in Driver Monitoring System Validation

Abstract

Driver errors are the primary cause of road traffic accidents, often resulting from inattention or distraction. Most modern cars are equipped with camera-based driver monitoring systems (DMS) to estimate the driver’s state, helping to minimize the risk of such accidents. Validation of the DMS requires large amounts of expensive data of driver faces to cover common driving scenarios. By simulating these scenarios with synthetic data, one could potentially improve the validation process. The investigated idea is to use various setups of autoencoders to generate synthetic data, with the possibility to control latent variables such as head position and rotation. The controllability is achieved through a proposed training step where the latent variables are swapped, enabling the autoencoders to have a structured latent space containing a steerable position or rotation representation. The results are benchmarked against a generative model called LivePortrait, and the compatibility of the synthetic data with existing open-source tracking software is investigated. The results demonstrate that the proposed model is capable of generating synthetic videos that are compatible with Google’s head rotation tracking algorithm from the MediaPipe framework. To enhance the practical value of these models, future work should focus on evaluating the synthetic videos using tracking algorithms from a real DMS and extending the model to allow for controlling eye gaze direction.