Comparing Waymo’s ‘Surprise’-Based Metrics with Classic Surrogate Safety Metrics: A Scenario-Based Comparative Study

Abstract

Traffic safety evaluation for automated driving requires methods that can identify potentially critical scenarios before crashes occur. Classical surrogate safety metrics, such as Time Headway (THW), Time-to-Collision (TTC), and Deceleration Rate to Avoid a Crash (DRAC), characterize physical conflict mainly in terms of distance, relative speed, and required braking. However, risk can also arise when surrounding road users behave in ways that deviate from the ego vehicle’s expectations. This the sis developed a computational framework for quantifying “surprise” in road traffic environments and investigated whether surprise-based metrics could provide com plementary information for identifying risky highway events, with a particular focus on cut-ins. To quantify surprise, a kinematics-based belief-generation model was developed in an ego-centered coordinate system to estimate the expected future motion of surround ing vehicles. Under a constant-velocity assumption, the model generated a Gaussian belief distribution over future vehicle states. Surprise was then computed by com paring the predicted belief with observed vehicle behavior using four surprise-based metrics: Shannon Surprisal, Residual Information, Bayesian Surprise, and Antithe sis. The method was evaluated using naturalistic highway trajectory data from the highD dataset. Surprise thresholds were derived from normal-driving baseline data from highD, and the resulting surprise-based detections were compared against detections obtained from a THW-based classical screening method and manually la belled risky cut-in events. The results showed that the THW-based classical baseline and the surprise-based approach were strongly related but not identical. The overlap analysis identified 941 shared events out of 1,225 unique detected events, corresponding to a Jaccard overlap of 0.7682. This indicated that many risky cut-ins contained both physical closeness and unexpected motion. The precision–recall comparison showed that the THW-based score achieved the highest average precision, with an AP of 0.982. Among the surprise-based metrics, Antithesis performed best with an AP of 0.949, followed by Bayesian Surprise with 0.912 and Residual Information with 0.781. The sensitivity analysis further showed that the history window had a clearer influence on the surprise-based results than the lookahead time. Overall, the findings confirm that surprise-based metrics can be useful for describing unexpected vehicle behaviour, but they should not be interpreted as direct replace ments for the THW-based classical baseline. Instead, they could serve as comple mentary indicators to specifically capture surprise, but from the results it is clear (and expected) that the surprise metrics do not capture situational urgency. A main limitation of the work was that the belief model used in this thesis was based on a simple constant-velocity assumption. Future work should investigate more advanced belief-generation models, such as machine-learning-based or interaction aware prediction models, to evaluate whether improved prediction can lead to more reliable surprise-based safety assessment.