Statistical Evaluation of Radar Simulation Models Towards Real Data
Publicerad
Författare
Typ
Examensarbete för masterexamen
Master's Thesis
Master's Thesis
Modellbyggare
Tidskriftstitel
ISSN
Volymtitel
Utgivare
Sammanfattning
This thesis aimed to create a method that could be used by Volvo Car Corporation
(VCC) to statistically evaluate the truthfulness and accuracy of the built-in
radar models used by VCC. Using pre-collected data from European New Car Assessment
Programme (Euro NCAP) scenarios—specifically Car-to-Car Rear moving
(CCRm) and Car-to-Bicyclist Nearside Adult Obstructed (CBNAO) — a method
was developed to extract real world data, enabling the recreation of actual scenarios
within a simulated environment. Furthermore, a comparison between the simulated
and real data was conducted. This comparison was conducted using the statistical
metric Double Validation Metric (DVM), which is a combination of the Area Validation
Metric (AVM), Corrected Area Validation Metric (CAVM), and the model
bias within the simulated data.
The developed method includes parsing of Hierarchical Data Format version 5
(HDF5) files, enabling reading and manipulating these files. It also features a Graphical
User Interface (GUI) that reads one real file and one simulated file, visualizes
the desired detection columns within a given timestamp interval, and a statistical
evaluator that performs all the calculations, plots the Empirical Cumulative Distribution
Function (eCDF), and analyzes their statistical characteristics.
The results demonstrated tendencies pointing towards both reliability and unreliability.
Different parameters showed varying degrees of correlation for different
scenarios, runs, and speeds. However, one clear trend was the high tendency for
detection differences between the simulated model and the real model, with the simulated
model outnumbering the real model. Based on this, the developed method
can be used to some extent to relay information about whether the radar model is
trustworthy. However, its effectiveness heavily depends on the specific area of usage.
In conclusion, the method effectively visualizes the detections, indicating where they
occur and how they differ in number. While the model may not be suitable for final
certification, it can be a valuable tool for statistical approximation of the radar
model’s performance.
Beskrivning
Ämne/nyckelord
Statistical Evaluation, Statistical Metrics, Real Data, Simulated Data, Double Validation Metric, Radar Models, Euro NCAP scenarios.