Comparative Evaluation of Detection Methods for Virtual Objects
Typ
Examensarbete för masterexamen
Master's Thesis
Master's Thesis
Program
Systems, control and mechatronics (MPSYS), MSc
Publicerad
2024
Författare
Horngacher, William
Modellbyggare
Tidskriftstitel
ISSN
Volymtitel
Utgivare
Sammanfattning
Abstract
With increasing global military spending, there is a growing need for modern training tools that can simulate realistic combat conditions without the risks and costs associated with live ammunition. SAAB Training and Simulation addresses this need with innovative laser-based systems that can be mounted on various weapon platforms. In certain situations, there is a need to provide real-time visual feedback by displaying virtual projectiles and explosions on a video screen, such as inside vehicles, enhancing training realism. This project aims to create automated testing capabilities for the video-generating unit of this system by creating a program that can detect the projectiles and explosions in the video feed.
Two different object detection models were tested. One was an algorithmic detection model called the Round Object Detector, which utilizes the Hough circle algorithm due to its efficiency in detecting circular shapes in an image. The other
model was the Faster Region-Based Convolutional Neural Network (Faster R-CNN) for its capability to identify complex features. A synthetic dataset with diverse backgrounds was created to train the Faster R-CNN, along with three manually annotated datasets from the video-generating unit for model evaluation. A selective detection technique enhanced detection accuracy by adjusting prediction confidences based on expected projectile positions. The Faster R-CNN displayed high precision across various backgrounds in the video generator tests, although recall varied, signifying challenges in consistent object detection. The Round Object Detector achieved high precision with simple backgrounds but struggled with detailed, colorful settings due to limitations in color filtering, leading to increased false detections. Selective detection marginally enhanced overall model performance, particularly improving the Faster R-CNN’s recall.
The project showed that the Faster R-CNN model could transfer its knowledge well from the synthetic dataset to the analog video frames. The false positive predictions were still few, but the false negative rate increased, which showed that fine-tuning the model on an annotated dataset from the real application could be a way to improve performance. The Round Object Detector performed significantly worse than the Faster R-CNN model, and it proved to be the color filter that was the main problem to get working correctly. Selective detection proved to be a useful tool for the Faster R-CNN, where it improved recall, which was the main challenge for both models when used on the video sequence.
Beskrivning
Ämne/nyckelord
Keywords: Object Detection, Faster R-CNN, Synthetic Dataset, Hough Circle Algorithm, Selective Detection.