Real-Time Multi-Object Tracking and Segmentation with Generated Data using 3D-modelling

Abstract

Multi-Object Tracking and Segmentation (MOTS) is an important branch of computer vision that has applications in many different areas. In recent developments these methods have been able to reach favorable speed-accuracy trade-offs, making them interesting for real-time applications. In this work different deep learning based MOTS methods have been investigated with the purpose of extending the DeepTrack framework with real-time MOTS capabilities. Deep learning methods rely heavily on the data on which they are trained. The collection and annotation of the data can however be very time-consuming. Therefor, a pipeline is developed and investigated that automatically produces synthetic data by utilizing 3D-modelling. The most accurate tracker achieves a MOTSA score of 94 and the tracker with the best speed-accuracy trade-off achieves a MOTSA score of 88. It is also observed that satisfactory results can be achieved in most situations with a quite general data generation pipeline, indicating that the developed pipeline could be used in different scenarios.