GAN-based water droplet removal

Abstract

This master thesis project studies how generative adversarial networks (GANs) can perform raindrop removal and reconstruct the scenes in single images. The models were trained by using synthetic droplet datasets, and real droplet datasets. The real droplet datasets were from Qian et al. datasets and the Reeds datasets [1]. The synthetic droplet datasets were generated from ground-truth images from prior datasets. The process was done by using OpenGL. Then the generated images were evaluated by full-reference quality metrics and non-reference quality metrics, such as SSIM and BRISQUE, then tested in object detection by a pre-trained DETR model and evaluated by mean average precision (mAP). After comparing the quantitative quality of the images generated by models trained by real droplet datasets and synthetic datasets, the result showed that the real droplet datasets yield better image quality than the synthetic datasets. In the object detection task, though it can enhance image quality in comparison to the degraded images, the generated images did not improve the result in this aspect. Thus, it was concluded that the synthetic datasets need to be more realistic to be able to reach comparable results as the real droplets. In the object detection task, GANs generated images from the information in the latent space. As a result, there were some objects which were corrupted, and this made the object detection model miss classify the objects. Apparently, it may not be suitable for high precision and safety tasks. In the aspect of the automated evaluation system, this thesis project investigated the design pattern for the conceptual design of the system. However, further functional and non-functional requirements are needed to be clarified for future implementa tio