Using Robots to Collect Data for Machine Vision Tasks

Abstract

The current final assembly process of automotive wire harnesses into vehicles predominantly relies on manual labor and skill. This reliance leads to safety and ergonomic issues when lifting heavy wire harnesses and applying high-pressure manipulations to components for 8 hours a day. This thesis combines collaborative robotics and artificial intelligence to collect connector data for machine vision tasks in the automotive industry, addressing the problem of insufficient data. The research investigates an approach using a robotic setup for automated data collection. The framework includes data acquisition utilizing a UR5 robot and an Intel RealSense D435 camera; robot-camera communication using a Raspberry Pi 4b as a bridge; and an automatic labeling tool. The collected dataset comprises 8 different connectors commonly used in automotive wire harnesses. The resultant datasets (first the manually annotated dataset and second the automatic annotated dataset) are evaluated using YOLOv8, a deep-learning based object detection model. The evaluation results present a higher accuracy (mAP50 = 93.5%) for the manually annotated dataset compared to the automatic labeling approach (mAP50 = 74.4%) which suggests that there is still room for improvement on the automatic labeling tool used. This accuracy difference is concluded to be due to the inability to control lighting conditions in the workspace in the lab.