A learning from demonstration approach for DORA: A Dexterous Robot Assistant

Abstract

Mobile manipulators are robotic systems with a mobile base and a robotic arm. They are increasingly useful for tasks performed in unstructured and dynamic environments. This project enhances the capabilities of a mobile manipulator called DORA, by incorporating imitation learning.

Preliminary experiments are conducted to assess DORA’s existing capabilities in both simulated and real-world environments. These tests confirm that DORA can successfully execute tasks based on its current functionality.

Imitation learning, specifically Learning from Demonstration (LfD), enables DORA to learn new tasks by replicating the movements of a human demonstrator. A teleoperation setup, consisting of an AprilTag and MoCap gloves, is used to collect four dimensional data capturing the robot’s 3D position and open/close state of the gripper.

The collected 4D data is used to train an imitation learning algorithm in a simulation environment. The comparison between the Dynamic Time Warping (DTW) and Fréchet distance (FD) scores for the training and test data shows that the trained imitation learning model generalizes well to the demonstrated motion patterns. In conclusion, this thesis lays a solid foundation for further improving DORA’s dexterity through imitation learning.