Hand gesture recognition in real time - Fast, accurate hand gesture recognition of static and dynamic hand gestures for on device classification

Abstract

Human machine interaction (HMI) is an important part of the entertainment industry in that it allows developers to create an engaging experience. However when it comes to contactless interaction through hand gestures, it still remains a challenge to develop algorithms that are both accurate and fast enough to run in real-time on device [1]. In this report, this challenge have been explored using MediaPipe hand landmark detector as a feature extraction algorithm coupled with different classifiers trained to recognize both static and dynamic hand gestures. Result show the application can run both in real-time, and achieve relatively high accuracy of 88.1% for dynamic hand gestures and 93.1% on static hand gestures on large scale datasets. Furthermore, fine-tuning the dynamic hand gesture recognition algorithm to a specific user, was shown to improve the accuracy to 97.5%.