Real-Time Decoding of Simultaneous Finger Movements by Multi-Label Representation Learning

Abstract

Classification of simultaneous finger movements is a yet unsolved problem in upper limb prosthetic control. The decoding of simultaneous movements is a crucial step towards restoration of function after an amputation, as it would allow for natural and intuitive finger control to be performed by a prosthetic hand in everyday tasks. Various pattern recognition algorithms have been evaluated in research, where most presented methods need extracted features from myoelectric signals. In recent years, deep learning approaches such as Convolutional Neural Networks (CNN) have entered this research field. An interesting property of these networks is that they are not only able to extract features automatically, but also learn how to extract them, known as representation learning. Even though representation learning has revolutionized many research areas, it is yet unknown whether these algorithms could outperform traditional machine learning methods or shallow Feed Forward Neural Networks (FFNN) commonly used in myoelectric signal classification. This project compares two novel CNN architectures: Temporal Convolutional Networks (TCN) and a CNN with Squeeze and Excitation operations (CNN-SE) against an FFNN, decoding simultaneous finger movements. The best performance obtained during real-time tests was through the CNN-SE with an average Motion Test completion rate of 90.5% compared to corresponding 68.5% for the FFNN. Both CNN-SE and TCN further showed higher averages in F1-score, as well as exact match ratio (EMR), compared to the FFNN, indicating representation learning might be beneficial in decoding simultaneous movements.