Towards Improvement of Human-Machine Interaction: Design of Multimodal Human Intent Recognition System

Abstract

This master thesis focuses on investigating the electrical brain activity, eye gaze and pupil behaviour in the scope of goal-directed movement intention recognition for human-machine interaction applications. Previous studies support that the electroencephalography (EEG) data is suitable for early motion recognition and prediction and the pupil size changes correlate with the difficulty of the task. However few studies have looked into neural correlates of goal-directed and no-goal movements as well as the correlation between the pupil changes, EEG data and hand motion. We explore these questions through a set of cue-based movement experiments that include changing goal, repeating goal and no-goal scenarios and are performed in collaboration with a robot. The results were analysed with regard to movement related cortical potentials (MRCP) and event related spectral perturbation (ERSP) of EEG data, evoked pupil response, gaze patterns as well as binary goal\no-goal classification of the data and correlation between different biosignals. Our results indicate that changing goal-directed movements are distinguishable from no-goal movements in EEG data in both temporal and time-frequency domains, when performing the task with a passive robot. Collaborative robot experiments showed great intersubject variability, therefore need to be further investigated. No correlation between evoked pupil response and MRCP was found in this study, however results suggest a correlation between MRCP and motion velocity profile.