A Natural User Interface and Touchless Interaction. Approach on Web Browsing

Abstract

The aim of this Master’s Thesis is to explore the principles of Natural Interaction and suggest an implementation of a web browsing application that uses them. The research that takes place is examining the ingredients of Natural User Interfaces and Touchless Interaction with a focus on Gestures and Voice. How can those be combined to control an interface? What are the most important things to mind when designing Natural Interfaces and Interactions? These are questions that this report attempts to answer. The final result of the thesis is a prototype web browser application that works only with gestures and voice. The prototype simulates browsing on a number of created web pages. It is implemented using the Microsoft Kinect for XBox sensor and Kinect SDK v1.7 with all the included tools and interaction controls. The browser is undergoing a usability test and the participants are observed and later interviewed to describe their experience. Based on that, the conclusions drawn are that designing an application that uses a Natural Interface needs to provide short and comfortable Gesture and Speech Vocabularies and Feedback mechanisms that inform the user at every point of the interaction. Furthermore, it is concluded that the design of the content for such an application should be different than the traditional design of a webpage for optimal use while responding to implicit input (contextawareness) adds to the naturalness of the interaction. Finally, the setting for using such an application is recommended to be part of a home entertainment system (e.g. smart TV) or public space installation rather than a desktop computer.