Wii remote based head tracking in 3D audio rendering

Abstract

The 3D audio scene rendered for a listener moving in a 3D environment depends directly on the listener's position and orientation in the three dimensional space. Tracking the listener's position and orientation is, therefore, an important component in interactive virtual environments.

By mounting four diodes on the listener's headset, a stationary Wii Remote can be used to track the head movements of the listener (both position and orientation) and provide the 3D audio rendering motor with the necessary 6DOF data on the listener's movements to do a fully interactive rendering of the audio scene.

In this project, a head tracking solution is developed and is integrated into a 3D audio rendering engine to obtain an interactive virtual environment. Head pose estimation is the central part in the head tracking algorithm. Prior to tracking, 3D feature points are obtained and used as a model. Pose is then estimated by solving a robust version of "Perspective n-Point Problem". In order to analyze the accuracy of the algorithm, results were validated primarily by simulation of a random walk movement of a listener in MATLAB and used as the input to the head tracking algorithm. Thereafter, the solution was compiled as a library in C and tested in real time tracking. A graphical output is used to demonstrate the applicability of our method in real time application. The final solution is integrated into a 3D audio rendering engine as a part of the virtual environment.

In relation to a closely related work by Johnny Chung Lee on Wii-Remote for head tracking (can be found in YouTube), this work tracks the full 6D0F movement of the listener (translation + orientation) while Chung lee's work only tracked the translation (3DOF) assuming that the orientation was the negated translation.