I'm posting here some updates of the Google Summer of Code EHCI project. This part of the project deals with head tracking with 6 degrees of freedom, a problem often referred as finding the pose of an object. Since no light is being generated from the head - as in some types of infra-red tracking - it needs to rely on natural features of the head. This implementation tries to follow the excellent work from Luca Vacchetti, Vincent Lepetit, and Pascal Fua, from the Computer Vision Laboratory of the Swiss Federal Institute of Technology (EPFL), "Fusing Online and Offline Information for Stable 3D Tracking in Real-Time". The paper is available here
There's a video on youtube showing current progress.
The algorithm starts automatically looking for a head in the image, through the famous Viola Jones algorithm.
After finding the head position, a feature tracking algorithm is started. It uses cvFindGoodFeatures to track in the region of interest defined by the head width and height. When these features are discovered, they are mapped back to a head model (I'm currently using a cylindrical model, but I plan to use the excellent head model by Len Van Der Westhuizen, which is available here, thanks Len!).
When the head model 3d points are known, as well as its corresponding 2d image points, DeMenthon's POSIT algorithm is used to find the initial pose estimation.
After that, an optical flow algorithm by Lucas-Kanade is used is used to track the points along the frames. These points are mapped back to original 3d points and the pose matrix is updated.
The source code shows how to deal with several important OpenCV functions, such as cvGoodFeaturesToTrack, cvCreatePOSITObject, cvPOSIT, and cvCalcOpticalFlowPyrLK, as well as some interesting OpenGL features like loading custum Model View, and Projection matrixes through glLoadMatrix.
I'd really like to thank God and everyone that has helped me develop this work with invaluable tutorials, papers, 3d models, and e-mails,
Explanation of the raw format: http://local.wasp.uwa.edu.au/~pbourke/dataformats/povraw/
The full report is available at http://code.google.com/p/ehci/wiki/6dofhead