WebCam + OpenGL + OpenCV head tracking = Immersive 3d environment

UPDATE! This sample has evolved to a project sponsored by Google. Please check http://code.google.com/p/ehci

This is my first attempt to create a 3d immersive environment using a simple webcam to track head's position.



This video shows my approach to combine OpenCV head tracking with a 3d OpenGL environment so that the user will see objects from their head's point of view. Since head tracking is 2d, no depth information is obtained. Besides that, it takes the face recognition algorithm around 200ms, which yields low fps (about 5). I'm currently trying to improve that :)

*******
Well, thanks to Walter Piechulla I could decrease detection time to around 20ms using the flag CV_HAAR_FIND_BIGGEST_OBJECT - which makes opencv detect only one face - in a parameter to the function cvHaarDetectObjects, that looks like this now:

cvHaarDetectObjects( small_img, cascade, storage, 1.1, 2, CV_HAAR_FIND_BIGGEST_OBJECT,
cvSize(30, 30) );

(by the way, I had to update OpenCV from CVS)
As soon as I get some time, I can make another video showing speedup :D
Thanks Piechulla

********
The new version is available here: http://danielbaggio.blogspot.com/2008/03/enhanced-version-of-head-tracking-and.html