This video shows my enhanced approach - using CV_HAAR_FIND_BIGGEST_OBJECT - to combine OpenCV head tracking with a 3d OpenGL environment so that the user will see objects from their head's point of view. Since this headtracking is 2d, no depth information has been obtained still, although I'm planning it for some next post. Now, it takes around 20 ms for the face recognition algorithm, which gives a nice refresh rate.
Thanks to Vadim Pisarevsky for the function.