A virtual keyboard

Some days ago I came up with the idea of a keyboard that can be used without having to type the letters. The user just stands in front of a camera and scrolls the letters by moving his head or hands to the appropriate direction…

And here it is:

http://www.youtu.be/JS–z8Cx9sQ

The steps:

First, I wrote a program using C++, Xlib and OpenGL in order to scroll letters with the mouse and “select” them by pressing a key:

Then, I made a small test that calculates the global motion orientation:

And finally, combined the two in a program that waits for Xevents and camera events using posix threads and unix pipes:

—

I didn’t like very much the motion detection part and I decided to replace the global orientation angle with the global motion vector orientation (calculated by the optical flow). Here is a small test that calculates the optical flow using the Lucas-Kanade method:

I added this code to my virtual keyboard… 🙂

and I think now it works much better xD