Project: Using a Wiimote to build a Minority Report computer interface

In May I wrote about third party hacks for the Wii Remote (Wiimote). Having spent the intervening period building some the applications posted on the web, I’ve developed a huge amount of respect for what is an incredibly sophisticated piece of kit and yet passes off as a £30 computer game controller.

The Wiimote contains a Bluetooth radio, an infrared camera capable of tracking upto four moving objects, a three-axis linear accelerometer, a joy stick, a motor (used to generate the rumble effect), a battery monitor, a speaker, numerous button inputs and four LEDs.

To begin to understand how extraordinarily clever this piece of kit is, connect it to your PC using Bluetooth and then download the library and test application developed by Brian Peek that gives you a read out of the 20 or more variables that the device is capable of measuring.

To pair a Wiimote with a PC hold down the ‘1’ and ‘2’ buttons on the remote and then run the Bluetooth new connection wizard on your PC. If you have any problems (there are Bluetooth compatibility issues with some flavours of PCs) read the Getting Connected section of the excellent article on the Microsoft Developer Network coding4fun blog.

Next, download and unzip Brian Peek’s .Net Managed Library for Nintendo's Wii Remote and run the WiimoteTest.exe executable file from the root folder. You can then watch how the accelerometer and infrared camera response to you swinging the Wiimote around and test out the buttons, LEDs and rumble function. I guarantee Mario Karts will never be the same again.

Brian Peek maintains a discussion and list of Wiimote applications that use his library. My favourite is the Minority Report style computer interface created by Jonny Chung Lee from the Human-Computer Interaction Institute at Carnegie Mellon University.

It was Jonny Chung Lee’s TEDtalk describing a series of Wiimote applications including a finger tracking application, a low cost whiteboard and a head tracking virtual reality display, that started me out exploring Wiimote hacks.

My favourite for its sheer elegance is the finger tracking application that allows you to interact with your PC by moving your hands in the air in front of the screen similar to how Tom Cruise manipulates computer images with his finger tips in the movie Minority Report. It is almost like magic.

The application uses an infra-red LED array (either a discrete device used for security applications or a series of individual infra-red LEDs assembled into an array) and a reflective sticker on each finger to reflect infra-red light into the Wiimote camera. Jonny Chung Lee describes the project in detail on his web site and has published sample code. Here’s his original video demonstration. My eldest daughter has shot a quick demo of my set-up which I’ll post in due course.