Eye-Controlled Wheelchair Advances from Talented Teenage Hackers

[Myrijam Stoetzer] and her friend [Paul Foltin], 14 and 15 years old kids from Duisburg, Germany are working on a eye movement controller wheel chair. They were inspired by the Eyewriter Project which we’ve been following for a long time. Eyewriter was built for Tony Quan a.k.a Tempt1 by his friends. In 2003, Tempt1 was diagnosed with the degenerative nerve disorder ALS  and is now fully paralyzed except for his eyes, but has been able to use the EyeWriter to continue his art.

This is their first big leap moving up from Lego Mindstorms. The eye tracker part consists of a safety glass frame, a regular webcam, and IR SMD LEDs. They removed the IR blocking filter from the webcam to make it work in all lighting conditions. The image processing is handled by an Odroid U3 – a compact, low cost ARM Quad Core SBC capable of running Ubuntu, Android, and other Linux OS systems. They initially tried the Raspberry Pi which managed to do just about 3fps, compared to 13~15fps from the Odroid. The code is written in Python and uses OpenCV libraries. They are learning Python on the go. An Arduino is used to control the motor via an H-bridge controller, and also to calibrate the eye tracker. Potentiometers connected to the Arduino’s analog ports allow adjusting the tracker to individual requirements.

The web cam video stream is filtered to obtain the pupil position, and this is compared to four presets for forward, reverse, left and right. The presets can be adjusted using the potentiometers. An enable switch, manually activated at present is used to ensure the wheel chair moves only when commanded. Their plan is to later replace this switch with tongue activation or maybe cheek muscle twitch detection.

First tests were on a small mockup robotic platform. After winning a local competition, they bought a second-hand wheel chair and started all over again. This time, they tried the Raspberry Pi 2 model B, and it was able to work at about 8~9fps. Not as well as the Odroid, but at half the cost, it seemed like a workable solution since their aim is to make it as cheap as possible. They would appreciate receiving any help to improve the performance – maybe improving their code or utilising all the four cores more efficiently. For the bigger wheelchair, they used recycled car windshield wiper motors and some relays to switch them. They also used a 3D printer to print an enclosure for the camera and wheels to help turn the wheelchair. Further details are also available on [Myrijam]’s blog. They documented their build (German, pdf) and have their sights set on the German National Science Fair. The team is working on English translation of the documentation and will release all design files and source code under a CC by NC license soon.

OpenCV knows where you’re looking with eye tracking

[John] has been working on a video-based eye tracking solution using OpenCV, and we’re loving the progress. [John]’s pupil tracking software can tell anyone exactly where you’re looking and allows for free head movement.

The basic idea behind this build is simple; when looking straight ahead a pupil is perfectly circular. When an eye looks off to one side, a pupil looks more and more like an ellipse to a screen-mounted video camera. By measuring the dimensions of this ellipse, [John]’s software can make a very good guess where the eye is looking. If you want the extremely technical breakdown, here’s an ACM paper going over the technique.

Like the EyeWriter project this build was based on, [John]’s build uses IR LEDs around the edge of a monitor to increase the contrast between the pupil and the iris.

After the break are two videos showing the eyetracker in action. Watching [John]’s project at work is a little creepy, but the good news is a proper eye tracking setup doesn’t require the user to stare at their eye.

Continue reading “OpenCV knows where you’re looking with eye tracking”

Kids type with their eyes, robot arm prints their words

F.A.T. took it to the next level, combining a couple of their projects for the Cinekid festival. This contraption lets kids write their names with their eyes for printing by a robot arm. The first part is a glasses-free version of the EyeWriter, originally developed as an assistive technology. The system uses some IR LEDs to generate a reflection on your eye that a PS3 camera can pick up and use to precisely track your gaze. Just look at each key on a virtual keyboard to spell out your message. From there, a robot arm used previously in the Robotagger project prints out the name on a big sheet of paper the kids can take home. This is cool, but more importantly it’s a great way to inspire the next generation of hackers and engineers. Check out the video after the break.

Continue reading “Kids type with their eyes, robot arm prints their words”

EyeWriter is the fruit of the KanEye project

The EyeWriter is an open source eye tracking initiative. This is the mature version of the KanEye project we covered in April. Collaboratively developed by Free Art and Technology (FAT), OpenFrameworks, and the Graffiti Research Lab, they seek to aid a friend of theirs who suffers from the degenerative muscle disease ALS.

They’ve come a long way since we last looked in on the progress. The hardware used is pretty much the same: a set of sunglasses sans lenses with the CCD from a Sony PlayStation 3 Eye mounted in front of one eye. IR LEDs surround the CCD and point at the eye to increase the contrast between pupil and the rest of the eye. The major improvement comes with the software. Eye tracking appears to be extremely precise and they’ve written a custom drawing program to take advantage of their interface. Check in on their developer page for source code and a video walk-through of the software.

After the break you can see video of [Tempt1] using the system to create some tags. We’re thankful for the success this project has seen as this guy can do a lot better with his eye than we can with our hands.

Continue reading “EyeWriter is the fruit of the KanEye project”