DIY Eye And Face Tracking For The Valve Index VR Headset

May 19, 2024 by Donald Papp 10 Comments

The Valve Index VR headset has been around for a few years now. It doesn’t come with eye or face tracking, but that didn’t stop inspired folks like [Physics-Dude] from adding DIY solutions in elegant and effective ways using a combination of hardware, open software, and 3D printable parts.

The whole assembly integrates tightly, thanks in part to the “frunk” designed into the Index for exactly this kind of thing.

This project leverages the EyeTrackVR project (and optionally, Project Babble for mouth tracking) which both have great applications particularly in social VR spaces.

These are open-source, self-contained and modular solutions intended for a variety of hardware platforms. Of course, every millimeter and gram tends to count when it’s something that gets worn on one’s head, so [Physics-Dude] tailored a solution specifically for the Valve Index. His project makes great use of the platform’s hacker-friendly hardware design.

[Physics-Dude] also makes excellent use of a certain widely-available “gumstick” style USB hub as an important part of his build. Combined with with the front-mounted USB port on the Index, it results in an extremely compact and tightly integrated solution that looks great. While it can be risky to rely on a particular off-the-shelf item in a build, doing so absolutely has its place here.

The documentation is fantastic, including welcome guidance on cable routing and step-by-step instructions. If you’ve been interested in adding eye tracking to a project, be sure to give it a look. Already have eye tracking in a project of your own? Tell us all about it!

Homemade electric fan showing a small camera peeking up above the central hub.

Keep Cool With This Face-Following Fan

August 21, 2021 by Chris Lott 8 Comments

[AchillesVM] decided to build a tabletop electric fan so it would track him as he moves around the room. Pan and tilt control is provided by a pair of servos controlled by a Raspberry Pi 3b+. How does it know where [AchillesVM} is? It captures the scene using a Raspberry Pi v2 Camera and uses OpenCV’s default face-tracking algorithm to find him. Well, strictly speaking, it tracks anyone’s face around the room. If multiple faces are detected, it follows the largest — which is usually the person closest to the fan.

The whole processing loop runs at 60 ms, so the speed of the servo mechanism is probably the limiting factor when it comes to following fast-moving house guests. At first glance it might look like an old fan from the 1920s, in fact [AchillesVM] built the whole thing by himself, 3D-printing case and using a few off-the-shelf parts (like the 25 cm R/C plane propeller).

It’s a work in progress, so follow his GitHub repository (above) for updates. Hopefully, there will be a front-mounted finger guard coming soon. If you like gadgets that interact with you as you move about, we’ve covered the face-tracking confectionery cannon back in 2014, and the head-tracking water blaster last year. In the “don’t try this” file goes the build that started a career — the eye-tracking laser robot.

Watchman Watches You Watching Him Watch You

March 15, 2020 by Kristina Panos 8 Comments

At this point, society has had over three decades to get used to the Blue Man Group. Maybe that’s why we’re less disturbed by [Graham Jessup]’s face-tracking Watchman than we should be. Either that, or it’s because it reminds us of Data from Star Trek: The Next Generation. Frankly, this is just way too cool to be dismissed out of hand as creepy.

The Watchman finds faces via video feed from a camera module positioned in his forehead as a third eye. The camera is connected to a Pi Zero that’s wearing a Google AIY vision bonnet. The Pi translates the face locations into servo positions and feeds them to an Arduino UNO located in the frontal lobe region to move the eyeballs and lids accordingly.

[Graham] had a bit of trouble with tracking accuracy at first, so he temporarily replaced the pupils with 5 mW lasers and calibrated them by tracking a printed stand-in of his head to avoid burning out his retinas.

This project builds on previous work by [Tjahzi] and the animatronic eye movements of [Will Cogley]. We can only imagine how awesome the Watchman would look with a pair of [Will]’s incredibly realistic eyeballs. Either way, we would totally trust the Watchman to defend our modest supply of toilet paper in the coming weeks. Check out a brief demo after the break, and a whole lot more clips on [Graham]’s site.

Continue reading “Watchman Watches You Watching Him Watch You” →

Your Face Is Going Places You May Not Like

January 2, 2019 by Bob Baddeley 62 Comments

Many Chinese cities, among them Ningbo, are investing heavily in AI and facial recognition technology. Uses range from border control — at Shanghai’s international airport and the border crossing with Macau — to the trivial: shaming jaywalkers.

In Ningbo, cameras oversee the intersections, and use facial-recognition to shame offenders by putting their faces up on large displays for all to see, and presumably mutter “tsk-tsk”. So it shocked Dong Mingzhu, the chairwoman of China’s largest air conditioner firm, to see her own face on the wall of shame when she’d done nothing wrong. The AIs had picked up her face off of an ad on a passing bus.

False positives in detecting jaywalkers are mostly harmless and maybe even amusing, for now. But the city of Shenzhen has a deal in the works with cellphone service providers to identify the offenders personally and send them a text message, and eventually a fine, directly to their cell phone. One can imagine this getting Orwellian pretty fast.

Facial recognition has been explored for decades, and it is now reaching a tipping point where the impacts of the technology are starting to have real consequences for people, and not just in the ways dystopian sci-fi has portrayed. Whether it’s racist, inaccurate, or easily spoofed, getting computers to pick out faces correctly has been fraught with problems from the beginning. With more and more companies and governments using it, and having increasing impact on the public, the stakes are getting higher.

Continue reading “Your Face Is Going Places You May Not Like” →

R3-14, The Personal Assistant Two Years In The Making

October 14, 2018 by Al Williams 1 Comment

One of the great things about hacking together projects these days is how many powerful subsystems are readily available to reuse. [Sanjeet] took full advantage of a whole slate of reusable pieces when he built R3-14 — a personal assistant robot that you can see in action in the video below.

Many people started out in electronics building something simple like a crystal radio or an LED cube. But how far could you get if your projects had to begin at the most basic level, by drawing out copper wire, fabricating coils, capacitors, semiconductor devices, and batteries? Even if you know how to do all those things, it would take a lot of time, so there is no shame in using off-the-shelf components. By the same token, [Sanjeet] uses Google Assistant, 433 MHz RF transmitters, and a Raspberry Pi as components in this build. Along the way, he also contributed some reusable pieces himself, including an LED library for the PI and a library to allow Siri to control a Raspberry Pi.

Continue reading “R3-14, The Personal Assistant Two Years In The Making” →

Hackaday Prize Entry: Selfie Bot Let’s You Vlog Hands Free

July 8, 2016 by Gerrit Coetzee 23 Comments

[Sergey Mironov] sent in his SelfieBot project. His company, Endurance Robots, sells a commercial version of the bot, which leads us to believe that in a strange and maybe brilliant move he decided to just sell the prototype stage of the product development as a kit. Since he also gave away the firmware, STLs, BOM, and made a guide so anyone can build it, we’re not complaining.

The bot is simple enough. Nicely housed hobby servos in a 3D printed case take care of the pan and tilt of the camera. The base of the bot encloses the electronics, which are an Arduino nano, a Bluetooth module, and the support electronics for power and motor driving.

To perform the face tracking, the build assumes you have a second phone. This is silly, but isn’t so unreasonable. Most people who’ve had a smart phone for a few years have a spare one living in a drawer as back-up. One phone runs the face tracking software and points the bot, via Bluetooth, towards the user. The other phone records the video.

The bot is pretty jumpy in the example video, but this can be taken care of with better motors. For a proof-of-concept, it works. A video of it in action after the break.

Continue reading “Hackaday Prize Entry: Selfie Bot Let’s You Vlog Hands Free” →

Hexapod Tank From Ghost In The Shell Brought To Life

June 3, 2016 by Donald Papp 34 Comments

Every now and then someone gets seriously inspired, and that urge just doesn’t go away until something gets created. For [Paulius Liekis], it led to creating a roughly 1:20 scale version of the T08A2 Hexapod “Spider” Tank from the movie Ghost in the Shell. As the he puts it, “[T]his was something that I wanted to build for a long time and I just had to get it out of my system.” It uses two Raspberry Pi computers, 28 servo motors, and required over 250 hours of 3D printing for all the meticulously modeled pieces – and even more than that for polishing, filing, painting, and other finishing work on the pieces after they were printed. The paint job is spectacular, with great-looking wear and tear. It’s even better seeing it in motion — see the video embedded below.

Continue reading “Hexapod Tank From Ghost In The Shell Brought To Life” →