Screenshot of the demonstration video that shows the desktop being unlocked with face recognition, with a camera feed and a terminal showing how the software works.

Open-Source FaceID With RealSense

April 4, 2023 by Arya Voronova 6 Comments

RealSense cameras have been a fascinating piece of tech from Intel — we’ve seen a number of cool applications in the hacker world, from robots to smart appliances. Unfortunately Intel did discontinue parts of the RealSense lineup at one point, specifically the LiDAR and face tracking-tailored models. Apparently, these haven’t been popular, and we haven’t seen these in hacks either. Until now, that is. [Lina] brings us a real-world application for the RealSense face tracking cameras, a FaceID application for Linux.

The project is as simple as it sounds: if the camera’s built-in face recognition module recognizes you, your lockscreen is unlocked. With the target being Linux, it has to tie into the Pluggable Authentication Modules (PAM) subsystem for authentication, and of course, there’s a PAM module for RealSense to go with it, aptly named pam_sauron. This module is written in Zig, a modern C-like language, so it’s both a good example of how to create your own PAM integrations, and a path towards doing that in a different language for once. As usual, there’s TODOs, like improving the UX and taking advantage of some security features RealSense cameras have, but it’s nevertheless a fun and self-sufficient application for one of the F4XX-series RealSense cameras in case you happen to own one.

Ever since the introduction of RealSense we’ve seen these cameras used in robotics and 3D scanning, thanks at least in part due to their ability to be used in Linux. Thankfully, Intel only discontinued the less popular RealSense cameras, which didn’t affect the main RealSense lineup, and the hacker-beloved depth cameras are still available for all of our projects. Wondering about the tech behind it? Here’s a teardown of a RealSense camera module intended for laptop use.

RealSense No Longer Makes Sense For Intel

August 20, 2021 by Roger Cheng 15 Comments

We love depth-sensing cameras and every neat hack they enabled, but this technological novelty has yet to break through to high volume commercial success. So it was sad but not surprising when CRN reported that Intel has decided to wind down their RealSense product line.

As of this writing, one of the better confirmations for this report can be found on the RealSense SDK GitHub repository README. The good news is that core depth-sensing RealSense products will continue business as usual for the foreseeable future, balanced by the bad news that some interesting offshoots (facial authentication, motion tracking) will be declared “End of Life” immediately and phased out over the next six months.

This information tells us while those living out on the bleeding edge will have to scramble, there is no immediate crisis for everyone else, whether they be researchers, hobbyists, or product planners. But this also means there will be no future RealSense cameras, kicking off many “What’s Next?” discussions in various communities. Like this thread on ROS (Robot Operating System) Discourse.

Three popular alternatives offer distinctly different tradeoffs. The “Been Around The Block” name is Occipital, with their more expensive Structure Pro sensor. The “Old Name, New Face” option is Microsoft Azure Kinect, the latest non-gaming-focused successor to the gaming peripheral that started it all. And let’s not forget OAK-D as the “New Kid On The Block” that started with a crowdfunding campaign and building an user community by doing things like holding contests. Each of these will appeal to a different niche, and we’ll keep our eye open in the future. Let’s see if any of them find the success that eluded the original Kinect, Google’s Tango, and now Intel’s RealSense.

[via Engadget]

Machine-Vision Archer Makes You The Target, If You Dare

June 30, 2021 by Dan Maloney 6 Comments

We’ll state right up front that it’s a really, really bad idea to let a robotic archer shoot an apple off of your head. You absolutely should not repeat what you’ll see in the video below, and if you do, the results are all on you.

That said, [Kamal Carter]’s build is pretty darn cool. He wisely chose to use just about the weakest bows you can get, the kind with strings that are basically big, floppy elastic bands that shoot arrows with suction-cup tips and are so harmless that they’re intended for children to play with and you just know they’re going to shoot each other the minute you turn your back no matter what you told them. Target acquisition is the job of an Intel RealSense depth camera, which was used to find targets and calculate the distance to them. An aluminum extrusion frame holds the bow and adjusts its elevation, while a long leadscrew and a servo draw and release the string.

With the running gear sorted, [Kamal] turned to high school physics for calculations such as the spring constant of the bow to determine the arrow’s initial velocity, and the ballistics formula to determine the angle needed to hit the target. And hit it he does — mostly. We’re actually surprised how many on-target shots he got. And yes, he did eventually get it to pull a [William Tell] apple trick — although we couldn’t help but notice from his, ahem, hand posture that he wasn’t exactly filled with self-confidence about where the arrow would end up.

[Kamal] says he drew inspiration both from [Mark Rober]’s dart-catching dartboard and [Shane Wighton]’s self-dunking basketball hoop for this build. We’d say his results put in him good standing with the skill-optional sports community.

Continue reading “Machine-Vision Archer Makes You The Target, If You Dare” →

Kinect Gave Us A Preview Of The Future, Though Not The One It Intended

November 9, 2020 by Roger Cheng 13 Comments

This holiday season, the video game industry hype machine is focused on building excitement for new PlayStation and Xbox consoles. Ten years ago, a similar chorus of hype reached a crescendo with the release of Xbox Kinect, promising to revolutionize how we play. That vision never panned out, but as [Daniel Cooper] of Engadget pointed out in a Kinect retrospective, it premiered consumer technologies that impacted fields far beyond gaming.

Kinect has since withdrawn from the gaming market, because as it turns out gamers are quite content with handheld controllers. This year’s new controllers for a PlayStation or Xbox would be immediately familiar to gamers from ten years ago. Even Nintendo, whose Wii is frequently credited as motivation for Microsoft to develop the Kinect, have arguably taken a step back with Joy-cons of their Switch.

But the Kinect’s success at bringing a depth camera to consumer price levels paved the way to explore many ideas that were previously impossible. The flurry of enthusiastic Kinect hacking proved there is a market for depth camera peripherals, leading to plug-and-play devices like Intel RealSense to make depth-sensing projects easier. The original PrimeSense technology has since been simplified and miniaturized into Face ID unlocking Apple phones. Kinect itself found another job with Microsoft’s HoloLens AR headset. And let’s not forget the upcoming wave of autonomous cars and drones, many of which will see their worlds via depth sensors of some kind. Some might even be equipped with the latest sensor to wear the Kinect name.

Inside the Kinect was also one of the earliest microphone arrays sold to consumers. Enabling the Kinect to figure out which direction a voice is coming from, and isolate it from other noises in the room. Such technology were previously the exclusive domain of expensive corporate conference room speakerphones, but now it forms the core of inexpensive home assistants like an Amazon Echo Dot. Raising the bar so much that hacks needed many more microphones just to stand out.

With the technology available more easily elsewhere, attrition of a discontinued device is reflected in the dwindling number of recent Kinect hacks on these pages. We still see a cool project every now and then, though. As the classic sensor bar itself recedes into history, others will take its place to give us depth sensing and smart audio. But for many of us, Kinect was the ambitious videogame peripheral that gave us our first experience.

Using RealSense Cameras With OS X And Linux

January 23, 2016 by Brian Benchoff 21 Comments

The original Microsoft Kinect was a revolution in computer vision. For less than one hundred dollars, the Kinect gave everyone a webcam with a depth sensor. If you’re doing anything with robots, 3D scanning, or anything else where a computer needs to know where it is in 3D space, it’s awesome. These depth-mapping cameras have improved over the years, with the latest and most capable hardware being Intel’s RealSense 3D camera.

Despite being a very capable depth camera, support for Linux and OS X doesn’t exist. Researchers, roboticists and IoT developers are slightly miffed about this, and it seems like Intel doesn’t care about people using their hardware on platforms that aren’t Windows.

Now, finally, that’s changed. A few developers have taken it upon themselves to build a cross-platform library for the F200, SR300, and R200 Intel RealSense depth cameras.

The librealsense library features proper RealSense camera support for Linux, OS X, and Windows and provides all the functionality of the official Intel SDK. This functionality includes native depth, color, and infrared streams, synthetic streams for rectified images, calibration information, and the most interesting feature: multi-camera capture.

The hardware required to use the RealSense camera is somewhat lightweight – any recent laptop should be able to capture depth images with a RealSense camera. The camera itself requires USB 3, though, so you won’t be building a 3D scanner with a RealSense camera and a Raspberry Pi quite yet. Still, it’s the latest advancement for giving robots 3D vision and building cheap, portable 3D scanners.

Teardown Of Intel RealSense Gesture Camera Reveals Projector Details

July 29, 2015 by Richard Baguley 23 Comments

[Chipworks] has just released the details on their latest teardown on an Intel RealSense gesture camera that was built into a Lenovo laptop. Teardowns are always interesting (and we suspect that [Chipworks] can’t eat breakfast without tearing it down), but this one reveals some fascinating details on how you build a projector into a module that fits into a laptop bezel. While most structured light projectors use a single, static pattern projected through a mask, this one uses a real projection mechanism to send different patterns that help the device detect gestures faster, all in a mechanism that is thinner than a poker chip.

It does this by using an impressive miniaturized projector made of three tiny components: an IR laser, a line lens and a resonant micromirror. The line lens takes the point of light from the IR laser and turns it into a flat horizontal line. This is then bounced off the resonant micromirror, which is twisted by an electrical signal. This micromirror is moved by a torsional drive system, where an electrostatic signal twists the mirror, which is manufactured in a single piece. The system is described in more detail in this PDF of a presentation by the makers, ST Micro. This combination of lens and rapidly moving mirrors creates a pattern of light that is projected, and the reflection is detected by the IR camera on the other side of the module, which is used to create a 3D model that can be used to detect gestures, faces, and other objects. It’s a neat insight into how you can miniaturize things by approaching them in a different way.