A few years ago, there was a rush of products on the market to detect motion. The idea being you could interact with your computer like they do on science fiction movies, with giant expressive hand motions in the air. Most of these were aimed at desktop computer users but one company, YouSpace, wanted to bring this technology to retail stores. [IMSAI Guy] got one of their sensor devices and decided to see what was inside it. You can see, too, in the video below.
The device appeared to have a laser inside, which motivated the teardown. We aren’t sure exactly what YouSpace had planned, but you can see their now-defunct website on the Wayback machine. The use cases listed didn’t really help us get a clear picture, so maybe that was part of the problem.
Getting into the device was the first challenge. Like many modern smartphones, there didn’t appear to be any fasteners, so you simply had to pry the case apart. Inside the case: a tiny circuit board and a metal assembly containing the laser and cameras that were easy to remove. The main PCB appears to be an Intel off-the-shelf board that was in many Intel RealSense products, and currently go for about $50 on eBay. The camera assembly looks a bit like an Intel D430, so it is possible the entire thing was off-the-shelf hardware. Even the little connector board is, technically, a D400 Interposer.
The peek into the structured light project under the microscope was interesting. We expected it would look different, and [IMSAI Guy] clearly didn’t expect its appearance either. The chip was made to beam a known pattern that the cameras would use to deduce the shape of the surfaces it hits.
If you can find these on the surplus market, they would probably be a good deal if you need this hardware which is typically pretty expensive. Just beware, though. Intel announced in late 2021 they were “winding down” RealSense. We don’t know if there will be third-party support in the future or if the whole product line will just be orphaned.
RealSense cameras have been a fascinating piece of tech from Intel — we’ve seen a number of cool applications in the hacker world, from robots to smart appliances. Unfortunately Intel did discontinue parts of the RealSense lineup at one point, specifically the LiDAR and face tracking-tailored models. Apparently, these haven’t been popular, and we haven’t seen these in hacks either. Until now, that is. [Lina] brings us a real-world application for the RealSense face tracking cameras, a FaceID application for Linux.
The project is as simple as it sounds: if the camera’s built-in face recognition module recognizes you, your lockscreen is unlocked. With the target being Linux, it has to tie into the Pluggable Authentication Modules (PAM) subsystem for authentication, and of course, there’s a PAM module for RealSense to go with it, aptly named pam_sauron. This module is written in Zig, a modern C-like language, so it’s both a good example of how to create your own PAM integrations, and a path towards doing that in a different language for once. As usual, there’s TODOs, like improving the UX and taking advantage of some security features RealSense cameras have, but it’s nevertheless a fun and self-sufficient application for one of the F4XX-series RealSense cameras in case you happen to own one.
Why? Getting a drone that can fly a path and even return home when the battery is low, signal is lost, or on command, is simple enough. Just go to your favorite retailer, search “gps drone” and you can get away for a shockingly low dollar amount. This is possible because GPS receivers have become cheap, small, light, and power efficient. While all of these inexpensive drones can fly a predetermined path, they usually do so by flying over any obstacles rather than around.
[Nick Rehm] has envisioned a quadcopter that can do all of the things a GPS-enabled drone can do, without the use of a GPS receiver. [Nick] makes this possible by using algorithms similar to those used by Google Maps, with data coming from a typical IMU, a camera for Computer Vision, LIDAR for altitude, and an Intel RealSense camera for detection of position and movement. A Raspberry Pi 4 running Robot Operating System runs the autonomous show, and a Teensy takes care of flight control duties.
What we really enjoy about [Nick]’s video is his clear presentation of complex technologies, and a great sense of humor about a project that has consumed untold amounts of time, patience, and duct tape.
As of this writing, one of the better confirmations for this report can be found on the RealSense SDK GitHub repository README. The good news is that core depth-sensing RealSense products will continue business as usual for the foreseeable future, balanced by the bad news that some interesting offshoots (facial authentication, motion tracking) will be declared “End of Life” immediately and phased out over the next six months.
This information tells us while those living out on the bleeding edge will have to scramble, there is no immediate crisis for everyone else, whether they be researchers, hobbyists, or product planners. But this also means there will be no future RealSense cameras, kicking off many “What’s Next?” discussions in various communities. Like this thread on ROS (Robot Operating System) Discourse.
Three popular alternatives offer distinctly different tradeoffs. The “Been Around The Block” name is Occipital, with their more expensive Structure Pro sensor. The “Old Name, New Face” option is Microsoft Azure Kinect, the latest non-gaming-focused successor to the gaming peripheral that started it all. And let’s not forget OAK-D as the “New Kid On The Block” that started with a crowdfunding campaign and building an user community by doing things like holding contests. Each of these will appeal to a different niche, and we’ll keep our eye open in the future. Let’s see if any of them find the success that eluded the original Kinect, Google’s Tango, and now Intel’s RealSense.
This holiday season, the video game industry hype machine is focused on building excitement for new PlayStation and Xbox consoles. Ten years ago, a similar chorus of hype reached a crescendo with the release of Xbox Kinect, promising to revolutionize how we play. That vision never panned out, but as [Daniel Cooper] of Engadget pointed out in a Kinect retrospective, it premiered consumer technologies that impacted fields far beyond gaming.
Kinect has since withdrawn from the gaming market, because as it turns out gamers are quite content with handheld controllers. This year’s new controllers for a PlayStation or Xbox would be immediately familiar to gamers from ten years ago. Even Nintendo, whose Wii is frequently credited as motivation for Microsoft to develop the Kinect, have arguably taken a step back with Joy-cons of their Switch.
But the Kinect’s success at bringing a depth camera to consumer price levels paved the way to explore many ideas that were previously impossible. The flurry of enthusiastic Kinect hacking proved there is a market for depth camera peripherals, leading to plug-and-play devices like Intel RealSense to make depth-sensing projects easier. The original PrimeSense technology has since been simplified and miniaturized into Face ID unlocking Apple phones. Kinect itself found another job with Microsoft’s HoloLens AR headset. And let’s not forget the upcoming wave of autonomous cars and drones, many of which will see their worlds via depth sensors of some kind. Some might even be equipped with the latest sensor to wear the Kinect name.
Inside the Kinect was also one of the earliest microphone arrays sold to consumers. Enabling the Kinect to figure out which direction a voice is coming from, and isolate it from other noises in the room. Such technology were previously the exclusive domain of expensive corporate conference room speakerphones, but now it forms the core of inexpensive home assistants like an Amazon Echo Dot. Raising the bar so much that hacks needed many more microphones just to stand out.
With the technology available more easily elsewhere, attrition of a discontinued device is reflected in the dwindling number of recent Kinect hacks on these pages. We still see a cool project every now and then, though. As the classic sensor bar itself recedes into history, others will take its place to give us depth sensing and smart audio. But for many of us, Kinect was the ambitious videogame peripheral that gave us our first experience.
When the Raspberry Pi 4 came out, [Frank Zhao] saw the potential to make a realtime 3D scanner that was completely handheld and self-contained. The device has an Intel RealSense D415 depth-sensing camera as the main sensor, which uses two IR cameras and an RGB camera along with the Raspberry Pi 4. The Pi uses a piece of software called RTAB-Map — intended for robotic applications — to take care of using the data from the camera to map the environment in 3D and localize itself within that 3D space. Everything gets recorded in realtime.
This handheld device can act as a 3D scanner because the data gathered by RTAB-Map consists of a point cloud of an area as well as depth information. When combined with the origin of the sensing unit (i.e. the location of the camera within that area) it can export a point cloud into a mesh and even apply a texture derived from the camera footage. An example is shown below the break. Continue reading “Handheld 3D Scanning, Using Raspberry Pi 4 And Intel RealSense Camera”→
“Know your enemy” is the essence of one of the most famous quotes from [Sun Tzu]’s Art of War, and it’s as true now as it was 2,500 years ago. It also applies far beyond the martial arts, and as the world squares off for battle against COVID-19, it’s especially important to know the enemy: the novel coronavirus now dubbed SARS-CoV-2. And now, augmented reality technology is giving a boost to search for fatal flaws in the virus that can be exploited to defeat it.
The video below is a fascinating mix of 3D models of viral structures, like the external spike glycoproteins that give coronaviruses their characteristic crown appearance, layered onto live video of [Tom Goddard], a programmer/analysts at the University of California San Francisco. The tool he’s using is called ChimeraX, a molecular visualization program developed by him and his colleagues. He actually refers to this setup as “mixed reality” rather than “augmented reality”, to stress the fact that AR tends to be an experience that only the user can fully appreciate, whereas this system allows him to act as a guide on a virtual tour of the smallest of structures.
Using a depth-sensing camera and a VR headset, [Tom] is able to manipulate 3D models of the SARS virus — we don’t yet have full 3D structure data for the novel coronavirus proteins — to show us exactly how SARS binds to its receptor, angiotensin-converting enzyme-2 (ACE-2), a protein expressed on the cell surfaces of many different tissue types. It’s fascinating to see how the biding domain of the spike reaches out to latch onto ACE-2 to begin the process of invading a cell; it’s also heartening to watch [Tom]’s simulation of how the immune system responds to and blocks that binding.
It looks like ChimeraX and similar AR systems are going to prove to be powerful tools in the fight against not just COVID-19, but in all kinds of infectious diseases. Hats off to [Tom] and his team for making them available to researchers free of charge.