Fresh from Microsoft Research is an ingenious way to reduce interference and decrease the error in a Kinect. Bonus: the technique only requires a motor with an offset weight, or just an oversized version of the vibration motor found in a pager.
Being the first of its kind of commodity 3D depth sensors, the tracking on a Kinect really isn’t that good. In every Kinect demo we’ve ever seen, there are always errors in the 3D tracking or missing data in the point cloud. The Shake ‘n’ Sense, as Microsoft Research calls it, does away with these problems simply by vibrating the IR projector and camera with a single motor.
In addition to getting high quality point clouds from a Kinect, this technique also allows for multiple Kinects to be used in the same room. In the video (and title pic for this post), you can see a guy walking around a room filled with beach balls in 3D, captured from an array of four Kinects.
This opens up the doors to a whole lot of builds that were impossible with the current iteration of the Kinect, but we’re thinking this is far too easy and too clever not to be though of before. We’d love to see some independent verification of this technique, so if you’ve got a Kinect project sitting around, strap a motor onto it, make a video and send it in.
Subscribe for weekly videos
This is truly elegant engineering, I love it!
Not want to be a spoiler, but this is meant to solve interference between *several* Kinects; from the youtube text: “Shake ‘n’ Sense is a novel yet simple mechanical technique for mitigating the interference when two or more Kinect cameras point at the same part of a physical scene”
It does not improve performance for a single kinect, as far as the info of the movie goes.
It’s true that it doesn’t improve the quality of a single sensor, but it DOES enable multiple sensors to be used on the same scene (stated in the summary and the movie). Multiple inputs can be combined to provide enhanced sensor quality, increased coverage, and increase the number of possibilities for this technology.
Also, this same method would work fine with improved optics in the Kinect 2, allowing for potential support for multiple sensors.
Who’da thunk it, so simple. I wonder any care needs to be taken to keep the weighted motors out of sync with each other.
I bet you could manage it for sure just by making all the weights vary in mass slightly.
It wouldn’t matter because the cameras aren’t from the same viewpoint so the motion of the pattern from one sensor will be vastly different when viewed from another sensor.
Look up the “V Motion Project” on vimeo. It was done for an energy drink company, but it uses this principle.
After you’ve been dubstepped out, the writeup is kinda interesting:
http://www.custom-logic.com/blog/v-motion-project-the-instrument/
I imagine this would work for a single Kinect, actually. It’s a physical version of a technique used to increase the number of bits of accuracy for an ADC: introduce noise into your input and average the measurements. Atmel actually has an application note describing it here (PDF)
Dithering on instruments to increase accuracy isn’t particularly new, although application to the kniect is obviously novel. This is definitely one of those things that now that you see it you’re surprised it hadn’t been done before.
Basically saccades, innit? Agreeing with those who are saying “How did everyone else not think of that?”
I’m surprised the kinect doesn’t have some latency in the IR sensor compared to the camera that causes problems with accuracy.
Infrared and visible light both travel at the same speed, so if there is any latency, it’s in the data processing.
Dithering…
I dunno…well, maybe…but wait, no — ah, whatever you think!
Is there a way to attenuate the noise so it doesn’t sound like my Xbox is getting an unending stream of text messages?
The vibration motor found in a pager… or for those of us living in 2012, a phone…
Dithering, true, but motion blur is the more important aspect here.
Basically each infrared projector stops putting out a point cloud and starts putting out a motion-blurred point cloud. Because the infrared camera is undergoing the same motion blur, it sees the points it is projecting as true points still, but each other kinect only sees blurry indistinct IR light, not a point cloud that its trying to measure.
Genius!
This technique works with a single kinect too. I wrote a piece of software that ran the motor up and down in slight increments, and captured depth data only while the lens was moving. It was hell on the motor, but it improved the capture quality significantly.
Does this have anything to do with the stochastic ressonance phenomena? http://en.wikipedia.org/wiki/Stochastic_resonance
If microsoft doesn’t put this into Kinect 2.0 they’re crazy.
I haven’t had a chance to read the source article, does this work with the stock firmware? Would this be able to improve accuracy using Kinect Games?
This makes me think of how a lock-in amplifier works.
This was originally published by Andrew Maimone and
Henry Fuchs at the IEEE VR conference this March, I was there and it was a pretty interesting talk, and the general feeling in the crowd was “why didn’t we think of that?”
Here’s a link to the paper from IEEE VR if anyone is interested in more details:
http://www.cs.unc.edu/~fuchs/kinect_VR_2012.pdf
And here I was going to mention that :P
It definitely is a simple solution to the interference issue though. I myself thought that turning the cameras/lasers on/off in sequence rapidly would’ve been the solution people would go for.
+1
Pretty simple and incredibly powerful!!!
Do you think this idea can help kinect to sense fast motions more accurately?
What the hell’s a “pager?” And why do my grandparents keep giving me these sheets of paper with all these numbers on them? I think they call them “chaks” or something like that…
i can see many hackaday reader (those who own an xbox) playing xbox without force feedback, because they needed the motor for their kinect
I’m sorry but may I ask that they do the voiceover for their video with an actual microphone and more importantly by a person not having a severe headcold? I found in unbearable and had to turn the sound off.
now if someone can make a case mod for the kenetic that takes advantage of this and the “glasses” research that improves the scan resolution that would be awesome. just another reason for me to get off my butt and go buy a kinetic sadly I’ll have to invest in a more powerful netbook to pull it off while on the go ^^;
How about applying super resolution techniques to a multiple Kinect setup?
http://en.wikipedia.org/wiki/Super-resolution