The tale of the Microsoft Xbox Kinect is one of those sad situations where a great product was used in an application that turned out to be a bit of a flop and was discontinued because of it, despite its usefulness in other areas. This article from the Guardian is a quick read on how this handy depth camera has found other uses in somewhat niche areas, with not a computer game in sight.
It’s rather obvious that a camera that can generate a 3D depth map, in parallel with a 2D reference image, could have many applications beyond gaming, especially in the hands of us hackers. Potential uses include autonomous roving robots, 3D scanning, and complex user interfaces—there are endless possibilities. Artists producing interactive art exhibits would sit firmly in that last category, with the Kinect used in countless installations worldwide.
Apparently, the Kinect also has quite the following in ghost-hunting circles, which as many a dubious TV show would demonstrate, seem almost entirely filmed under IR light conditions. The Kinect’s IR-based structured light system is well-suited for these environments. Since its processing core runs a machine learning application specifically trained to track human figures, it’s no surprise that the device can pick up those invisible, pesky spirits hiding in the noise. Anyway, all of these applications depend on the used-market supply of Kinect devices, over a decade old, that can be found online and in car boot sales, which means one day, the Kinect really will die off, only to be replaced with specialist devices that cost orders of magnitude more to acquire.
In the unlikely event you’ve not encountered non-gaming applications for the Kinect, here’s an old project to scan an entire room to get you started. Just to be perverse, here’s a gaming application that Microsoft didn’t think of, and to round out, the bad news that Microsoft has really has abandoned the product.
Talking about the Kinect and Not a mention of the VR sandbox?
Agreed, that is a great project.
https://ar-sandbox.eu/augmented-reality-sandbox-software/
Please do not forget its predecessor by Oliver Kreylos at UCDavis.
I made an instance of Oliver Kreylos’ great sARndbox for a Maker Faire:
https://web.cs.ucdavis.edu/~okreylos/ResDev/SARndbox/LinkDownload.html
THAT makes more sense because I see it’s a Linux installation and that’s what I recall since I’ve not run Windows since the late ’90s.
When I was working at Mozilla, in one of my many unofficial side projects I added Kinect support to Firefox, which made the RGB camera and depth map available to normal web content via a JavaScript API. TypeArrays were just being added as an experimental feature – they were still very limited and still had a lot of performance bottlenecks, but it worked well enough. It made it really easy to play around with the data and experiment with drawing it to a Canvas element that was part of other web content. Fun times.
Is the app available?
For anyone that is interested: The older version of the Kinect is the version to snag – it is much easier to interface with via open source libraries on all the common development environments. Because of this, there is much higher demand for the older version and ebay prices reflect that.
It’s been a while since I bought and tinkered with a used Kinect, so there might have been some developments cracking the newer version since I was last looking into them.
Indeed. A peripheral killed too soon and not often enough thought of for what it’s best for. Very strange, a fairly unique case. There are Chinese knock-offs these days which interface with the same libraries which are still produced, but they aren’t as good and the documentation is very spotty. I think Lumoplay uses one.
I got lucky then and snagged a bag full of them from shopgoodwill a decade ago
I just had another look. They are still selling “untested” lots for cheap. Coming to about less than $5/ea without psus
Yeah the new hardware is MUCH better but unfortunately the drivers that Microsoft provide don’t make any use of it anyway. So there is no point in getting the newer one, and the old one is easier to get going.
I just looked into it for VRChat support (full limb tracking etc)
My first MakerFaire project way back in 2012 was a OpenCV app that detected the position of your hands and applied it as differential steering values to drive a Roomba around. It was a hit when it worked, but the Sun washed out the IR light which meant a lot of the day it really struggled.
Said project: https://blog.aaroneiche.com/2012/09/20/me-at-the-portland-makerfaire/
One thing to note, especially for battery powered projects like an autonomous robot, the kinect is quite power hungry.
It’s a bit of a faff but there are ways to bring the power cost way down, the 12v bits are needed for the motors if memory serves but the rest can run from USB.
https://medium.com/robotics-weekends/how-to-turn-old-kinect-into-a-compact-usb-powered-rgbd-sensor-f23d58e10eb0
It was an interesting technology and great gadget to hack. It was not a ‘great product’. A great product has or creates a demand for its existence and is affordably produced to create profit and long term preferably repeat customers. Kinect was not that.
Yeah, I concur with that. It’s easy to just call everything a ‘product’ when really it’s a product of engineering that I think of. A thing. A gadget. A commercial failure. I still own three.
Microsoft didn’t invent the Kinect technology, they licensed it from Primesense. After Primesense worked with Google and demo’d an Android tablet with an embedded PrimeSense 3D camera, Apple purchased Primesense and mothballed it for 10+ years.
When the Kinect first hit the market, Microsoft didn’t provide any 3rd party use libraries so Adafruit put out a reward for the first working library. It took about a week and then another week before Microsoft threatened to sue Adafruit. Limur didn’t back down and Microsoft backed off. The Kinect became a hit in the Linux robotics field. Asus and others also licensed the tech from Primesense and Primesense published lots of libraries for use. It all disappeared once Apple closed the deal.
Some internet archives were around with the Primesense code last I looked over 15 years ago.
I have seen Xbox Kinect being used at an airport for bag drop off (Changi Airport)
I know that somewhat after Kinect was launched, some manufacturers began to offer CMOS sensors with RGBI (Red, Green, Blue, Infrared) filters instead of Bayer ones, pointing to this kind of applications, but I can’t find them anywhere. Does anybody know if there is today a reasonably-priced USB cam with that kind of sensor/filter? (I have some ideas that I want to try, and I need that).
DFRobot do a camera sensor that can manage to track 4 IR sources at one time, it’s incredibly popular with modern DIY lightgun projects.
But that isn’t what I need… I want a camera that can give me, at the same time, an infrared and a color picture.
Funny story, Singapore Airport automated bag drop machines use Kinects. Personally confirmed 2 months ago when I was there:
https://www.reddit.com/r/mildlyinteresting/comments/1hqbe5f/bag_drops_at_singapore_airport_use_xbox_kinect/
I love the kinect.
I built an AR Sandbox with Oliver Kreylos’s instructions and software.
I’m also using it as a person tracker to drive a big person-follower halloween-eye :)
The project has not been completely abandoned. It morphed into Azure Kinect and the same technology has been repackaged as the Orbbec Femto Bolt.
https://azure.microsoft.com/en-us/products/kinect-dk
https://store.orbbec.com/products/femto-bolt
The XBOX Kinect is an incredible piece of hardware for what it costs. Once all the used ones finally break there is going to be a real void left. What other device can give you high resolution time of flight depth images at 30Hz for a few hundred bucks? I don’t think there are any.
I saw them being used at Disney world. The Peter Pan line had interactive holographic butterflies I looked up and to my surprise saw the Kinect on the ceiling pointing down.
I used the Kinect for a few dance performances. I controlled or puppeteered a character in a game engine while I danced. I have also seen them used at Arttechhouse for interactive projections
There is one inside of a Tempo: https://tempo.fit/studio-suite
I remember when I was just discovering vrchat and wanted to use my xb1 kinect for tracking
Then I learned that I needed a $300 adapter that I could only get secondhand from China to actually connect it to my pc and decided against it
It’s been closer to a decade now, wonder if theres a cheaper way to do that, it’s just been sitting connected to my xbox under my TV collecting dust…
I think there is actually, I don’t know where I found it, but you can connect a normal usb 3 cable and solder some wires internally to connect a power supply and then it works, at least it did for me.
As someone in the paranormal community, I have used Kinect for ghost hunting.
I think it was over 10 years ago, I was trying to test 3d scanning using a kinect as well as using a laser light projected as a horizontal beam.
This was for a client who was fabricating parts with their CNC for cars and motorcycles.
I remember using David Scanner and I think there was some other software to do this. While it worked, the results were not what the client wanted as the resolution was too low.
I think I spent a few hours tinkering with this setup. It’s great how far 3d scanning technology has come, but it’s still nowhere near the cost of a $20 laser light or your Kinect and some free software.
Then again, you can now use your cellphone as a 3d scanner and get better results.