3D Display Using A Kinect

Youtube user [programming4fun] got a Kinect for father’s day and with just a little bit of code, came up with a 3D display using only a Kinect. Instead of the usual Kinect hacks like computer vision or playing Mario, we think the ‘Holographic display’ for the Kinect is one of the most useful implementations of the Kinect we’ve seen.

The build uses the Kinect SDK. The end result is a lot like the Kinect snowglobe we covered earlier, only instead of custom acrylic domes and a projector, this build enables pseudo-3D on any display. The hack works by having the Kinect track the users head. From that, it’s pretty simple to display a projection of a 3D model.

The system only supports a 3D display for one person, although with shutter glasses, that could be could be doubled. [programming4fun] says he’s thinking about adding anaglyph 3D – while the effect on video is pretty amazing, he says it’s not entirely convincing in real life.

We talked to the extremely modest [programming4fun], and he says the build isn’t technologically impressive at all. We’d disagree with that assessment because interfaces requiring movement have been around since 1963 and they still haven’t caught on for a wide audience. While there have been a few motion controlled devices that have worked well, most of them have been pretty bad. The Kinect 3D display seems like it would have some utility with a Microsoft Surface type device.

[programming4fun] says if there’s enough interest he might be convinced to clean up his code and create an installer. Check out the video after the break.

UPDATE: [programming4fun] uploaded another demo with a “behind the scenes” look after this was published. Check it out.

[youtube=http://www.youtube.com/watch?v=YDHJEbTPs7o&w=470]

31 thoughts on “3D Display Using A Kinect

  1. I’m kind of surprised this wasn’t a day one project for Microsoft after Johnny Lee’s headtacking demos with a Wii remote and LED glasses. Seems like a neat (if occasionally silly) thing to slap on various first-person games.

  2. Finally !! I’ve said it before, I Will not buy a Kinect until someone does exactly this. NOw instead of things popping out why not try making a window into another wOrLd. This is what I want a Kinect to do: http://hackaday.com/2010/04/16/virtual-windows-that-track-a-viewers-position/ .

    Can you imagine having this as your screen saver, On your PC or Xbox.. Im in the Mile High but I want a Penthouse NewYork View today…Tomorrow I’ll have a window overlooking Tokyo’s skyline…..

  3. This looks like it would go along very well with a 3D glasses setup. As-is, this won’t provide a 3D depth as it’s still on a flat screen. But with 3D glasses, this looks like it would be able to provide a true 3D experience where you can peer around an object, something that typical 3D setups can’t do.

  4. This is the second time I’ve seen this done, but this seems to be the better implementation. The only serious problem I can see is that it will only work for one person at a time because it can only track one set of eyes.

    But this still wouldn’t look 100% right in person anyway because the observer isn’t using stereoscopic vision to view the object. The answer to that is to keep one eye shut of course, then it would look normal, and would probably blend in with the background pretty well.

    Perhaps a more realistic model would enhance the effect further.

    You watch, in a year or so we’ll be seeing this on sale as a SolidWorks plug-in :D

  5. Absolutely stunning. I also like the second video, where he shows the effect using a beamer on a wall. It looks really spooky.

    I guess, wardy is right though, this looks better on video than in real life, because of the missing stereoscopic view. So perhaps the next step would be to combine that with shutter glasses and you have your holo deck.

  6. Thank you for the article and the comments on my little hack!

    To answer some questions:
    I didn’t know Winscape, but it actually looks even simpler to do with Kinect than this: since distances are so big, there is no perspective change involved. It would simply be a matter of rotating the camera around a huge hemisphere projecting a panoramic image or half a cube with cube maps. Video backgrounds would be somewhat more challenging to implement and obtain.

    Open source… well, the demo uses my own garage-built 3D engine. It isn’t particularly well written and uses old-fashioned DirectX9, so I don’t think anybody would be interested in that. The relevant code for this demo is perhaps 10 lines of code and 10 lines of shadercode. I’ll try to set up a blog this week with this relevant code and some explanation on how to use it in your own project. Microsoft made it very clear in their SDK EULA that commercial use is strictly forbidden at this point.

    Thanks again!
    Davy

  7. So, its basicly what Johney-Lee did with a wiimote set up, but for the Kinect (which means no IR lights either side of your eyes needed).

    Cool, but given Microsoft employs Johney Lee, it seems bezire this isnt commercial for all first person games already.

    Id also like to see Nintendo do this for the 3DS, but that would have the (slight) added complication of needing to flip the L/R eyes of their autosteroscopic display based on where you are.
    Shouldnt be a big issue though, this form of headtracking has already been demod on the DSi, and the 3DS has a simliar front-facing camera and a lot more power to play with.

  8. mmm … I’m using this technique for a video installation I’m finishing now, soon to be installed in a science museum.

    but combining it with video maping…

    more info later

  9. Very nice,
    To make the display 3D you’d really need to know the position of the eyes and to know that the precise angle of the head rather than just it’s position in 3D-space, but by knowing its position and the display’s position and assuming that it will be directed at the display and horizontal to the floor you can overcome that I imagine.

  10. BTW that projection on the wall version is especially eye-catching, and you’d sell that to advertisers in no time, you can set it up that only one person at a time is in the vicinity like in a doorway or something
    I bet intel or valvesoftware or google for instance would not mind paying you a few grand to have that catch the attention of the guys with money at some tradeshow or visiting their HQ, and then the single-person limitation actually helps keep it original and interesting.
    Might also be cute to distract people and catch their attention as they go through metaldetectors at entrances, which also limits things to one person at a time.

  11. @Olm- I know, and I avoid ads as much as possible too, but that’s also because the focus of advertisers is on annoying people and making their brandname stick in their heads that way, but you can do nice advertising too if you choose and as I said you could have say intel use it to wow steve jobs when he visits for talks – which is limited and 1 on 1 stuff.

  12. @will As said the original is the one with the wiimote, and it doesn’t matter to me who’s on first anymore, the internet ‘somebody already said that’ thing has saturated me to such concerns, and I’m sure this guy did not steal the idea to claim it his from the ones you linked to, it’s one of those ideas a number of people get, a number but not that many.
    I also like the idea of using it on a horizontal monitor and with a projector, that adds a lot to it.

    1. This would be perfect for a 3D (Stereo 3D) “Phantogram” where the image appears to come up out of the table. It’s impressive as it is, but adding the motion tracking would be fantastic.

Leave a Reply to HirudineaCancel reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.