3D display using a kinect

Youtube user [programming4fun] got a Kinect for father’s day and with just a little bit of code, came up with a 3D display using only a Kinect. Instead of the usual Kinect hacks like computer vision or playing Mario, we think the ‘Holographic display’ for the Kinect is one of the most useful implementations of the Kinect we’ve seen.

The build uses the Kinect SDK. The end result is a lot like the Kinect snowglobe we covered earlier, only instead of custom acrylic domes and a projector, this build enables pseudo-3D on any display. The hack works by having the Kinect track the users head. From that, it’s pretty simple to display a projection of a 3D model.

The system only supports a 3D display for one person, although with shutter glasses, that could be could be doubled. [programming4fun] says he’s thinking about adding anaglyph 3D – while the effect on video is pretty amazing, he says it’s not entirely convincing in real life.

We talked to the extremely modest [programming4fun], and he says the build isn’t technologically impressive at all. We’d disagree with that assessment because interfaces requiring movement have been around since 1963 and they still haven’t caught on for a wide audience. While there have been a few motion controlled devices that have worked well, most of them have been pretty bad. The Kinect 3D display seems like it would have some utility with a Microsoft Surface type device.

[programming4fun] says if there’s enough interest he might be convinced to clean up his code and create an installer. Check out the video after the break.

UPDATE: [programming4fun] uploaded another demo with a “behind the scenes” look after this was published. Check it out.

Comments

  1. fco says:

    it’s great when people do fantastic things and their ego doesn’t fly up.
    Two thumbs up!
    One for the work, the other for his attitude.

  2. mindbleach says:

    I’m kind of surprised this wasn’t a day one project for Microsoft after Johnny Lee’s headtacking demos with a Wii remote and LED glasses. Seems like a neat (if occasionally silly) thing to slap on various first-person games.

  3. sun god says:

    Finally !! I’ve said it before, I Will not buy a Kinect until someone does exactly this. NOw instead of things popping out why not try making a window into another wOrLd. This is what I want a Kinect to do: http://hackaday.com/2010/04/16/virtual-windows-that-track-a-viewers-position/ .

    Can you imagine having this as your screen saver, On your PC or Xbox.. Im in the Mile High but I want a Penthouse NewYork View today…Tomorrow I’ll have a window overlooking Tokyo’s skyline…..

  4. Elias says:

    Very nice hack!

  5. Max says:

    Made me go woah

  6. M4CGYV3R says:

    Do want virtual ping-pong…

  7. GryphoN says:

    This looks like it would go along very well with a 3D glasses setup. As-is, this won’t provide a 3D depth as it’s still on a flat screen. But with 3D glasses, this looks like it would be able to provide a true 3D experience where you can peer around an object, something that typical 3D setups can’t do.

  8. gonzotron says:

    Next step should be to interact and (virtually)touch it with your hands using kinect.

  9. Mike says:

    @sun god and @programming4fun – want to start a project for this? We could make an opensource version of winscape!

    Let me know what you think.

    Mike

  10. Spork says:

    Really cool. I’d like to see more of this, keep up the good work.

  11. wardy says:

    This is the second time I’ve seen this done, but this seems to be the better implementation. The only serious problem I can see is that it will only work for one person at a time because it can only track one set of eyes.

    But this still wouldn’t look 100% right in person anyway because the observer isn’t using stereoscopic vision to view the object. The answer to that is to keep one eye shut of course, then it would look normal, and would probably blend in with the background pretty well.

    Perhaps a more realistic model would enhance the effect further.

    You watch, in a year or so we’ll be seeing this on sale as a SolidWorks plug-in :D

  12. Joost says:

    Great demo!

    Source code or an installer would be awesome!

  13. jukus says:

    Second that – Source Code would be very much appreciated. I would make sure you licence properly with clauses for commercial use though.

  14. TomF says:

    Absolutely stunning. I also like the second video, where he shows the effect using a beamer on a wall. It looks really spooky.

    I guess, wardy is right though, this looks better on video than in real life, because of the missing stereoscopic view. So perhaps the next step would be to combine that with shutter glasses and you have your holo deck.

  15. Thank you for the article and the comments on my little hack!

    To answer some questions:
    I didn’t know Winscape, but it actually looks even simpler to do with Kinect than this: since distances are so big, there is no perspective change involved. It would simply be a matter of rotating the camera around a huge hemisphere projecting a panoramic image or half a cube with cube maps. Video backgrounds would be somewhat more challenging to implement and obtain.

    Open source… well, the demo uses my own garage-built 3D engine. It isn’t particularly well written and uses old-fashioned DirectX9, so I don’t think anybody would be interested in that. The relevant code for this demo is perhaps 10 lines of code and 10 lines of shadercode. I’ll try to set up a blog this week with this relevant code and some explanation on how to use it in your own project. Microsoft made it very clear in their SDK EULA that commercial use is strictly forbidden at this point.

    Thanks again!
    Davy

  16. Darkflame says:

    So, its basicly what Johney-Lee did with a wiimote set up, but for the Kinect (which means no IR lights either side of your eyes needed).

    Cool, but given Microsoft employs Johney Lee, it seems bezire this isnt commercial for all first person games already.

    Id also like to see Nintendo do this for the 3DS, but that would have the (slight) added complication of needing to flip the L/R eyes of their autosteroscopic display based on where you are.
    Shouldnt be a big issue though, this form of headtracking has already been demod on the DSi, and the 3DS has a simliar front-facing camera and a lot more power to play with.

  17. nism0 says:

    Been there, done that. http://www.youtube.com/watch?v=U8l8jxsU2N8

    But still, great demo!

  18. Olm- says:

    mmm … I’m using this technique for a video installation I’m finishing now, soon to be installed in a science museum.

    but combining it with video maping…

    more info later

  19. Whatnot says:

    Very nice,
    To make the display 3D you’d really need to know the position of the eyes and to know that the precise angle of the head rather than just it’s position in 3D-space, but by knowing its position and the display’s position and assuming that it will be directed at the display and horizontal to the floor you can overcome that I imagine.

  20. Whatnot says:

    BTW that projection on the wall version is especially eye-catching, and you’d sell that to advertisers in no time, you can set it up that only one person at a time is in the vicinity like in a doorway or something
    I bet intel or valvesoftware or google for instance would not mind paying you a few grand to have that catch the attention of the guys with money at some tradeshow or visiting their HQ, and then the single-person limitation actually helps keep it original and interesting.
    Might also be cute to distract people and catch their attention as they go through metaldetectors at entrances, which also limits things to one person at a time.

  21. Olm- says:

    @Whatnot : please no, advertising is already too invasive … advertising is not art nor knowledge, only noise.

  22. Hirudinea says:

    Just amazing, hurry up and release the code, finally a reason to but a kinect!

  23. Whatnot says:

    @Olm- I know, and I avoid ads as much as possible too, but that’s also because the focus of advertisers is on annoying people and making their brandname stick in their heads that way, but you can do nice advertising too if you choose and as I said you could have say intel use it to wow steve jobs when he visits for talks – which is limited and 1 on 1 stuff.

  24. Will says:

    It’s been done. See this video from 2010.

    http://kinecthacks.net/crazy-head-tracking-androids/

    And there are a few more jagged ones from mid-november 2010 too.

    It works on the same concept.

  25. xorpunk says:

    This is literally the only way you can do this besides pixel shifting on two lcds. There is no single screen algo without supplemental hardware.

  26. Whatnot says:

    @will As said the original is the one with the wiimote, and it doesn’t matter to me who’s on first anymore, the internet ‘somebody already said that’ thing has saturated me to such concerns, and I’m sure this guy did not steal the idea to claim it his from the ones you linked to, it’s one of those ideas a number of people get, a number but not that many.
    I also like the idea of using it on a horizontal monitor and with a projector, that adds a lot to it.

  27. Will says:

    Yeah, it’s true, and it is a very nice design. It looks like it works very well, from the vid.

  28. midnight says:

    I was thinking if done correctly could this be used in the same way on video glasses for example being able to see it from all angles with motion tracking

    • Eddie Bowers says:

      This would be perfect for a 3D (Stereo 3D) “Phantogram” where the image appears to come up out of the table. It’s impressive as it is, but adding the motion tracking would be fantastic.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s