Virtual Reality Gets Real with 3 Kinect Cameras

kinects-capture

No, that isn’t a scene from a horror movie up there, it’s [Oliver Kreylos'] avatar in a 3D office environment. If he looks a bit strange, it’s because he’s wearing an Oculus Rift, and his image is being stitched together from 3 Microsoft Kinect cameras.

[Oliver] has created a 3D environment which is incredibly realistic, at least to the wearer. He believes the secret is in the low latency of the entire system. When coupled with a good 3D environment, like the office shown above, the mind is tricked into believing it is really in the room. [Oliver] mentions that he finds himself subconsciously moving to avoid bumping into a table leg that he knows isn’t there. In [Oliver's] words, “It circumnavigates the uncanny valley“.

Instead of pulling skeleton data from the 3 Kinect cameras, [Oliver] is using video and depth data. He’s stitching and processing this data on an i7 Linux box with an Nvidia Geforce GTX 770 video card. Powerful hardware for sure, but not the cutting edge monster rig one might expect. [Oliver] also documented his software stack. He’s using Vrui VR Toolkit, the Kinect 3D Video Capture Project, and the Collaboration Infrastructure.

We can’t wait to see what [Oliver] does when he gets his hands on the Kinect One (and some good Linux drivers for it).

Comments

  1. neo says:

    Do not try and bend the spoon. That’s impossible. Instead… only try to realize the truth.

  2. RunnerPack says:

    It’s not mentioned in his post, but going by his model, I’m assuming he’s not using “Shake ‘n’ Sense”.

  3. markaeric says:

    I saw this mentioned on some other site earlier today but I ignored it. Too bad, because it’s really amazing. The designer did a good job of explaining why it’s actually brilliant given the level of presence it provides (down to being able to interact with your clothing). This is too cool.

  4. Java says:

    I like the idea of creating a virtual enviroment just to be sitting in the same borring room I sit all day at the office…

    pd: red pill for me, please

  5. Doc Oct says:

    It was really amusing when one of the kinects got out of alignment and his form was warped by it so he had multiple left hands and so on. I’d love to try out the out of body experience from doing this sort of thing. The other thing I thought was neat was basically having this meshed AR/VR experience along with the ability to pop up 2d windows. I think it was basically mapped to his view but it would also be neat if you could open a window in 3d space as well.

  6. Waterjet says:

    This could take masturbation to an entirely new level.

  7. millsy says:

    That’s perhaps the most exciting thing I’ve seen since Jonny Chung Lee’s head tracking video.

  8. Steven says:

    Imagine the future of movies
    you could make a 3D animated movie where you are in the scene watching the people do their thing at the same time you can move around and get different views as if you were a ghost watching real events unfold.

  9. El Warpismo says:

    This obviously is a step in the right direction, but I mean, come on, just because I ducked in Captain Eo dosen’t mean that they had circumvented the uncanny valley.

  10. rasz_pl says:

    this needs modded Kinects, swapping vision camera for FullHD one would transfor this from pixelated mes to sharply textured geometry mess :)

    or instead of modding just mount fullHd webcams rigidly to kinect frame above vision camera and calibrate accordingly

    basically kinect 1 has enough depth info, bug VGA vision cam resolution is PATHETIC and destroys the immersion.

    • The kinect v1 depth information is very low resolution. Much lower than the actual sensor resolution of the camera. Also, it doesn’t matter if you increase the resolution of that camera, as the number of IR spots would not increase.

      What would do the trick is to use the kinect v2 sold with the xbone. As it has a Time of Flight 3d cam instead.

      • rasz_pl says:

        depth is 640×480 (interpolated from the IR matrix projection)
        depth is not pa problem here, shitty blurry textures are, upgrading visible light camera to fullHD will transform this from Wolfenstein/Doom level of graphics to Quake 1 being run at fullHD = still shitty geometry, but SHARP textures.

  11. Simon says:

    The comments on his blog refer to much better quality expected from the XBone Kinect which does not use structured light. MS has recently announced that the XBone will be available without Kinect, and that the Kinect will be available as a separate accessory… let the hacking begin.

  12. Kaaaaaaaang says:

    It looks like Frodo when wears the ring.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 96,405 other followers