Real-time wireframe video effect overlay with Kinect

linedance

[Francois] over at 1024 Architecture has been working on a project we think you’ll be likely to see in a professional music video before too long. Using his Kinect sensor, ha has been tracking skeletal movements, adding special effects to the resulting wire frame with Quartz Composer. While this idea isn’t new, the next part is. He takes the QC tweaked video stream and then projects it back over the performer using MadMapper to match the video to the body movements, recording the resultant display.

The project started out with a few hiccups, including a noticeable delay between the body tracking and the display. It caused the performer to have to move more slowly than he would like, so things had to be tweaked. [Francois] first tested the latency between his computer and the projector by displaying a timecode representation on the screen as well as via the projector. He found the projector to have a latency of 1 frame at 60 fps, which wasn’t too bad. This led him to believe the culprit was his Kinect, and he was right.  There was a 6 frame delay, so he locked the video output to 30 fps in hopes of cutting that delay in half.

The effect is slightly reminiscent of Tron, but with more distortion. We can’t wait to see more projects similar to this one in the future.

The resulting video embedded below is pretty cool in our opinion, but you can judge for yourself.

Comments

  1. Solidacid says:

    That looks amazing, it’s a shame kinect is so expensive..

  2. Steve says:

    Wow, loved the part where he shakes a virtual dangling rope at 20 seconds into the video! Not much into the dancing, but to each their own. I’m actually quite surprised how quickly “real” applications came to be after how quickly it was hacked! I’m also surprised how little delay the Kinect has. Not bad for $150!

  3. Bob D says:

    @Solidacid Expensive? I’m guessing you aren’t aware that before the Kinect this kind of tech cost many thousands of dollars.

    This looks fantastic and I’m glad to see some artistic applications coming out of it so quickly.

  4. Change the scheme, alter the mood, electrify the boys and girls!

  5. darkith says:

    I wonder if adding a little motion compensation/prediction (like FPS games do to compensate for connection latency) would smooth out some of the action (with a slight penalty for rapid changes in direction).

    D.

  6. IJ Dee-Vo says:

    150 dollars vs 10000..ya expensive. Nice work!

  7. IJ Dee-Vo says:

    granted expensive relates to how much ypu can afford

  8. Solidacid says:

    it might not be relatively expensive bit it IS expensive. if you disagree, feel free to send me one, i’d love to get hacking..

  9. EFH says:

    Extremely cool!

  10. Grovenstien says:

    DAMN IT SOMEONE GOT THERE BEFORE ME!

  11. IJ Dee-Vo says:

    @Solidacid buying for me=not expensive, buying for others very expensive

  12. echodelta says:

    Altering and controlling projected reality. This will be the greatest user interface since buttons and levers.

  13. t&p says:

    You mean you have to use your hands?
    That’s like a baby’s toy!

  14. bzroom says:

    very bad ass

  15. BristolGarry says:

    This looks very similar to a technique that was used on Kylie Minogue’s video “Get Outta My Way”, but I am sure is a WHOLE lot less expensive than the system that was used there. Just as impressive, though!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 96,369 other followers