Real-time Wireframe Video Effect Overlay With Kinect


[Francois] over at 1024 Architecture has been working on a project we think you’ll be likely to see in a professional music video before too long. Using his Kinect sensor, ha has been tracking skeletal movements, adding special effects to the resulting wire frame with Quartz Composer. While this idea isn’t new, the next part is. He takes the QC tweaked video stream and then projects it back over the performer using MadMapper to match the video to the body movements, recording the resultant display.

The project started out with a few hiccups, including a noticeable delay between the body tracking and the display. It caused the performer to have to move more slowly than he would like, so things had to be tweaked. [Francois] first tested the latency between his computer and the projector by displaying a timecode representation on the screen as well as via the projector. He found the projector to have a latency of 1 frame at 60 fps, which wasn’t too bad. This led him to believe the culprit was his Kinect, and he was right.  There was a 6 frame delay, so he locked the video output to 30 fps in hopes of cutting that delay in half.

The effect is slightly reminiscent of Tron, but with more distortion. We can’t wait to see more projects similar to this one in the future.

The resulting video embedded below is pretty cool in our opinion, but you can judge for yourself.

[vimeo w=470%5D

15 thoughts on “Real-time Wireframe Video Effect Overlay With Kinect

  1. Wow, loved the part where he shakes a virtual dangling rope at 20 seconds into the video! Not much into the dancing, but to each their own. I’m actually quite surprised how quickly “real” applications came to be after how quickly it was hacked! I’m also surprised how little delay the Kinect has. Not bad for $150!

  2. @Solidacid Expensive? I’m guessing you aren’t aware that before the Kinect this kind of tech cost many thousands of dollars.

    This looks fantastic and I’m glad to see some artistic applications coming out of it so quickly.

  3. I wonder if adding a little motion compensation/prediction (like FPS games do to compensate for connection latency) would smooth out some of the action (with a slight penalty for rapid changes in direction).


  4. This looks very similar to a technique that was used on Kylie Minogue’s video “Get Outta My Way”, but I am sure is a WHOLE lot less expensive than the system that was used there. Just as impressive, though!

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.