[Francois] over at 1024 Architecture has been working on a project we think you’ll be likely to see in a professional music video before too long. Using his Kinect sensor, ha has been tracking skeletal movements, adding special effects to the resulting wire frame with Quartz Composer. While this idea isn’t new, the next part is. He takes the QC tweaked video stream and then projects it back over the performer using MadMapper to match the video to the body movements, recording the resultant display.
The project started out with a few hiccups, including a noticeable delay between the body tracking and the display. It caused the performer to have to move more slowly than he would like, so things had to be tweaked. [Francois] first tested the latency between his computer and the projector by displaying a timecode representation on the screen as well as via the projector. He found the projector to have a latency of 1 frame at 60 fps, which wasn’t too bad. This led him to believe the culprit was his Kinect, and he was right. There was a 6 frame delay, so he locked the video output to 30 fps in hopes of cutting that delay in half.
The effect is slightly reminiscent of Tron, but with more distortion. We can’t wait to see more projects similar to this one in the future.
The resulting video embedded below is pretty cool in our opinion, but you can judge for yourself.
15 thoughts on “Real-time Wireframe Video Effect Overlay With Kinect”
That looks amazing, it’s a shame kinect is so expensive..
Wow, loved the part where he shakes a virtual dangling rope at 20 seconds into the video! Not much into the dancing, but to each their own. I’m actually quite surprised how quickly “real” applications came to be after how quickly it was hacked! I’m also surprised how little delay the Kinect has. Not bad for $150!
@Solidacid Expensive? I’m guessing you aren’t aware that before the Kinect this kind of tech cost many thousands of dollars.
This looks fantastic and I’m glad to see some artistic applications coming out of it so quickly.
Change the scheme, alter the mood, electrify the boys and girls!
I wonder if adding a little motion compensation/prediction (like FPS games do to compensate for connection latency) would smooth out some of the action (with a slight penalty for rapid changes in direction).
150 dollars vs 10000..ya expensive. Nice work!
granted expensive relates to how much ypu can afford
it might not be relatively expensive bit it IS expensive. if you disagree, feel free to send me one, i’d love to get hacking..
DAMN IT SOMEONE GOT THERE BEFORE ME!
@Solidacid buying for me=not expensive, buying for others very expensive
Altering and controlling projected reality. This will be the greatest user interface since buttons and levers.
You mean you have to use your hands?
That’s like a baby’s toy!
very bad ass
This looks very similar to a technique that was used on Kylie Minogue’s video “Get Outta My Way”, but I am sure is a WHOLE lot less expensive than the system that was used there. Just as impressive, though!
Please be kind and respectful to help make the comments section excellent. (Comment Policy)