A Group of MIT, Microsoft, and Adobe researchers have managed to reproduce sound using video alone. The sounds we make bounce off every object in the room, causing microscopic vibrations. The Visual Microphone utilizes a high-speed video camera and some clever signal processing to extract an audio signal from these vibrations. Using video of everyday objects such as snack bags, plants, Styrofoam cups, and water, the team was able to reproduce tones, music and speech. Capturing audio from light isn’t exactly new. Laser microphones have been around for years. The difference here is the fact that the visual microphone is a completely passive device. No laser or special illumination is required.
The secret is in the signal processing, which the team explains in their SIGGRAPH paper (pdf link). They used a complex steerable pyramid along with wavelet filters to obtain local pixel motion values. These local values are averaged into a global motion value. From this global motion value the team is able to measure movement down to 1/1000 of a pixel. Plenty of resolution to decode audio data.
Most of the research is performed with high-speed video cameras, which are well outside the budget of the average hacker. Don’t despair though, the team did prove out that the same magic can be performed with consumer cameras, albeit with lower quality results. The team took advantage of the rolling shutter found in most of today’s CMOS imager based consumer cameras. Rolling shutter CMOS sensors capture images one row at a time. Each row can be processed in a similar fashion to the frames of the high-speed camera. There are some inter-frame gaps when the camera isn’t recording anything though. Even with the reduced resolution, it’s easy to pick out “Mary had a little lamb” in the video below.
We’re blown away by this research, and we’re sure certain organizations will be looking into it for their own use. Don’t pull out your tin foil hats yet though. Foil containers proved to be one of the best sound reflectors.
Continue reading “Focus Your Ears with The Visual Microphone”
We see a lot of video game tech coming out of the three console giants (Microsoft, Sony, and Nintendo). With one look we can usually predict what is going to be a flop. Case and point is the Wii U whose sales have been less than extraordinary and Sony Move which is motion control directed as hardcore games who we believe are perfectly happy with the current evolution of their dual shock controllers. But this time around we think Microsoft has it nailed. They’re showing off technology they call IllumiRoom which uses a projector to bring your entire gaming room into the experience.
The image above is not doctored. This is a picture of IllumiRoom in action. A projector on the coffee table automatically calibrates to the room (using Kinect 3D data for mapping) in order to show realistic graphic rendering on the non-flat projection surfaces. In our mind, this comes straight out of Kinect hacking projects like the Hadouken projector. With this in place, the game designers are given free rein to come up with all kinds of different ways to use the feature. Stick with us after the break to see what they’ve developed.
Continue reading “Microsoft IllumiRoom breaks your video game out of its television prison”
For all of you that found yourselves wanting to use Kinect to control something but had no idea what to do with it, or how to get the data from it, you’re in luck. Kineticspace is a tool available for Linux/mac/windows that gives you the tools necessary to set up gesture controls quickly and easily. As you can see in the video below, it is fairly simple to set up. You do you action, set the amount of influence from each body part (basically telling it what to ignore), and save the gesture. This system has already been used for tons of projects and has now hit version 2.0.
Continue reading “Kinetic Space: software for your Kinect projects”
We think most would agree that the Microsoft Kinect is a miraculous piece of hardware. The affordable availability of a high-quality depth camera was the genesis of a myriad of hacks. And now it seems that type of data is making an intriguing 3D display possible.
What you see above is a 3D monitor concept that Microsoft developed. It starts off looking much like a tablet PC, but the screen can be lifted up toward the user whose arms reach around it to get at the keyboard underneath. There is as depth camera that can see the hands and fingers of the user to allow manipulation of the virtual environment. But that’s only part of the problem. You need some way to align the user’s eyes with what’s on the screen. They seem to have solved that problem too, using another depth camera to track the location of the user’s head. This means that you can lean from one side to the other and the perspective of the virtual 3D desktop will change to preserve the apparent distance of each object.
Don’t miss the show-and-tell video after the break. As long as there’s only one viewer this looks like a perfect non-glasses alternative to current 3D hardware offerings. Continue reading “Microsoft shows off their transparent 3D desktop prototype”
If you’ve got a crazy ingenious idea for Microsoft’s Kinect peripheral, but don’t have the means to make your dream a reality, the Kinect Accelerator just might be the opportunity you’ve been waiting for.
Microsoft, having performed a complete 180-degree turnaround from their initial stance on Kinect hacking, is embracing developers more than ever with this new program. They are offering a $20,000 along with development space to ten startup companies, in hopes of turning out some incredible Kinect applications. At the end of the three month program, each group will have the opportunity to present their creations to a group of angel investors, which is a fantastic opportunity.
Obviously competition to gain entry into the program will be pretty fierce, but if you think you have what it takes, get your application in now. Judging by the Kinect Accelerator FAQ section, this looks to be something geared towards small tech startups rather than individuals, but it never hurts to give it a shot.
Yes, the Kinect is over one year old now, and after some initial unhappiness from [Microsoft], it’s become a hacker’s best friend. [Eric] decided to celebrate this with an Article all about how it works. If you’re new to this piece of hardware and want to get into working with it, this should be a good hacking introduction. If you’ve been reading [HAD] lately, you will have noticed this information being used to “build a Kinect bot for 500 bones.”
Some interesting facts in this article include that the Kinect measures 307200 distance point, known as a “point cloud” in the gaming area. From this, it’s able to construct a 3D image of the environment around it and allow interaction. Such interesting hardware didn’t take long to hack after Adafruit announced a $3000.00 bounty to open it up to the masses. This only took four days to do, making one wonder why, with their incredible resources, [Microsoft] wouldn’t either more effectively lock it down or officially open it to be hacked and modified to begin with. Our vote would be to officially open it up, but no one consulted us on the decision.
Microsoft has thrown its hat into the open source hardware hobby market. Their offering is called the Gadgeteer. We’d love to tell you all about it, but the big M didn’t make it very easy to find out about the device and it’s addons. When we set out to find what processor is running on the board we were happy to see that they do call it an Open Source Hardware project, but no schematic is posted. When we did finally navigate to the hardware documentation it’s a file that must be downloaded and you’ve got to agree to their licensing before grabbing it. So that’s as far as we went, and now we’ll go back to using more open tools.
For those of you who aren’t scared off by the lack of openness, the first thing you’ll notice about this board is that it’s full of connector headers. Instead of the small rows that Arduino uses, the Gadgeteer is meant to use ribbon cables to connect to various breakout boards. You can program for the platform in C# using the .NET framework. This means using Microsoft Visual Studio for those that are already acquainted with the platform. But regular readers will note that we’re always looking for Linux support in our IDEs and you won’t find that here.
[Thanks Hrasdt (and several others) via Slashdot]