High Speed Video From Cheap Digital Camera

Some researchers from Oxford University have come up with a way to produce high-speed video from a one mega-pixel camera. They’re calling the method Temporal Pixel Multiplexing. This method adds a digital micromirror device in line with the camera lens. These chips house over a million mirrors and can be found in home theater projectors. By placing one in front of the digital camera, a longer exposure can be used while the DMD redirects the light. This way, one high-resolution image actually contains multiple frames of lower-resolution video. The video is still decent quality and, at a far lower cost than common high-speed video equipment, this is a worthwhile trade off.

[Thanks Andrew via NewScientist]

36 thoughts on “High Speed Video From Cheap Digital Camera

  1. so it just splits up the sensor into smaller matrices and each one of those takes a picture in succession? isn’t that how normal video is taken?

    i don’t understand how you can have both a series of low resolution images (of a video) that combine to make a high resolution picture, it seems like you could have one or the other (if there is movement then the low resolution pictures won’t be able to combine to make a high resolution one, and vice versa). How am I understanding this wrong.

  2. It makes a high resolution image from the series of lower resolution images, nearly equivalent to a single longer exposure. This is actually a really clever hack, because it gets around the problem of needing to read the pixels extremely quickly. The Casio EX-FC100 is a 9 megapixel camera that is able to take 1,000 fps video, but it does it at about 60×200 pixels with relatively poor quality.

  3. It sounds like it might be analogous to old fashioned movie film. Each frame of video is captured on an adjacent ‘section’ of the sensor, like the individual frames of a longer segment of film.

    It reminds me of a hack I pulled off on an old adventure game I was making. The engine couldn’t do full-screen movies, but it was capable of scrolling backgrounds, so I made a huge background with the correct number of frames, and rapidly twiddled the viewport to display my movie.

  4. @kirov
    You are right of course, you don’t get something for nothing.
    The way this works is to capture the image four successive times, on four different sets of pixels that are evenly mixed and spread out across the sensor area. Micro mirrors are used to the light to these pixels.
    After the 4 images are taken the whole ccd will have been exposed, hence the full resolution image. Also you will have 4 distinct time periods when different images were captured, although with 1/4 of the resolution for each frame.

    So far so good, the catch is of course the 4 time periods which make up a single image must be fast if you wish to avoid image blur etc. This means that only a small quantity of light will reach each pixel (1/4 of usual) so the sensor must be much more sensitive, and therefore expensive (or await newer tech…)

    This is obviously an interesting idea but my cell camera is crap enough without restricting the light even more…

  5. This is a bulky, roundabout way to achieve an identical result to what high speed cameras already do. They put multiple images on the same frame, meaning that when the data is read out from the sensor, you have a lot of reduced-resolution images for a much higher framerate, and the same data bandwidth. However, that’s what most high speed cameras already do. Since image sensors are basically giant arrays of shift registers, the most time is spent shifting out all that data, not actually capturing the light. If you only read a smaller portion of the sensor, then you can do it many more times per second. The output is the same as the above approach, but doesn’t require any micromirror devices.

  6. some hobbyist supplier could make a lot of friends with a (relatively) cheap DMD, that doesn’t need an FPGA to control it… (not a lot of people might be willing to dismantle a perfectly good projector just to get the DMD and video input -> control board for projects).

  7. @pascal

    TI recently updated their low-end pico projector kit to allow direct manipulation of the DMD binary patterns.

    It’s only VGA resolution, but it can do 1400-2400 fps with binary patterns and is (only) $350, as opposed to the $5000+ they charge for the high-end dev board. Considering the time and expense required to hack up an FPGA interface, it’s a pretty good deal. (I plan to use one for a high-speed structured light scanner)

    http://focus.ti.com/analog/docs/memsmidlevel.tsp?sectionId=622&tabId=2447

  8. @maceGR
    This is both MASSIVELY cheaper than a traditional high-FPS sensor, and massively simpler. Instead of having to VERY rapidly read off the pixels from a sensor, you just have to flip the mirrors in the DMD quickly (i.e. do what they were designed to do). Its far cheaper and easier to make a large, high resolution, low FPS sensor than to create a small, low resolution, high fps sensor. Plus, you can still take high-resolution images with your sensor.

  9. One big problem with this approach is that all data must be captured RAW with no compression (since compression would annihilate all the motion data encoded in the adjacent pixels). And the idea that your “still” image would be of comparable quality to a regular camera is absurd because as Rod noted each pixel will only be receiving a fraction of the light available.

    So I don’t see the advantage… even if implemented directly on CCD to accommodate high speed video capture, you would have some unwanted jitter because pixels would be “moving” over time.

  10. Using an expensive DMD to make a cheap camera work better isn’t very pactical – but brilliant, creative thinking. I’m inspired. Who knows where great ideas like these may lead? This is why I read Hackaday.

  11. Capturing the images in RAW isn’t a problem or a bad thing. It’s a complete non-issue. Hell, if you are trying to make measurements you’d rather have the RAW image anyhow (or some loss-less compression).

  12. I think you guys are missing the point here, the entire point of the hack is to A) Make a 1 Megapixel camera take highspeed video and to B) Make it do it quite a bit cheaper than what a highspeed camera that already does this would cost. I don’t think that anyone is trying to re-invent the wheel here.

  13. “This method adds a digital micromirror device in line with the camera lens”

    that kind of defeats the tight wadity of the project.

    the whole idea of using a cheap camera (in my view) is to save money from having to pay thousands for a high speed camera and possibly even having the purchase get hung up in the red tape of export restrictions verification ( high speed along with thermal cameras are one of the many controlled devices that is illegal to export and a criminal background check may be performed)

  14. I’m not certain but I believe this is what Nikon already does with their DSLR cameras and this is also why they get the “wobble” effect when doing pans and sweeps with their DSLR cameras. I’ve guessed that they are doing this a WHILE back.

  15. @razor386
    Nope, that’s just the rolling shutter effect.

    @Chris
    You’re underestimating just how expensive high-speed cameras are. Compare a few hundred for the CCD plus a few hundred for the DMD (say, 1000 total), versus several tens of thousands for a high-speed camera.

  16. @intepid
    How are you with car analogies? It sounds kind of like tuning your car suspension for drifting instead of doing the quarter mile.

    You’re simply optimizing performance for a particular kind of behaviour (normally a very expensive kind of behaviour). It’s easy, and cheap to get high pixel count ccds. Wal-Mart has 12 mega-pixel cameras around $150 usd. The same cannot be said for high-speed sensors.

    This is a great hack, and could allow hobbyist researchers access to high-speed video data at low cost.

  17. I didn’t make my point clearly, which was: if you access a smaller portion of the sensor, you can access it at a much higher speed than full frame captures. This is a purely electronic solution using existing technology and is how high speed cameras have worked for decades.

  18. @macegr: that reminds me of a weird effect i got by taking a picture out of a (fast) moving train. the picture wasn’t blurred, but straight vertical lines were bent. one could clearly see, how the camera had already moved forward by the time the bottom rows of the sensor were read out.

  19. tbh this ‘amazing’ idea works on the exact same principles as interlacing which is used as standard in video already and is something that we are desperately trying to get rid of.

    Well done for reinventing the wheel

  20. @tantris
    That’s the Rolling Shutter effect,an artefact of the way pixel data is read out of a sensor

    @macegr
    Yes, that’s how current high-speed capture works. Except the resolution is very low, and the capture hardware is bespoke and VERY expensive. This method is cheap (DMDs are really not that expensive), and uses existing cheap, mass produced capture hardware.

  21. I’m a professional photographer. I got tired of dragging my D3,D200 or other large cameras around, and worrying about it getting damaged. I decided to buy a camera just for fun that could fit in my purse and I wouldn’t have a heart attack if it broke. Well I didn’t hold that high of expectations of the camera, but after getting it, I’ve had a lot of fun with it. Not the most amazing pictures come from it (I am used to very high quality pictures), but they are good enough shots of my family. It’s easy to use, really easy to set, FAST, and good low light. You don’t always need the flash. It’s very small, and can fit easily in my purse (that are all quiet small purses actually). This is a perfect camera if you just want some fun shots that are clear, and you want an easy to use fast camera. This is NOT good for “professional” type shots (why do people write reviews of these cameras and get all down on them for not being professional grade cameras? They never claim to be!).

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.