High speed video from cheap digital camera

Some researchers from Oxford University have come up with a way to produce high-speed video from a one mega-pixel camera. They’re calling the method Temporal Pixel Multiplexing. This method adds a digital micromirror device in line with the camera lens. These chips house over a million mirrors and can be found in home theater projectors. By placing one in front of the digital camera, a longer exposure can be used while the DMD redirects the light. This way, one high-resolution image actually contains multiple frames of lower-resolution video. The video is still decent quality and, at a far lower cost than common high-speed video equipment, this is a worthwhile trade off.

[Thanks Andrew via NewScientist]

Comments

  1. nubie says:

    That is cool, neat use of DMD devices.

  2. Holanddez1' says:

    By : Holanddez1′

    Advance

  3. There is a god…

    Hack a Day is getting decent content once again, and it doesn’t even use Arduinos… O.O

  4. googfan says:

    Gotta try

  5. kirov says:

    so it just splits up the sensor into smaller matrices and each one of those takes a picture in succession? isn’t that how normal video is taken?

    i don’t understand how you can have both a series of low resolution images (of a video) that combine to make a high resolution picture, it seems like you could have one or the other (if there is movement then the low resolution pictures won’t be able to combine to make a high resolution one, and vice versa). How am I understanding this wrong.

  6. jzoe says:

    It makes a high resolution image from the series of lower resolution images, nearly equivalent to a single longer exposure. This is actually a really clever hack, because it gets around the problem of needing to read the pixels extremely quickly. The Casio EX-FC100 is a 9 megapixel camera that is able to take 1,000 fps video, but it does it at about 60×200 pixels with relatively poor quality.

  7. andar_b says:

    It sounds like it might be analogous to old fashioned movie film. Each frame of video is captured on an adjacent ‘section’ of the sensor, like the individual frames of a longer segment of film.

    It reminds me of a hack I pulled off on an old adventure game I was making. The engine couldn’t do full-screen movies, but it was capable of scrolling backgrounds, so I made a huge background with the correct number of frames, and rapidly twiddled the viewport to display my movie.

  8. Rod says:

    @kirov
    You are right of course, you don’t get something for nothing.
    The way this works is to capture the image four successive times, on four different sets of pixels that are evenly mixed and spread out across the sensor area. Micro mirrors are used to the light to these pixels.
    After the 4 images are taken the whole ccd will have been exposed, hence the full resolution image. Also you will have 4 distinct time periods when different images were captured, although with 1/4 of the resolution for each frame.

    So far so good, the catch is of course the 4 time periods which make up a single image must be fast if you wish to avoid image blur etc. This means that only a small quantity of light will reach each pixel (1/4 of usual) so the sensor must be much more sensitive, and therefore expensive (or await newer tech…)

    This is obviously an interesting idea but my cell camera is crap enough without restricting the light even more…

  9. macegr says:

    This is a bulky, roundabout way to achieve an identical result to what high speed cameras already do. They put multiple images on the same frame, meaning that when the data is read out from the sensor, you have a lot of reduced-resolution images for a much higher framerate, and the same data bandwidth. However, that’s what most high speed cameras already do. Since image sensors are basically giant arrays of shift registers, the most time is spent shifting out all that data, not actually capturing the light. If you only read a smaller portion of the sensor, then you can do it many more times per second. The output is the same as the above approach, but doesn’t require any micromirror devices.

  10. pascal says:

    some hobbyist supplier could make a lot of friends with a (relatively) cheap DMD, that doesn’t need an FPGA to control it… (not a lot of people might be willing to dismantle a perfectly good projector just to get the DMD and video input -> control board for projects).

  11. kirov says:

    @rod:

    that is a really clever observation, that with frame rates so fast the motion blur is minimal and they can be combined into a higher resolution image.

    reminds me of the old (mechanical) televisions ( http://en.wikipedia.org/wiki/Mechanical_television ) which used a spinning disk with pinholes where light would pass through

  12. jonzilla9000 says:

    @pascal

    TI recently updated their low-end pico projector kit to allow direct manipulation of the DMD binary patterns.

    It’s only VGA resolution, but it can do 1400-2400 fps with binary patterns and is (only) $350, as opposed to the $5000+ they charge for the high-end dev board. Considering the time and expense required to hack up an FPGA interface, it’s a pretty good deal. (I plan to use one for a high-speed structured light scanner)

    http://focus.ti.com/analog/docs/memsmidlevel.tsp?sectionId=622&tabId=2447

  13. EdZ says:

    @maceGR
    This is both MASSIVELY cheaper than a traditional high-FPS sensor, and massively simpler. Instead of having to VERY rapidly read off the pixels from a sensor, you just have to flip the mirrors in the DMD quickly (i.e. do what they were designed to do). Its far cheaper and easier to make a large, high resolution, low FPS sensor than to create a small, low resolution, high fps sensor. Plus, you can still take high-resolution images with your sensor.

  14. intepid says:

    One big problem with this approach is that all data must be captured RAW with no compression (since compression would annihilate all the motion data encoded in the adjacent pixels). And the idea that your “still” image would be of comparable quality to a regular camera is absurd because as Rod noted each pixel will only be receiving a fraction of the light available.

    So I don’t see the advantage… even if implemented directly on CCD to accommodate high speed video capture, you would have some unwanted jitter because pixels would be “moving” over time.

  15. Chris says:

    Using an expensive DMD to make a cheap camera work better isn’t very pactical – but brilliant, creative thinking. I’m inspired. Who knows where great ideas like these may lead? This is why I read Hackaday.

  16. Blind says:

    Capturing the images in RAW isn’t a problem or a bad thing. It’s a complete non-issue. Hell, if you are trying to make measurements you’d rather have the RAW image anyhow (or some loss-less compression).

  17. Tachikoma says:

    DMD hacks are quite interesting. Here is another one used as a “single pixel camera” (compressive imaging):

    http://dsp.rice.edu/cscamera

  18. Merrick says:

    Or from 2004 (using webcams): http://graphics.stanford.edu/papers/highspeedarray/

    I was thinking of trying light field photography on a single DLSR CMOS sensor (since it scans anyway) for high speed.

    See: http://graphics.stanford.edu/papers/lfcamera/

  19. Josh says:

    I think you guys are missing the point here, the entire point of the hack is to A) Make a 1 Megapixel camera take highspeed video and to B) Make it do it quite a bit cheaper than what a highspeed camera that already does this would cost. I don’t think that anyone is trying to re-invent the wheel here.

  20. ejonesss says:

    “This method adds a digital micromirror device in line with the camera lens”

    that kind of defeats the tight wadity of the project.

    the whole idea of using a cheap camera (in my view) is to save money from having to pay thousands for a high speed camera and possibly even having the purchase get hung up in the red tape of export restrictions verification ( high speed along with thermal cameras are one of the many controlled devices that is illegal to export and a criminal background check may be performed)

  21. razor386 says:

    I’m not certain but I believe this is what Nikon already does with their DSLR cameras and this is also why they get the “wobble” effect when doing pans and sweeps with their DSLR cameras. I’ve guessed that they are doing this a WHILE back.

  22. EdZ says:

    @razor386
    Nope, that’s just the rolling shutter effect.

    @Chris
    You’re underestimating just how expensive high-speed cameras are. Compare a few hundred for the CCD plus a few hundred for the DMD (say, 1000 total), versus several tens of thousands for a high-speed camera.

  23. jeditalian says:

    video after the break?
    was there a video at all? c’mon i dont want to read about high speed photography, i want to see it in action!

  24. razor386 says:

    Rolling shutter? The mirror is flipped up and the sensor is exposed the entire time I believe or are you talking about something different?

  25. Freax says:

    @razor386:
    The camera still has to read the single images, and it doesn’t do this all pixels at the same time, but one after another. So you get a similar effect.

  26. rallen says:

    @intepid
    How are you with car analogies? It sounds kind of like tuning your car suspension for drifting instead of doing the quarter mile.

    You’re simply optimizing performance for a particular kind of behaviour (normally a very expensive kind of behaviour). It’s easy, and cheap to get high pixel count ccds. Wal-Mart has 12 mega-pixel cameras around $150 usd. The same cannot be said for high-speed sensors.

    This is a great hack, and could allow hobbyist researchers access to high-speed video data at low cost.

  27. tantris says:

    dmds are still expensive
    a suggestion for a bulkier but cheaper solution:
    get 4 cheap webcams and a rotating first surface mirror (from a broken laser printer).

  28. Freax says:

    @tantris:that will not work, because your rotating mirror can’t switch between den cameras in a short time and will “smear” the image over the sensors.

  29. tantris says:

    @freax: yes, you’d need a slotted disc as well. problem might be the short exposure time however.

  30. macegr says:

    I didn’t make my point clearly, which was: if you access a smaller portion of the sensor, you can access it at a much higher speed than full frame captures. This is a purely electronic solution using existing technology and is how high speed cameras have worked for decades.

  31. tantris says:

    @macegr: that reminds me of a weird effect i got by taking a picture out of a (fast) moving train. the picture wasn’t blurred, but straight vertical lines were bent. one could clearly see, how the camera had already moved forward by the time the bottom rows of the sensor were read out.

  32. bob says:

    tbh this ‘amazing’ idea works on the exact same principles as interlacing which is used as standard in video already and is something that we are desperately trying to get rid of.

    Well done for reinventing the wheel

  33. EdZ says:

    @tantris
    That’s the Rolling Shutter effect,an artefact of the way pixel data is read out of a sensor

    @macegr
    Yes, that’s how current high-speed capture works. Except the resolution is very low, and the capture hardware is bespoke and VERY expensive. This method is cheap (DMDs are really not that expensive), and uses existing cheap, mass produced capture hardware.

  34. Baros says:

    I’m a professional photographer. I got tired of dragging my D3,D200 or other large cameras around, and worrying about it getting damaged. I decided to buy a camera just for fun that could fit in my purse and I wouldn’t have a heart attack if it broke. Well I didn’t hold that high of expectations of the camera, but after getting it, I’ve had a lot of fun with it. Not the most amazing pictures come from it (I am used to very high quality pictures), but they are good enough shots of my family. It’s easy to use, really easy to set, FAST, and good low light. You don’t always need the flash. It’s very small, and can fit easily in my purse (that are all quiet small purses actually). This is a perfect camera if you just want some fun shots that are clear, and you want an easy to use fast camera. This is NOT good for “professional” type shots (why do people write reviews of these cameras and get all down on them for not being professional grade cameras? They never claim to be!).

  35. Thanks for the great write-up. , Content is good. I would hope to be back tracking.

  36. Jody Ibara says:

    I would hope to be back tracking. Thanks for the great write-up.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 96,357 other followers