Unwrapping 360 degree video

[Golan Levin] found a way to unwrap the 360 degree images he created with his camera. He’s using a Sony Bloggie HD camera which comes with a 360 degree attachment for the lens. This produces a donut shaped image (seen in the upper left) that was not all that palatable to [Golan]. He used Processing and openFrameworks to create a program that lets him unwrap the donut into a flat image, or create a ring of video where the viewer is at the center and can scroll left or right to see the rest of the filmed environment. He released the source so you can adapt the program if you’re using a different 360 video setup.

[Thanks Kyle]

Comments

  1. SheeEttin says:

    I don’t know about video, but the GIMP can do this for still images. It’s a simple polar-to-rect conversion. (Filters > Distorts > Polar coordinates.)

  2. cknopp says:

    I’m surprised that a camera that comes with a 360 degree lense doesnt come with software to make it usable.

    This is exactly why the users are better than the manufacturers. A manufacturer has to pay someone to hit quota on drivers and such, where a man with a need will always deliver a better product!

    If only the manufacturers would pick up on this, and pay these types to develop software in their homes for the cost of a single product, the user base could increase exponentially due to better ability to maximize the potential of the products.

  3. Dielectric says:

    I saw this working at the hardware level a few years ago with some crazily expensive video processing chips. They could use a single image sensor looking at a mirrored hemisphere and do a virtual pan/tilt/zoom transform. You’d get a PTZ surveillance cam with no moving parts, and no way to tell which way it was looking at any given time.

  4. Travis says:

    If you think the simple unwrapping is cool, you should look at the more general class of “computational cameras”:

    http://www.hizook.com/blog/2009/06/26/computational-cameras-exploiting-megapixels-and-computers-redefine-modern-camera

    You can do some amazing things with mirrors & lenses.

  5. Whatnot says:

    Weird to read this:
    “The Sony Bloggie is capable of recording non-panoramic HD video at resolutions up to 1920×1080 (30fps). Unfortunately (and this isn’t well-advertised), the Bloggie only records panoramic digital video at 1280×720. In video mode, the camera auto-detects the panoramic lens attachment (magnetically, we suspect) and switches to the lower resolution.”

    What the hell is that all about? You’d think it would force max resolution if it has to capture not a small bit of a scene but a full 360.
    But it’s sony eh, they often do things seemingly mainly simply to annoy their customers.

  6. biozz says:

    you can do this in photoshop with a filter tool … i dont have it on hand but i use it to do the revere and wrap panoramics

  7. Matt Downer says:

    I’ve experimented with this a few times a couple of months ago. Found some online group that made several apps to suit different needs for 360 ideas. I downloaded a few and it actually worked quite well. You really should use a HD camera though as I used my Panasonic TZ3 which has good clear reso but still came out a bit fuzzy (360 video). Photos work fine! Give it a go! I went pretty cheap and used things like metal bowls, Christmas tree glass balls, appliance caps. Not sure what else there is that could be used for a reflective dome.

  8. Muzza says:

    “Although Sony released unwarping software for Microsoft Windows, they — inexplicably — didn’t release Bloggie unwarping software for Mac OSX.”

    This is from the web site…. so they did release unwarping software…. just not for poop operating systems.. I mean that a big challenge to write a program to do the conversion but you could have just used the windows software. :)

  9. Golan Levin says:

    Hi, Golan here… author of the Bloggie project posted above.

    SheeEttin, Biozz: It is well-known that Photoshop and GIMP implement polar-to-rectangular conversion as a simple filter. The project I present here (1) does this with video as well as stills; (2) can be trivially modified to work with *real-time* polar video fed from the Bloggie’s analog output, and (3) offer complete source code for popular open-source programming toolkits. Being able to unwarp real-time video means that it is then possible to create computer-vision based interactions that use panoramic video — for example, 360-degree movement detection or face recognition.

    CKnopp: Sony has released Bloggie unwarping software for Windows, but not Mac or Linux; that was part of my motivation in releasing code for the cross-platform Processing and OpenFrameworks environments, which can compile across all three operating systems. I also agree that Sony’s Windows software is not that great; for example, it assumes that the Bloggie lens is always perfectly centered, when in fact, it has quite a range of movement.

    Whatnot: I can’t explain why the Bloggie switches to a lower resolution for the panoramic video. It’s dumb! Interestingly, it’s only for the video; the Bloggie can take higher-resolution (5MP) panoramic stills.

    Golan

  10. Cynyr says:

    @Travis,
    Thank you for that, very interesting, well worth the hour of time I spent watching. Being able to use a single lens to get a “3d” image is very interesting, both from a 3d TV view, and a computer vision point of view.

  11. axodus says:

    @Whatnot, Golan
    I think the camera might need to do some processing in order to fix and focus the panoramic image.
    processing means time so sony must have sacrifice resolution in order to maintain a decent fps in the video output. or so it’s my theory…

  12. JanMartin says:

    How the Bloggie switches resolution:

    There are two magnets embedded in the base of the panoramic lens. The Bloggie detects the magnetic field and switches resolution. One can remove one magnet and then apply the lens the one way or the other, picking the resolution.

    Unfortunately it seems that at the higher resolution a part of the resulting doughnut image is cut off. Which is a shame. Me and Golan are investigating this.

    You are welcome to give it a try. Please let me know your findings:
    janmartin AT diy-streetview DOT org

    Jan

  13. Whatnot says:

    @axodus You do think in a ‘sane universe’ there must be a reason, but then again even if that was the reason, that the processing somehow caused it, then why didn’t they improve the hardware to fix that issue instead of this weird broken setup?

    It could also be a stupid bug of course where it is suppose to switch to max resolution but due to a small typo in the code or an error is the documentation about the resolution switches it does the opposite.

  14. dan says:
  15. alberto says:

    @Travis,
    Thank you for posting the link, very very very interesting.

  16. Justin says:

    Um, okay 1st of all the Sony Bloggie does come with an unwrapping software that is very advanced and has other features, so before you flame shit you’d be wise to either buy one and use it or at least review some Youtube videos regards to the topic. This is why the user is the user, they use.. the developers develop, develo(P).

    I don’t like reading stupid shit..

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 96,695 other followers