Supercon: Alex Hornstein’s Adventures In Hacking The Lightfield

We are all familiar with the idea of a hologram, either from the monochromatic laser holographic images you’ll find on your bank card or from fictional depictions such as Princes Leia’s distress message from Star Wars. And we’ve probably read about how the laser holograms work with a split beam of coherent light recombined to fall upon a photographic plate. They require no special glasses or headsets and  possess both stereoscopic and spatial 3D rendering, in that you can view both the 3D Princess Leia and your bank’s logo or whatever is on your card as 3D objects from multiple angles. So we’re all familar with that holographic end product, but what we probably aren’t so familiar with is what they represent: the capture of a light field.

In his Hackaday Superconference talk, co-founder and CTO of holographic display startup Looking Glass Factory Alex Hornstein introduced us to the idea of the light field, and how its capture is key to  the understanding of the mechanics of a hologram.

Capturing the light field with a row of GoPro cameras.
Capturing the light field with a row of GoPro cameras.

His first point is an important one, he expands the definition of a hologram from its conventional form as one of those monochromatic laser-interference photographic images into any technology that captures a light field. This is, he concedes, a contentious barrier to overcome. To do that he first has to explain what a light field is.

When we take a 2D photograph, we capture all the rays of light that are incident upon something that is a good approximation to a single point, the lens of the camera involved. The scene before us has of course countless other rays that are incident upon other points or that are reflected from surfaces invisible from the single point position of the 2D camera. It is this complex array of light rays which makes up the light field of the image, and capturing it in its entirety is key to manipulating the result. This is true no matter the technology used to bring it to the viewer. A light field capture can be used to generate variable focus 2D images after the fact as is the case with the Lytro cameras, or it can be used to generate a hologram in the way that he describes.

One possible future use of the technology, a virtual holographic aquarium.
One possible future use of the technology, a virtual holographic aquarium.

The point of his talk is that complex sorcery isn’t required to capture a light field, something he demonstrates in front of the audience with a volunteer and a standard webcam on a sliding rail. Multiple 2D images are taken at different points, which can be combined to form a light field. The fact that not every component of the light field has been captured doesn’t matter as much as that there is enough to create the holographic image from the point of view of the display. And since he happens to be head honcho at a holographic display company he can show us the result. Looking Glass Factory’s display panel uses a lenticular lens to combine the multiple images into a hologram, and is probably one of the most inexpensive ways to practically display this type of image.

Since the arrival of the Lytro cameras a year or two ago the concept of a light field is one that has been in the air, but has more often been surrounded by an air of proprietary marketing woo. This talk breaks through that to deliver a clear explanation of the subject, and is a fascinating watch. Alex leaves us with news of some of the first light field derived video content being put online and with some decidedly science-fiction possible futures for the technology. Even if you aren’t planning to work in this field, you will almost certainly encounter it over the next few years.

21 thoughts on “Supercon: Alex Hornstein’s Adventures In Hacking The Lightfield

    1. Slicing images into lines and viewing them through Lenticular lens, has a long history lookup “Lenticular printing”.

      Using full motion video is interesting and appealing in many ways, and I think it should be explored as a media for communication (I can picture it being used for telepresence conference calls if the images can be sliced up and combined prior to compression and transmission to reduce bandwidth requirements). But as a friend of mine put it, it is not real 3D unless you can look up a Scots kilt and check if they are going commando.

    2. I rather thought he was not a good speak. Far too much casual pausing and memes. And the ums, at least early on. I think he hit his stride midway once he got into the meat of the subject.

      Still enjoyed the presentation overall.

    1. Also very interested in this. I’ve been looking at their patent applications but I’ve only seen designs similar to their previous products (light being projected into the viewing volume at a right angle and scattered).

      Given that the footage linked to in the article is a 5×9 array of small videos from different angles I assume there’s either an array of 45 spherical integral lenses between the glass and LCD or the glass contains the lens array.

      Its difficult to tell from any of the footage released (maybe someone who’s seen it in person can answer this) but I assume the front glass is actually acrylic and might actually be a hollow box. Lenticular lens arrays are only a few mm thick and a higher quality one might be thicker but the front glass is *really* thick. Unless that’s somehow required for the functioning of the device I think the front glass (and the “picture frame” rectangular outline visible inside it) is actually just a visual cue of what the functional field of view of the display is.

    1. I was going to say something like this. Considering I remember first seeing the Lytro proof-of-concept prototype a while ago and the speed at which time flies compared to my perception these days, that means I had to first read about them _at least_ a decade ago. It seems I wasn’t wrong…

  1. Arn’t you guys a wee bit worried that everything lookglass has done thus far, had been done before. I consider myself a hologram master…over the years i have watch them come and go.. I have thought of most of them many years before someone else…I still have a few holograms that no one has put out there yet. That actually do whats expected…my biggest problem is no capital and watch companies like this put ideas out there thats been around . Need to form a team or join one ..Stay Tune

  2. The speaker is overselling light fields a bit by saying they are like holograms but digital instead of analog. Holograms support an infinity of viewpoints and focus planes whereas light fields are limited to a discrete, often small (45 here) number of viewpoints.

    I’ve seen an hologram of a microscope observing a fly. If you put your eye on the entrance pupil of the virtual microscope you can see a magnified view of the fly. You can’t do that with a light field display, you’re mostly lacking focus cues. And there is no comparison in resolution. With 45 views for a light field display, even with a 4K LCD you only get a 320p resolution per view.

    But in VR light fields are still the closest thing to reality you can experiment, you can try Welcome to Light Fields on Steam if you have a headset. Unfortunately only static views for now since the amount of data is enormous, even if it compresses well (generally 1:1000, still resulting in 50~200MB for a single image).

    1. The same thing could be said of digital cameras. “They aren’t really pictures, they are just a discrete number of pixels”. Yes it is still a long way from the analogue holograms, but you’ve got to start somewhere. This technology can definitely be used to produce focal cues as well, but there is a trade-off between the number of discrete focal planes and field of view (for a given number of ray angles). People are also experimenting with temporal multiplexing and compressive algorithms to increase the effective resolution (both spatial and angular).

  3. I have designed a real volumetric device (https://www.lumindustries.com/3d-vis/) and it is really volumetric: many people at the same time can look at it, from every point of view (the new one, not show, you can even look at the 3d object from the bottom) but all these people calling hologram/real 3D even pepper’s ghost illusion – based pyramids, and POV displays that show a flat rendering of a 3d moving object, really confuse the audience. I have been to many faires and every time have to explain that having just one (POV displays), four (pepper’s ghost pyramids) or a limited set of point of views on plane (lightfield, 3Ds screen) is NOT a hologram, nor a real volumetric image.

  4. Lenticular 3D TV’s have been around for a number of years now and don’t have a thick block of plastic stuck to the front and are much bigger too:-

    https://www.amazon.co.uk/Akwil-47Enabl3D-Lenticular-Holographic-Auto-Stereoscopic/dp/B01N20KSGD

    https://www.philips.co.uk/c-p/BDL5071VS_00/signage-solutions-50-inch-edge-led-backlight-ultra-hd-autostereoscopic-3d

    Sony has developed a lenticular sheet that you can stick to your laptop:-

    https://www.cnet.com/news/sony-lenticular-sheet-glasses-free-3d-on-any-laptop/

    And let us not forget the Nintendo 3DS :)

    1. This has a bit more views than 3D TVs, the Enabl3D TV has 9 views and the Philips 27 views, this one has 45 views at a much smaller size.

      I think the thick block of plastic serves a specific purpose, it probabIy allows objects to appear floating in the air instead of stuck to the screen.

      As for the Nintendo 3DS, it’s a parallax barrier and not a lenticular lens and it only has 2 views.

  5. One of the core phenomena that defines holography, in the recording and display of 3D information, is the interference of waves of light. If whatever tech does not utilize interference in some central way then in my opinion it cannot be called holography. I don’t understand why countless techniques for creating and displaying 3D imagery, both old and new, must constantly latch on to (and in doing so dilute the meaning of) the word holography. Are they so insecure in the potential permanence of their creations or innovations that they cannot bring themselves to create a new name for it, or as is much more commonly the case when innovating on an old technique, calling it by its original name? I didn’t need to watch the video, the above photo of the line of cameras was enough to tell me that this does not involve interference and so has nothing to do with holography. Of course I know what motivates this “broadening of definitions” and I must say that more and more, marketing makes me sick.

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.