Holographic Cellphones Coming Thanks To AI

Issac Asimov foresaw 3D virtual meetings but gave them the awkward name “tridimensional personification.” While you could almost do this now with VR headsets and 3D cameras, it would be awkward at best. It is easy to envision conference rooms full of computer equipment and scanners, but an MIT student has a method that may do away with all that by using machine learning to simplify hologram generation.

As usual, though, the popular press may be carried away a little bit. The key breakthrough here is that you can use TensorFlow to generate real-time holograms at a few frames per second using consumer-grade processing power found in a high-end phone from images with depth information, which is also available on some phones. There’s still the problem of displaying the hologram on the other side, which your phone can’t do. So any implication that you’ll download an app that enables holograms phone calls is hyperbole and images of this are in the realm of photoshop.

Still, the idea that you can generate 3D from a camera essentially in real-time is pretty exciting. Even if your phone is a few years away from a holographic display, we can imagine this working its way into VR headsets much sooner. After all, VR headsets must choose if they connect to a big computer or have limited capabilities with an onboard cellphone-like computer.

The technique essentially trained a neural network to predict what a scene would look like as a hologram using supervisory holograms as a reference. However, the recent paper expands on this, using a two-stage training regime with one training set having supervision and another training cycle without supervision.f

If you want to give any of this a shot, the models and other things you’d need are on GitHub. This probably isn’t quite ready for casual weekend hacking, but if you are a hologram guru, perhaps you’ll find a way to get us a little closer to the kind of things you see in popular videos about the topic, like the one below.

We need a good holographic display, of course. If you need a refresher on holograms, talk to [Brian McEvoy].

20 thoughts on “Holographic Cellphones Coming Thanks To AI

  1. Every time this topic comes up, nirvana is just around the corner and the latest thing is the magic key to unlock the future. For the time being I’ll put this latest thing in the same bucket with the previous crop of latest thing.

    But one day someone will make it work, and that should be pretty cool.

    1. Yeah, it’s one of those core handful of perpetually “just around the corner” technologies like nuclear fusion and flying cars that the media and near future scifi love to talk about.
      And like them it IS cool and WILL be a big deal when it actually does happen.

  2. Since the nature link is broken and the video is gibberish, the actual paper and a decent video abstract is here: http://cgh.csail.mit.edu/

    This is useful for cheaply creating the interference pattern needed by a phase-only spatial light modulator display… which will be useful if one of those *ever* gets developed into a consumer-grade hardware device.

    1. Thanks, I’d assumed this was a DeepFocus-like NN for producing interference patterns from RGB-D, but the article was so garbled I was starting to question if it was something else instead.

  3. Great! Can’t wait to see what “your car’s warranty is about to expire”, “you social security account has been suspeneded”, or the “Hello, this is Mike from Microsoft, your computer is infected” will look like in 3D.

  4. On the display end, a system that is aware of the viewer position and updated appropriately could display an image from the hologram in realtime on an ordinary monitor. It wouldn’t be an in-the-room hologram but it would turn a flat monitor image into a window for one viewer. It’d still be a significant improvement.

    1. Amazon tried this with their Fire Phone. It had 4 front facing cameras to track the user’s face and adjust the 3d scene’s camera accordingly.

      It was a giant flop not because of the cameras, but because Amazon insisted on using their also-ran appstore with few compelling apps and even fewer that could take advantage of their face tracking tech.

      Nowadays most phones have an ML accelerator that can do the same with a single camera and much less power draw. Sadly this face tracking capability is not often used for this purpose, and instead feeds TikTok face filters and camera focus algorithms.

    2. I had a similar idea, but concluded for the real effect one needed a 3D TV that also tracked user position – at least the tracking could be facilitated by wearing 3d glasses with markers on them.

  5. Undoubtedly, this time the technology will really be implemented and it will be before 2030 in smartphones. However, I wonder why real holography has not already been implemented in devices with significantly higher performance than smartphones such as desktop computers?

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.