Anime Inspired Holographic Virtual Assistant

a 3d printed case, sitting on a table with cactuses in the background, with a 3d rendered holo assistant reflected in a cone of polycarbonate sheets from a flat HDMI display pointed up

[Jessp] has created a very cute and endearing DIY virtual assistant called Maria. The build combines a 3D printed housing that uses a modern take on the “Pepper’s Ghost” illusion to render a virtual, three-dimensional anime inspired assistant that can take commands to get information about the weather, play music or set timers.

The hub houses a Raspberry Pi 4B and a 3.2 inch LCD HDMI screen mounted flat on its back to render the perspective corrected “Maria” character using a technique borrowed from the Pepper’s Cone project. Polycarbonate sheets are formed into a cone, allowing for the 3D effect of rendering the virtual assistant model. A consumer grade mini USB microphone is used to receive voice commands along with a consumer grade USB speaker for audio feedback. The virtual assistant offloads the text to speech services to Google Cloud, along with using a weather API and Spotify developer account to for its musical options.

All source code is available on [Jessp]’s GitHub page, including build instructions and STL files for the housing. We’ve featured open source voice assistants in the past, including Mycroft and a even a HAL-9000 virtual assistant (running Kalliope) but it’s nice to see further experimentation in this space.

28 thoughts on “Anime Inspired Holographic Virtual Assistant

    1. It seems like there is no real depth information (no stereo vision), and still it’s convincing enough to trick our vision. So, yes, it’s not hologram, but an illusion.

      The effect used is called “Pepper’s Ghost”, but using a cone instead of flat reflector, hence Pepper’s Cone. You could say it doesn’t exist, but you could say the same of Pepper’s Ghost. It’s just a reflection after all.

      1. You’re the one talking about the “Pepper’s Cone effect”, the article states “… using a technique borrowed from the Pepper’s Cone project. ” – one can call their project any way they want.

        It’s you who is mistaken.

        1. It appears that “any” angle is a bit misleading. As I see it, it can be viewed from only one angle but by detection of the rotation of the projection device itself (and adjusting the projected image for that specific view angle) the image is adjusted so that the illusion of multiple view angles is realized. In other words, 2D with a lame form of tracking, using the accelerometer/gyro in the tablet itself (of which the screen acts like “the projector”). Or with a face/head-tracker observing the location of the viewer and adjusting the projected image accordingly.

          1. Exactly. If you look at the example in the video shown in the article, you can see the screen as well as the reflection. It’s clear that if you looked at it from the side, her image would be on its side. And probably pretty distorted. I skipped through the whole video to see if they ever showed it from another angle, and of course they didn’t. Seems rather pointless, since only one person can view it at a time, and the computer has to know where they are standing.

  1. It’s not a true blue hologram maybe, but a nice optical illusion! I like it! 🙂

    It’s maybe also a cool project for schools to awaken the pupils’/students’ interest in science, computing, holography etc.

    1. Wiring a screen above might be too much of a challenge. An alternative would be to block the view of the screen so light can only travel from the screen to the reflector.
      (Baffles, fresnel lens, or similar)

  2. Related, but not identical: gatebox dot ai slash en is selling something fairly similar to this, but only in Japan (and only speaking Japanese) and apparently it’s somewhat popular. There’s an actual demand for something like this.

  3. I don’t get the reason for using the cone reflector. In the linked Pepper’s cone project, the purpose is to show different perspectives as you rotate the device (still looking from a single vantage point though). Here you can only see the character correctly from the front side, as it would happen with a classical plane glass (which would probably be more effective as it is insensitive to the observer’s height). What am I missing?

    1. I think the advantage here is that you don’t need a servo to move a flat mirror to track the audience such as used in the original peppers ghost.
      I’m not sure why the ghost isn’t viewable from around the cone axis, possibly because the distortion correction math only works correctly that way?

      1. You can see that the screen is placed in front of the cone to project the image only for one direction. I’m pretty confident that the illusion only works if watched from a specific direction. It would be interesting to hear the author’s explanation. For Halloween, I made a quick and dirty “box of ghosts” using a box, an acetate sheet and a tablet. It could be watched from limited angles with no apparent distortion. I don’t think it would have been the same with the cone approach.

  4. On advantage of the cone is that since you can see through it it gives a floating in air effect too.
    Mind you, I find the repeated compulsive/nervous-disorder type motion of the character in the video rather annoying. (No offence meant to people with such disorders.)

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.