Going To The Park With Your Augmented Reality Girlfriend

Lonely? Bored? Really into J-pop? If you’re any of these things, here’s the build for you. It’s an augmented reality system that allows you to go on a date with one of Japan’s most popular virtual singers.

The character chosen to show off this augmented reality girlfriend tech is [Hatsune Miku], a voice synthesizer personified as a doll-eyed anime  avatar. [Miku] is an immensely popular character in Japan, with thousands of people going to her concerts, so choosing her for this augmented reality girlfriend project was an obvious choice.

The build details for this hack are a little sparse, confounded by the horrible Google Translate results of the blog linked in the YouTube description. From what we can gather from the video and this twitter account, the build is based on an ASUS Xtion Kinect clone and a nice pair of video goggles.

We’re expecting the comments for this post to fill up with, ‘Japan is really weird’ comments, but we can see a few very, very cool applications of this tech. For instance, think how cool it would be to be guided around a science museum by [Einstein], or around Philadelphia by [Ben Franklin].

50 thoughts on “Going To The Park With Your Augmented Reality Girlfriend

  1. He we full Hacker. Can’t find an appropriate girlfriend? Make one!

    Very cool project. My first thought was also virtual docent tours, but I could see it in more formal educational environments too. (I’d trade my crappy class-assigned lab partner for Darwin, Hooke, or Galileo in an instant.)

    1. Yep, he slaps her on the head and plays with her tie but doesn’t cop a feel, weird! This augmented reality would be pretty cool if you hooked it up to the internet and call of duty though.

      1. @justice099
        A friendly reminder that race is a social construct! There are indeed genetic differences between groups of people but these differences do not line up with popular conception of race. In fact, the concept of which groups are in which races changes over time. For example, Irish people didn’t used to be counted as “real whites.”

        tl;dr: arguing over which groups are really races is really dumb.

      2. @justice099

        As a Japanese American I am extremely offended, I can’t believe stuff like this gets posted on this website. Was a big HaD fan but this is getting rediculous.

  2. That guy is awesome, he actually programed it so when you hit her on her head she flinches, just like a real woman. How long until someone combines this with the video game Rapelay?

  3. I guess this could be helpful for socially isolated people (stressed/traumatised) or for counselling of patients/prisoners who could hurt a real person. Or more specifically having a virtual presence of loved ones who aren’t anymore.

  4. This could be really awesome hack. It can be used for healing of traumatised/stressed patients or prisoners which can hurt themselves or their counsellor. And most interestingly it can be used making a virtual presence of loved ones which aren’t available or not anymore.

    1. or it could be combined with internet and linked to a canmera that looks at the counselor, and wallah! You’ve got the nearest thing to Star Wars-style hologram communication. The counselor could communicate with the patient but be totally safe. Or this tech could be a cool way of talking to far awat friends. Forget video chat, you could see your friend right in your world aqnd interact with her.

    1. Yeah – from the article title, I was expecting something like “My girlfriend made these really cool Augmented Reality Glasses (or worked on Google Glass), and we went to the park where she told me the names of all the people and plants we saw.”

  5. While this application is unsettelingly strange the concept at heart is interesting and shows a lot of promise for future gaming in the real world just so long as reality is discernible from the augmented reality counterparts. The simulation is a bit buggy in the video, we in the industry/hobby of CG have taken great pains to keep characters from sliding and popping in 3D and there are piles of noticeable artifacts from this process but they seem to be more a matter of hardware than coding or animation (with some exception) then there is the matter of range and sensitivity that come into play with reality clipping (such as the scene where the tree occludes the AR character or the demonstrators arm) one would almost need two maybe three depth cameras synced to the rgb capture. I think it’ll be a few years before we can start going to our local park to quest WOW or have a virtual Easter egg hunt in our back yard jungle (assuming you live on a dirt lot like I do and the jungle in question would be AR CG)

    Snow Crash anyone?

  6. ok…..so Hack-a-day missed out the single most important discovery in the history of physics at cern and yet reports on some socially inept, anime fans over engineered wet dream!!!!???

  7. Ignoring the actual content here, this is awesome from a hacker perspective and has fantastic implications. Telepresence, etc, there’s enormous potential.

    The actual implementation?

    Obligatory: Japan, WTF?

    Also: Seriously, dude. You were sure to code her responding to being groped and hit in the head. This is why you have to make your own girlfriend.

  8. ok,

    my japanese isnt 100% but from looking at the video, and applied some logic

    he used several off the shelf components, pc, kinect like hw, some oss motion and dimension tracker(ptam) and a lot of coding to make this happened.

    all done by a single guy

    imagine what kind of potential it could reach when you have such a system developed by well finance and research company. it would well exceed google’s project glass on so many different level.

    imagine what kind of application you could use it for,
    education, simulation, medicine, engineering, navigation, natural interface, entertainment, human interaction.

    i assumed the use of ir depth sensor (kinect), is to detect the tree and his hand that are “infront” of the virtual model. since ptam can only tell the dimention space in reference to where the viewer is to.

    if you combine it with better depth sensor, gyroscope, accelerometer, barometer, compass, tiny computer, battery, gps, microphone, speaker, etc. you would build killer application.

    if it can be made it into one small package like google glass at a affordable price, it would change how we would interact the world, the concept of current computing interface would just feel old and inadequate.

    our imagination, concept and expression would no longer confined to a 2d plane (or fake 3d on a 2d screen), but full 3d space. you can bring up a 3d model infront of you, analyze data in a way that you cannot or very hard to visualize. like looking at protein structure, virus, black hole, planet orbit, etc.

    this can all actually be achieved by current technology, all you need to do is mass produce to make it cheaper and have everyone using it. whats left are open standards such as interface, api, models, protocols, etc. something like iso. if not, we would be stuck with millions of different system which cant be interoperated. imagine how suck it would be that you want to load a data set for your cnc model, but realized that it is locked behind some drm, or vendor specific hw/sw (till you switch or pay).

    i see this is one small step in really awesome direction, and will keep a close watch in the ar/vr development.

    1. Augmented reality is a very exciting field. Especially in getting people to play again in a physical world eg outside. Imagine what this tech could bring to virtual hunting, paint-balling and air-soft games etc… Education eg Augmented reality zoos with extinct animals in augmented environments. I hope a big development house picks this up.

  9. Or, how cool it would be to go on a date with your real girlfriend while she’s out of the state getting her masters degree at Cornell.

    * thinks about that statement for a while *

    I mean… with you each having a set of the hardware of course and the date being coordinated in real time and over the internet and all that. Not like, going on a date with a virtual version of her, that’s just weird.

  10. The possible applications of this technology are amazing, many of which the comments have already pointed out. The Miku girlfriend thing is probably just a way to help catch the attention of an internet audience and Japan’s popular culture.

    But that aside, this is just another sign that 2013-2015 will be the dawning years of augmented reality, and though we still have obviously large leaps and bounds to make.

  11. It might help people to know that Miku is a standard reference character used by those working in AR/VR and 3D display technology in Japan. And with good reason. Aside from her immense popularity, her name and image are freely licensed for noncommercial use; she has her own open source 3D rendering software MikuMikuDance which has become the most widely used 3D animation program in Japan (search for [MMD] on youtube), and there are lots of freely available 3D models of Miku.

Leave a Reply to mattCancel reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.