Sharing Virtual and Holographic Realities via Vive and Hololens

An experimental project to mix reality and virtual reality by [Drew Gottlieb] uses the Microsoft Hololens and the HTC Vive to show two users successfully sharing a single workspace as well as controllers. While the VR user draws cubes in midair with a simple app, the Hololens user can see the same cubes being created and mapped to a real-world location, and the two headsets can even interact in the same shared space. You really need to check ou the video, below, to fully grasp how crazy-cool this is.

Two or more VR or AR users sharing the same virtual environment isn’t new, but anchoring that virtual environment into the real world in a way that two very different headsets share is interesting to see. [Drew] says that the real challenge wasn’t just getting the different hardware to talk to each other, it was how to give them both a shared understanding of a common space. [Drew] needed a way to make that work, and you can see the results in the video embedded below.

With only a few days to devote to a proof of concept, [Drew] went with a quick and dirty solution. To align the virtual and real room spaces, the Hololens user begins by picking up a Vive controller and is prompted to manually align it with a floating virtual controller. Once the user has manually aligned these virtual and real world points, the software uses that intersection as an anchor to lock the real and virtual objects into the same understanding of space. This worked even better than expected, but there is some slight error because the manual alignment is never perfect. Being one degree off doesn’t make much noticeable difference near the alignment point but as you move further away the misalignment increases. [Drew] suggests that prompting the alignment of three points instead of just one would be a step to even better results.

The shared environment consists of a simple block-building application and is manipulated with the Vive controllers, but is independent of the actual headsets. As a result, you don’t need to actually use the VR headset. You can use the Hololens with the VR controllers and create some cubes in the middle of the room instead of in an empty virtual space. In addition, someone else can put on either the VR headset (or another Hololens) and not only see, but interact in the same environment.

If you have access to a Vive and a Hololens and would like to try it out for yourself, [Drew] has all the code available on GitHub.

Mixed Reality as a field is seeing a resurgence and is filled with interesting developments. We recently featured an article on how things got to where they are in both AR and VR. Interested in making your own innovations in this area? The upcoming tracking hardware for the Vive is explicitly made to make development of new ideas easier.

13 thoughts on “Sharing Virtual and Holographic Realities via Vive and Hololens

  1. VR is becoming more and more interesting, We may be using this in future to show clients designs or for say a few architects to build their buildings in a VR environment with the ability to share idea’s virtually.

      1. I was the same and I did have a lot of Lego. Looking at Lego prices now it really is shocking some kits/boxes cost around $500. They are an a amazing toy prices of plastics have dropped 10 fold since I was a kid, There is a lot more automation in factories yet Lego still seems to cost a hell of a lot more now than it did then.

        1. With the notable exception of Dimensions, the Lego games have never been known for DLC.

          A VR Lego game would likely be an expansion of Lego Digital Designer, which is currently free anyway. It makes money in its own way because it’s a friendly Lego CAD tool that lets you order custom sets. I’d expect a similar approach.

    1. What’s really interesting and encouraging to me is that current VR and AR exists in a way that allows a developer with off the shelf hardware and downloadable software to throw together a proof of concept like this to test an idea, and to do so in a matter of days (including hacking together a draw-some-cubes application to go with it!)

    1. The reason it’s called hololens is because the glasses use a holographic optical system to project the image in the eye.
      So although the objects aren’t holographic the lens system showing it to you is.

  2. Seeing this perpetuates the original ‘wide field of view’ of the hololens, which isn’t a real thing, I assume this whole thing is MS financed then.

    Good for them. Not honest of course, but it’s MS..

    1. I created this project. This thing was my own idea and I was not paid by Microsoft; I did it in my university’s lab. I also can’t control the field of view Windows renders the real-time capture with.

      I will say, if I stand several feed back while wearing the HoloLens for this project, I can generally see the entire scene within the device’s narrow field of view. And the edges of the screen aren’t really noticeable, so it feels quite real!

      1. But the video overlay clearly suggests it’s a ‘entire vision’ device. And that is very suspicious to me. (I assume that that was post rendered then?) Anyway it’s painting the same rosy picture MS consistently does and the consistency is too consistent for my taste.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s