An experimental project to mix reality and virtual reality by [Drew Gottlieb] uses the Microsoft Hololens and the HTC Vive to show two users successfully sharing a single workspace as well as controllers. While the VR user draws cubes in midair with a simple app, the Hololens user can see the same cubes being created and mapped to a real-world location, and the two headsets can even interact in the same shared space. You really need to check ou the video, below, to fully grasp how crazy-cool this is.
Two or more VR or AR users sharing the same virtual environment isn’t new, but anchoring that virtual environment into the real world in a way that two very different headsets share is interesting to see. [Drew] says that the real challenge wasn’t just getting the different hardware to talk to each other, it was how to give them both a shared understanding of a common space. [Drew] needed a way to make that work, and you can see the results in the video embedded below.
Projection mapping is pretty magical; done well, it’s absolutely miraculous when the facade of a building starts popping out abstract geometric objects, or crumbles in front of our very eyes. “Dynamic projection mapping onto deforming non-rigid surface” takes it to the next level. (Watch the video below.)
A group in the Ishikawa Watanabe lab at the University of Tokyo has a technique where they cover the target with a number of dots in an ink that is only visible in the infra-red. A high-speed (1000 FPS!) camera and some very fast image processing then work out not only how the surface is deforming, but which surface it is. This enables them to swap out pieces of paper and get the projections onto them in real time.
Some of the more interesting consumer hardware devices of recent years have been smart glasses. Devices like Google Glass or Snapchat Spectacles, eyewear incorporating a display and computing power to deliver information or provide augmented reality on an unobtrusive wearable platform.
Raspberry Pi Zero Smart Glass aims to provide an entry into this world, with image recognition and OCR text recognition in a pair of glasses courtesy of a Raspberry Pi Zero. Unusually though it does not take the display option of other devices of having a mirror or prism in the user’s field of view, instead it replaces the user’s entire field of view with a display and re-connects them to the world through the Raspberry Pi camera.
The display in question is an inexpensive set of “3D Virtual Stereo Digital Video glasses”, of the type that can be found fairly easily on your favourite auction site. They aren’t particularly high-resolution, but the Pi can easily drive them with its composite video output. The electronics and camera are mounted on a headband, in a custom 3D-printed enclosure. All files can be downloaded from the project page.
There is some Python software, but it’s fair to say that there is not a clear demo on the project page showing it working. However this is no reason to disregard this project, because even if its software has yet to achieve its full potential there is value elsewhere. The 3D-printed Raspberry Pi enclosure should be of use to many other similar wearable projects, and we’d almost say it’s worthy of a project all of its own.
Augmented reality is all the rage right now, and it’s all because of Pokemon. Of course, this means the entire idea of augmented reality is now wrapped up in taking pictures of Pidgeys in their unnatural setting. There are more useful applications of augmented reality, as [vijayvictory]’s Hackaday Prize entry shows us. He’s built an augmented reality helmet for firefighters that will detect temperature, gasses, smoke and the user’s own vital signs, displaying the readings on a heads up display.
The core of the build is a Particle Photon, a WiFi-enabled microcontroller that also gives this helmet the ability to relay data back to a base station, ostensibly one that’s not on fire. To this, [vijayvictory] has added an accelerometer, gas sensor, and a beautiful OLED display mounted just behind a prism. This display overlays the relevant data to the firefighter without obstructing their field of vision.
Right now, this system is fairly basic, but [vijayvictory] has a few more tricks up his sleeve. By expanding this system to include a FLIR thermal imaging sensor, this augmented reality helmet will have the ability to see through smoke. By integrating this system into an existing network and adding a few cool WiFi tricks, this system will be able to located a downed firefighter using signal trilateralization. It’s a very cool device, and one that should be very useful, making it a great entry for The Hackaday Prize.
Just in case anyone secretly had the idea that Valve Software’s VR and other hardware somehow sprang fully-formed from a lab, here are some great photos and video of early prototypes, and interviews with the people who made them. Some of the hardware is quite raw-looking, some of it is recognizable, and some are from directions that were explored but went nowhere, but it’s all fascinating.
The accompanying video (embedded below) has some great background and stories about the research process, which began with a mandate to explore the concepts of AR and VR and determine what could be done and what was holding things back.
One good peek into this process is the piece of hardware shown to the left. You look into the lens end like a little telescope. It has a projector that beams an image directly into your eye, and it has camera-based tracking that updates that image extremely quickly.
The result is a device that lets you look through a little window into a completely different world. In the video (2:16) one of the developers says “It really taught us just how important tracking was. No matter [how you moved] it was essentially perfect. It was really the first glimpse we had into what could be achieved if you had very low persistence displays, and very good tracking.” That set the direction for the research that followed.
Troy New York’s Tech Valley Center of Gravity is following up their January IoT Hackathon with another installment. The April 16-17 event promises to be a doozy, and anyone close to the area with even a passing interest in gaming and AR/VR should really make an effort to be there.
Not content to just be a caffeine-fueled creative burst, TVCoG is raising the bar in a couple ways. First, they’re teaming up with some corporate sponsors with a strong presence in the VR and AR fields. Daydream.io, a new company based in the same building as the CoG, is contributing a bunch of its Daydream.VR smartphone headsets to hackathon attendees, as well as mentors to get your project up and running. Other sponsors include 1st Playable Productions and Vicarious Visions, game studios both located in the Troy area. And to draw in the hardcore game programmers, a concurrent Ludum Dare game jam will be run by the Tech Valley Game Space, with interaction and collaboration between the AR/VR hackers and the programmers encouraged. Teams will compete for $1000 in prizes and other giveaways.
This sounds like it’s going to be an amazing chance to hack, to collaborate, and to make connections in the growing AR/VR field. And did we mention the food? There was a ton of it last time, so much they were begging us to take it home on Sunday night. Go, hack, create, mingle, and eat. TVCoG knows how to hackathon, and you won’t be disappointed.
Thanks to [Duncan Crary] for the heads up on this.
Think of Virtual Reality and it’s mostly fun and games that come to mind. But there’s a lot of useful, real world applications that will soon open up exciting possibilities in areas such as medicine, for example. [Victor] from the Shackspace hacker space in Stuttgart built an Augmented Reality Ultrasound scanning application to demonstrate such possibilities.
But first off, we cannot get over how it’s possible to go dumpster diving and return with a functional ultrasound machine! That’s what member [Alf] turned up with one day. After some initial excitement at its novelty, it was relegated to a corner gathering dust. When [Victor] spotted it, he asked to borrow it for a project. Shackspace were happy to donate it to him and free up some space. Some time later, [Victor] showed off what he did with the ultrasound machine.
As soon as the ultrasound scanner registers with the VR app, possibly using the image taped to the scan sensor, the scanner data is projected virtually under the echo sensor. There isn’t much detail of how he did it, but it was done using Vuforia SDK which helps build applications for mobile devices and digital eye wear in conjunction with the Unity 5 cross-platform game engine. Check out the video to see it in action.