virtual reality

134 Articles

See The Science Behind VR Display Design, And What Makes A Problem Important

August 3, 2020 by 28 Comments

VR headsets are more and more common, but they aren’t perfect devices. That meant [Douglas Lanman] had a choice of problems to address when he joined Facebook Reality Labs several years ago. Right from the start, he perceived an issue no one seemed to be working on: the fact that the closer an object in VR is to one’s face, the less “real” it seems. There are several reasons for this, but the general way it presents is that the closer a virtual object is to the viewer, the more blurred and out of focus it appears to be. [Douglas] talks all about it and related issues in a great presentation from earlier this year (YouTube video) at the Electronic Imaging Symposium that sums up the state of the art for VR display technology while giving a peek at the kind of hard scientific work that goes into identifying and solving new problems.

Early varifocal prototype

[Douglas] chose to address seemingly-minor aspects of how the human eye and brain perceive objects and infer depth, and did so for two reasons: one was that no good solutions existed for it, and the other was that it was important because these cues play a large role in close-range VR interactions. Things within touching or throwing distance are a sweet spot for interactive VR content, and the state of the art wasn’t really delivering what human eyes and brain were expecting to see. This led to years of work on designing and testing varifocal and multi-focal displays which, among other things, were capable of presenting images in a variety of realistic focal planes instead of a single flat one. Not only that, but since the human eye expects things that are not in the correct focal plane to appear blurred (which is itself a depth cue), simulating that accurately was part of things, too.

The entire talk is packed full of interesting details and prototypes. If you have any interest in VR imaging and headset design and have a spare hour, watch it in the video embedded below.

Continue reading “See The Science Behind VR Display Design, And What Makes A Problem Important” →

Force Feedback Mouse Really Shakes Things Up

May 17, 2020 by 21 Comments

This is a very exciting time for those who like to spend their downtime exploring virtual worlds. The graphics in some big-budget titles are easily approaching photorealism, and immersive multi-channel sound can really make you believe you’ve been transported to another place or time. With another generation or two of GPU development and VR hardware, the line between gaming and reality is bound to get awful blurry.

That said, we’re still a far way off from the holodeck aboard the Enterprise. A high-end PC and the latest in VR can fool your eyes and ears, but that still leaves your other senses out of the fun. That’s why [Jatin Patel] has developed this clever force-feedback mouse using an array of solenoids.

The idea is pretty simple: a Python program on the computer listens for mouse click events, and tells an attached Arduino to fire off the solenoids when the player pulls the virtual trigger. It’s naturally not a perfect system, as it would seem that clicking in the game’s menus would also start your “gun” firing. But as you can see in the video after the break, when it works, it works very well. The moving solenoids don’t just vibrate the mouse around, the metallic clacking actually accentuates the gun sound effects from the game.

With this kind of tactile feedback and an omnidirectional treadmill to keep us moving, we’d be pretty close to fooling our senses into thinking we’re actually somewhere else. Which frankly, sounds quite appealing right about now.

Continue reading “Force Feedback Mouse Really Shakes Things Up” →

Peek At The Off-Ear Speaker Prototypes For Valve’s VR

May 6, 2020 by 23 Comments

The Valve Index VR headset incorporates a number of innovations, one of which is the distinctive off-ear speakers instead of headphones or earbuds. [Emily Ridgway] of Valve shared the design and evolution of this unusual system in a deep dive into the elements of the Index headset. [Emily] explains exactly what they were trying to achieve, how they determined what was and wasn’t important to deliver good sound in a VR environment, and what they were able to accomplish.

First prototype, a proof-of-concept that validated the basic idea and benefits of off-ear audio delivery.

Early research showed that audio was extremely important to providing a person with a good sense of immersion in a VR environment, but delivering a VR-optimized audio experience involved quite a few interesting problems that were not solved with the usual solutions of headphones or earbuds. Headphones and earbuds are optimized to deliver music and entertainment sounds, and it turns out that these aren’t quite up to delivering on everything Valve determined was important in VR.

The human brain is extremely good at using subtle cues to determine whether sounds are “real” or not, and all kinds of details come into play. For example, one’s ear shape, head shape, and facial geometry all add a specific tonal signature to incoming sounds that the brain expects to encounter. It not only helps to localize sounds, but the brain uses their presence (or absence) in deciding how “real” sounds are. Using ear buds to deliver sound directly into ear canals bypasses much of this, and the brain more readily treats such sounds as “not real” or even seeming to come from within one’s head, even if the sound itself — such as footsteps behind one’s back — is physically simulated with a high degree of accuracy. This and other issues were the focus of multiple prototypes and plenty of testing. Interestingly, good audio for VR is not all about being as natural as possible. For example, low frequencies do not occur very often in nature, but good bass is critical to delivering a sense of scale and impact, and plucking emotional strings.

“Hummingbird” prototype using BMR drivers. Over twenty were made and lent to colleagues to test at home. No one wanted to give them back.

The first prototype demonstrated the value of testing a concept as early as possible, and it wasn’t anything fancy. Two small speakers mounted on a skateboard helmet validated the idea of off-ear audio delivery. It wasn’t perfect: the speakers were too heavy, too big, too sensitive to variation in placement, and had poor bass response. But the results were positive enough to warrant more work.

In the end, what ended up in the Index headset is a system that leans heavily on Balanced Mode Radiator (BMR) speaker design. Cambridge Audio has a short and sweet description of how BMR works; it can be thought of as a hybrid between a traditional pistonic speaker drivers and flat-panel speakers, and the final design was able to deliver on all the truly important parts of delivering immersive VR audio in a room-scale environment.

As anyone familiar with engineering and design knows, everything is a tradeoff, and that fact is probably most apparent in cutting-edge technologies. For example, when Valve did a deep dive into field of view (FOV) in head-mounted displays, we saw just how complex balancing different features and tradeoffs could be.

Evolution Of A Backpack VR System

March 3, 2020 by 8 Comments

Persistence is what a hacker needs to make it to their goal. That’s exactly what it took for [Erik] to make an untethered VR backpack system.

Starting way back in the Spring of 2019, [Erik] began working on an untethered VR system. Sure, the Oculus Quest was coming out, but it wouldn’t be compatible with the game library of PC based systems. [Erik] decided he wanted the best of both worlds, so he decided to build a backpack that carries a computer powerful enough to drive the Rift S.

The initial system was to use a cut-up backpack, an HP mini PC with an external Nvidia 1060 GPU, and a basic DC-DC converter. The result? Just about nothing worked. The HP’s boot process didn’t play well with an external GPU.

[Erik] went through several iterations of this project. He switched over to a standard PC motherboard and tried a few different DC-DC converters. He settled on a device from HDPLEX rated at 200 watts continuous. The converter plugs directly into a standard 24-pin ATX motherboard power connector and isn’t much larger than the connector itself.

The old backpack with its added padding and wood frame gave way to a Zotac VR go backpack. Only the straps and frame of the Zotac are used, with [Erik’s] custom parts mounted using plywood and 3D printed parts. The outer frame is aluminum, with acrylic panels.

Power comes from 7000 mAH LiFe batteries, with each pack providing an hour of runtime. The Backpack can hold two packs though, so wiring them up in parallel should double that runtime.

We have to say this is an extremely well-documented build. [Erik] explains how he chose each component and the advantages (and pitfalls) of the choices he made. An example would be the RAM he picked. He chose DDR4 with a higher spec than he needed, just so he could undervolt the parts for longer run-times.

Not everything in VR is fun and games though – you can ditch that monitor and go with a VR desktop.

Esper Makes Virtual Reality From Live Reality

March 2, 2020 by 3 Comments

There’s a scene in Bladerunner where Deckard puts a photograph in a magical machine that lets him zoom and enhance without limit, and even see around obstacles. In today’s climate, this is starting to seem more plausible, what with all the cameras everywhere. [Jasper van Loenen] explores this concept in Esper, a technological art installation he created in Seoul, Korea during an artist residency.

Esper is a two-part piece that turns virtual reality on its head by showing actual reality in VR. It covers two adjoining rooms, one to record reality, and the other for real-time virtual viewing on headsets. The first is outfitted with 60 ESP32 cameras on custom mounts, all pointing in different directions from various perches and ceiling drops. [Jasper] used an Android app based on openFrameworks to map the cameras’ locations in 3D space. The room next door is so empty, it’s even devoid of FOMO. You don’t want to miss this one, so check it out after the break.

Recreating sci-fi props is all fun and games until the dystopia arrives. Then again, the fact that we can all easily access 70,000 or so insecure surveillance cameras is a pretty good start.

Continue reading “Esper Makes Virtual Reality From Live Reality” →

Building Cameras For The Immersive Future

February 28, 2020 by 6 Comments

Thus far, the vast majority of human photographic output has been two-dimensional. 3D displays have come and gone in various forms over the years, but as technology progresses, we’re beginning to see more and more immersive display technologies. Of course, to use these displays requires content, and capturing that content in three dimensions requires special tools and techniques. Kim Pimmel came down to Hackaday Superconference to give us a talk on the current state of the art in advanced AR and VR camera technologies.

[Kim]’s interest in light painting techniques explored volumetric as well as 2D concepts.
Kim has plenty of experience with advanced displays, with an impressive resume in the field. Having worked on Microsoft’s Holo Lens, he now leads Adobe’s Aero project, an AR app aimed at creatives. Kim’s journey began at a young age, first experimenting with his family’s Yashica 35mm camera, where he discovered a love for capturing images. Over the years, he experimented with a wide variety of gear, receiving a Canon DSLR from his wife as a gift, and later tinkering with the Stereorealist 35mm 3D camera. The latter led to Kim’s growing obsession with three-dimensional capture techniques.

Through his work in the field of AR and VR displays, Kim became familiar with the combination of the Ricoh Theta S 360 degree camera and the Oculus Rift headset. This allowed users to essentially sit inside a photo sphere, and see the image around them in three dimensions. While this was compelling, [Kim] noted that a lot of 360 degree content has issues with framing. There’s no way to guide the observer towards the part of the image you want them to see.

Continue reading “Building Cameras For The Immersive Future” →