With lockdown regulations sweeping the globe, many have found themselves spending altogether too much time inside with not a lot to do. [Peter Hall] is one such individual, with a penchant for flying quadcopters. With the great outdoors all but denied, he instead endeavoured to find a way to make flying inside a more exciting experience. We’d say he’s succeeded.
The setup involves using a SteamVR virtual reality tracker to monitor the position of a quadcopter inside a room. This data is then passed back to the quadcopter at a high rate, giving the autopilot fast, accurate data upon which to execute manoeuvres. PyOpenVR is used to do the motion tracking, and in combination with MAVProxy, sends the information over MAVLink back to the copter’s ArduPilot.
The build relies on Google-Cardboard-style optics, which are typically designed to work with a smartphone as the display. Instead, an 800×480 display intended for use with the Raspberry Pi is installed, hooked up over HDMI. An MPU6050 IMU is then installed to monitor the headset’s movements, hooked up to an Arduino Micro that passes this information to the attached PC. The rest of the build simply consists of cable management and power supply to all the hardware. It’s important to get this right, so that one doesn’t get tangled up by the umbilical when playing.
While it won’t outperform a commercial unit, the device nevertheless offers stereoscopic VR at a low cost. For a very cheap and accessible VR experience that’s compatible with the PC, it’s hard to beat. Others have done similar work too. Video after the break.
The Valve Index VR headset incorporates a number of innovations, one of which is the distinctive off-ear speakers instead of headphones or earbuds. [Emily Ridgway] of Valve shared the design and evolution of this unusual system in a deep dive into the elements of the Index headset. [Emily] explains exactly what they were trying to achieve, how they determined what was and wasn’t important to deliver good sound in a VR environment, and what they were able to accomplish.
Early research showed that audio was extremely important to providing a person with a good sense of immersion in a VR environment, but delivering a VR-optimized audio experience involved quite a few interesting problems that were not solved with the usual solutions of headphones or earbuds. Headphones and earbuds are optimized to deliver music and entertainment sounds, and it turns out that these aren’t quite up to delivering on everything Valve determined was important in VR.
The human brain is extremely good at using subtle cues to determine whether sounds are “real” or not, and all kinds of details come into play. For example, one’s ear shape, head shape, and facial geometry all add a specific tonal signature to incoming sounds that the brain expects to encounter. It not only helps to localize sounds, but the brain uses their presence (or absence) in deciding how “real” sounds are. Using ear buds to deliver sound directly into ear canals bypasses much of this, and the brain more readily treats such sounds as “not real” or even seeming to come from within one’s head, even if the sound itself — such as footsteps behind one’s back — is physically simulated with a high degree of accuracy. This and other issues were the focus of multiple prototypes and plenty of testing. Interestingly, good audio for VR is not all about being as natural as possible. For example, low frequencies do not occur very often in nature, but good bass is critical to delivering a sense of scale and impact, and plucking emotional strings.
The first prototype demonstrated the value of testing a concept as early as possible, and it wasn’t anything fancy. Two small speakers mounted on a skateboard helmet validated the idea of off-ear audio delivery. It wasn’t perfect: the speakers were too heavy, too big, too sensitive to variation in placement, and had poor bass response. But the results were positive enough to warrant more work.
In the end, what ended up in the Index headset is a system that leans heavily on Balanced Mode Radiator (BMR) speaker design. Cambridge Audio has a short and sweet description of how BMR works; it can be thought of as a hybrid between a traditional pistonic speaker drivers and flat-panel speakers, and the final design was able to deliver on all the truly important parts of delivering immersive VR audio in a room-scale environment.
Starting way back in the Spring of 2019, [Erik] began working on an untethered VR system. Sure, the Oculus Quest was coming out, but it wouldn’t be compatible with the game library of PC based systems. [Erik] decided he wanted the best of both worlds, so he decided to build a backpack that carries a computer powerful enough to drive the Rift S.
The initial system was to use a cut-up backpack, an HP mini PC with an external Nvidia 1060 GPU, and a basic DC-DC converter. The result? Just about nothing worked. The HP’s boot process didn’t play well with an external GPU.
[Erik] went through several iterations of this project. He switched over to a standard PC motherboard and tried a few different DC-DC converters. He settled on a device from HDPLEX rated at 200 watts continuous. The converter plugs directly into a standard 24-pin ATX motherboard power connector and isn’t much larger than the connector itself.
The old backpack with its added padding and wood frame gave way to a Zotac VR go backpack. Only the straps and frame of the Zotac are used, with [Erik’s] custom parts mounted using plywood and 3D printed parts. The outer frame is aluminum, with acrylic panels.
Power comes from 7000 mAH LiFe batteries, with each pack providing an hour of runtime. The Backpack can hold two packs though, so wiring them up in parallel should double that runtime.
We have to say this is an extremely well-documented build. [Erik] explains how he chose each component and the advantages (and pitfalls) of the choices he made. An example would be the RAM he picked. He chose DDR4 with a higher spec than he needed, just so he could undervolt the parts for longer run-times.
There’s a scene in Bladerunner where Deckard puts a photograph in a magical machine that lets him zoom and enhance without limit, and even see around obstacles. In today’s climate, this is starting to seem more plausible, what with all the cameras everywhere. [Jasper van Loenen] explores this concept in Esper, a technological art installation he created in Seoul, Korea during an artist residency.
Esper is a two-part piece that turns virtual reality on its head by showing actual reality in VR. It covers two adjoining rooms, one to record reality, and the other for real-time virtual viewing on headsets. The first is outfitted with 60 ESP32 cameras on custom mounts, all pointing in different directions from various perches and ceiling drops. [Jasper] used an Android app based on openFrameworks to map the cameras’ locations in 3D space. The room next door is so empty, it’s even devoid of FOMO. You don’t want to miss this one, so check it out after the break.
Thus far, the vast majority of human photographic output has been two-dimensional. 3D displays have come and gone in various forms over the years, but as technology progresses, we’re beginning to see more and more immersive display technologies. Of course, to use these displays requires content, and capturing that content in three dimensions requires special tools and techniques. Kim Pimmel came down to Hackaday Superconference to give us a talk on the current state of the art in advanced AR and VR camera technologies.
Kim has plenty of experience with advanced displays, with an impressive resume in the field. Having worked on Microsoft’s Holo Lens, he now leads Adobe’s Aero project, an AR app aimed at creatives. Kim’s journey began at a young age, first experimenting with his family’s Yashica 35mm camera, where he discovered a love for capturing images. Over the years, he experimented with a wide variety of gear, receiving a Canon DSLR from his wife as a gift, and later tinkering with the Stereorealist 35mm 3D camera. The latter led to Kim’s growing obsession with three-dimensional capture techniques.
Through his work in the field of AR and VR displays, Kim became familiar with the combination of the Ricoh Theta S 360 degree camera and the Oculus Rift headset. This allowed users to essentially sit inside a photo sphere, and see the image around them in three dimensions. While this was compelling, [Kim] noted that a lot of 360 degree content has issues with framing. There’s no way to guide the observer towards the part of the image you want them to see.
Stereoscopic vision works by having the brain fuse together what both eyes see, and this process is called binocular fusion. The small differences between what each eye sees mostly conveys a sense of depth to us, but DiCE uses some of the quirks of binocular fusion to trick the brain into perceiving enhanced contrast in the visuals. This perceived higher contrast in turn leads to a stronger sense of depth and overall image quality.
To pull off this trick, DiCE displays a different contrast level to both eyes in a way designed to encourage the brain to fuse them together in a positive way. In short, using a separate and different dynamic contrast range for each eye yields an overall greater perceived contrast range in the fused image. That’s simple in theory, but in practice there were a number of problems to solve. Chief among them was the fact that if the difference between what each eyes sees is too great, the result is discomfort due to binocular rivalry. The hard scientific work behind DiCE came from experimentally determining sweet spots, and pre-computing filters independent of viewer and content so that it could be applied in real-time for a consistent result.
Things like this are reminders that we experience the world only through the filter of our senses, and our perception of reality has quirks that can be demonstrated by things like this project and other “sensory fusion” edge cases like the Thermal Grill Illusion, which we saw used as the basis for a replica of the Pain Box from Dune.