Augmented reality (AR) in the classroom has garnered a bit of interest over the years, but given the increased need for remote and virtual learning these days, it might be worth taking a closer look at what AR can offer. Purdue University’s C Design Lab thinks they’ve found a solution in their Meta-AR platform. The program allows an instructor to monitor each student’s work in real-time without being in the same classroom as the student. Not only that, but the platform allows students to collaborate in real-time with each other giving each other tips and feedback while also being able to interact with each other’s work, no matter where they may be physically located.
The Valve Index VR headset incorporates a number of innovations, one of which is the distinctive off-ear speakers instead of headphones or earbuds. [Emily Ridgway] of Valve shared the design and evolution of this unusual system in a deep dive into the elements of the Index headset. [Emily] explains exactly what they were trying to achieve, how they determined what was and wasn’t important to deliver good sound in a VR environment, and what they were able to accomplish.
First prototype, a proof-of-concept that validated the basic idea and benefits of off-ear audio delivery.
Early research showed that audio was extremely important to providing a person with a good sense of immersion in a VR environment, but delivering a VR-optimized audio experience involved quite a few interesting problems that were not solved with the usual solutions of headphones or earbuds. Headphones and earbuds are optimized to deliver music and entertainment sounds, and it turns out that these aren’t quite up to delivering on everything Valve determined was important in VR.
The human brain is extremely good at using subtle cues to determine whether sounds are “real” or not, and all kinds of details come into play. For example, one’s ear shape, head shape, and facial geometry all add a specific tonal signature to incoming sounds that the brain expects to encounter. It not only helps to localize sounds, but the brain uses their presence (or absence) in deciding how “real” sounds are. Using ear buds to deliver sound directly into ear canals bypasses much of this, and the brain more readily treats such sounds as “not real” or even seeming to come from within one’s head, even if the sound itself — such as footsteps behind one’s back — is physically simulated with a high degree of accuracy. This and other issues were the focus of multiple prototypes and plenty of testing. Interestingly, good audio for VR is not all about being as natural as possible. For example, low frequencies do not occur very often in nature, but good bass is critical to delivering a sense of scale and impact, and plucking emotional strings.
“Hummingbird” prototype using BMR drivers. Over twenty were made and lent to colleagues to test at home. No one wanted to give them back.
The first prototype demonstrated the value of testing a concept as early as possible, and it wasn’t anything fancy. Two small speakers mounted on a skateboard helmet validated the idea of off-ear audio delivery. It wasn’t perfect: the speakers were too heavy, too big, too sensitive to variation in placement, and had poor bass response. But the results were positive enough to warrant more work.
In the end, what ended up in the Index headset is a system that leans heavily on Balanced Mode Radiator (BMR) speaker design. Cambridge Audio has a short and sweet description of how BMR works; it can be thought of as a hybrid between a traditional pistonic speaker drivers and flat-panel speakers, and the final design was able to deliver on all the truly important parts of delivering immersive VR audio in a room-scale environment.
As anyone familiar with engineering and design knows, everything is a tradeoff, and that fact is probably most apparent in cutting-edge technologies. For example, when Valve did a deep dive into field of view (FOV) in head-mounted displays, we saw just how complex balancing different features and tradeoffs could be.
Keeping track of position is crucial in a lot of situations. On Earth, it’s usually relatively straight-forward, with systems having been developed over the centuries that would allow one to get at least a rough fix on one’s position on this planet. But for a satellite out in space, however, it’s harder. How do they keep their communications dishes pointed towards Earth?
The stars are an obvious orientation point. The Attitude and Articulation Control Subsystem (AACS) on the Voyager 1 and 2 space probes has the non-enviable task of keeping the spacecraft’s communication dish aligned precisely with a communications dish back on Earth, which from deep space is an incomprehensibly tiny target.
Back on Earth, the star tracker concept has become quite popular among photographers who try to image the night skies. Even in your living room, VR systems also rely on knowing the position of the user’s body and any peripherals in space. In this article we’ll take a look at the history and current applications of this type of position tracking. Continue reading “Star Trackers: Telling Up From Down In Any Space”→
Persistence is what a hacker needs to make it to their goal. That’s exactly what it took for [Erik] to make an untethered VR backpack system.
Starting way back in the Spring of 2019, [Erik] began working on an untethered VR system. Sure, the Oculus Quest was coming out, but it wouldn’t be compatible with the game library of PC based systems. [Erik] decided he wanted the best of both worlds, so he decided to build a backpack that carries a computer powerful enough to drive the Rift S.
The initial system was to use a cut-up backpack, an HP mini PC with an external Nvidia 1060 GPU, and a basic DC-DC converter. The result? Just about nothing worked. The HP’s boot process didn’t play well with an external GPU.
[Erik] went through several iterations of this project. He switched over to a standard PC motherboard and tried a few different DC-DC converters. He settled on a device from HDPLEX rated at 200 watts continuous. The converter plugs directly into a standard 24-pin ATX motherboard power connector and isn’t much larger than the connector itself.
The old backpack with its added padding and wood frame gave way to a Zotac VR go backpack. Only the straps and frame of the Zotac are used, with [Erik’s] custom parts mounted using plywood and 3D printed parts. The outer frame is aluminum, with acrylic panels.
Power comes from 7000 mAH LiFe batteries, with each pack providing an hour of runtime. The Backpack can hold two packs though, so wiring them up in parallel should double that runtime.
We have to say this is an extremely well-documented build. [Erik] explains how he chose each component and the advantages (and pitfalls) of the choices he made. An example would be the RAM he picked. He chose DDR4 with a higher spec than he needed, just so he could undervolt the parts for longer run-times.
There’s a scene in Bladerunner where Deckard puts a photograph in a magical machine that lets him zoom and enhance without limit, and even see around obstacles. In today’s climate, this is starting to seem more plausible, what with all the cameras everywhere. [Jasper van Loenen] explores this concept in Esper, a technological art installation he created in Seoul, Korea during an artist residency.
Esper is a two-part piece that turns virtual reality on its head by showing actual reality in VR. It covers two adjoining rooms, one to record reality, and the other for real-time virtual viewing on headsets. The first is outfitted with 60 ESP32 cameras on custom mounts, all pointing in different directions from various perches and ceiling drops. [Jasper] used an Android app based on openFrameworks to map the cameras’ locations in 3D space. The room next door is so empty, it’s even devoid of FOMO. You don’t want to miss this one, so check it out after the break.
Thus far, the vast majority of human photographic output has been two-dimensional. 3D displays have come and gone in various forms over the years, but as technology progresses, we’re beginning to see more and more immersive display technologies. Of course, to use these displays requires content, and capturing that content in three dimensions requires special tools and techniques. Kim Pimmel came down to Hackaday Superconference to give us a talk on the current state of the art in advanced AR and VR camera technologies.
[Kim]’s interest in light painting techniques explored volumetric as well as 2D concepts.Kim has plenty of experience with advanced displays, with an impressive resume in the field. Having worked on Microsoft’s Holo Lens, he now leads Adobe’s Aero project, an AR app aimed at creatives. Kim’s journey began at a young age, first experimenting with his family’s Yashica 35mm camera, where he discovered a love for capturing images. Over the years, he experimented with a wide variety of gear, receiving a Canon DSLR from his wife as a gift, and later tinkering with the Stereorealist 35mm 3D camera. The latter led to Kim’s growing obsession with three-dimensional capture techniques.
Through his work in the field of AR and VR displays, Kim became familiar with the combination of the Ricoh Theta S 360 degree camera and the Oculus Rift headset. This allowed users to essentially sit inside a photo sphere, and see the image around them in three dimensions. While this was compelling, [Kim] noted that a lot of 360 degree content has issues with framing. There’s no way to guide the observer towards the part of the image you want them to see.
Virtual reality holds the promise of an immersive experience that can satisfy our senses to a level comparable with… well, reality. The field has come a long way, but Sarah Vollmer makes a good point that many of the VR systems currently in use are bulky and difficult to transfer from person to person.
While headsets have become smaller and lighter and now feature improved motion tracking and resolution, their ability to affect the user’s other senses hasn’t seen nearly the same advancement. Haptic feedback systems need to catch up with headsets, and how to unobtrusively allow users feel simulated physical contact in VR is an area Sarah is researching as part of her PhD work. This is the topic of her 2019 Hackaday Superconference talk which you’ll find embedded below.
