The Valve Index VR headset incorporates a number of innovations, one of which is the distinctive off-ear speakers instead of headphones or earbuds. [Emily Ridgway] of Valve shared the design and evolution of this unusual system in a deep dive into the elements of the Index headset. [Emily] explains exactly what they were trying to achieve, how they determined what was and wasn’t important to deliver good sound in a VR environment, and what they were able to accomplish.
Early research showed that audio was extremely important to providing a person with a good sense of immersion in a VR environment, but delivering a VR-optimized audio experience involved quite a few interesting problems that were not solved with the usual solutions of headphones or earbuds. Headphones and earbuds are optimized to deliver music and entertainment sounds, and it turns out that these aren’t quite up to delivering on everything Valve determined was important in VR.
The human brain is extremely good at using subtle cues to determine whether sounds are “real” or not, and all kinds of details come into play. For example, one’s ear shape, head shape, and facial geometry all add a specific tonal signature to incoming sounds that the brain expects to encounter. It not only helps to localize sounds, but the brain uses their presence (or absence) in deciding how “real” sounds are. Using ear buds to deliver sound directly into ear canals bypasses much of this, and the brain more readily treats such sounds as “not real” or even seeming to come from within one’s head, even if the sound itself — such as footsteps behind one’s back — is physically simulated with a high degree of accuracy. This and other issues were the focus of multiple prototypes and plenty of testing. Interestingly, good audio for VR is not all about being as natural as possible. For example, low frequencies do not occur very often in nature, but good bass is critical to delivering a sense of scale and impact, and plucking emotional strings.
The first prototype demonstrated the value of testing a concept as early as possible, and it wasn’t anything fancy. Two small speakers mounted on a skateboard helmet validated the idea of off-ear audio delivery. It wasn’t perfect: the speakers were too heavy, too big, too sensitive to variation in placement, and had poor bass response. But the results were positive enough to warrant more work.
In the end, what ended up in the Index headset is a system that leans heavily on Balanced Mode Radiator (BMR) speaker design. Cambridge Audio has a short and sweet description of how BMR works; it can be thought of as a hybrid between a traditional pistonic speaker drivers and flat-panel speakers, and the final design was able to deliver on all the truly important parts of delivering immersive VR audio in a room-scale environment.
As anyone familiar with engineering and design knows, everything is a tradeoff, and that fact is probably most apparent in cutting-edge technologies. For example, when Valve did a deep dive into field of view (FOV) in head-mounted displays, we saw just how complex balancing different features and tradeoffs could be.
What is a Balanced Mode Radiator? It looks like they are playing games with cone breakup and calling it a new technology? Is this an actual breakthrough or just a load of marketing bull****?
https://www.cambridgeaudio.com/sites/default/files/paragraphs/image/bmr-technology-small1.gif
Looks interesting to me
It truly looks like some audiophile woo marketing bullshit to me. Could be wrong. But that kind of thing is so common that I’ve found that you can pretty much declare every big audio “breakthrough” to be charlatanism and you’d be correct 99.5% of the time.
The little illustrations and animations are pretty nonsensical.
I’m not sure why the Cambridge audio marketing non-explanation of the tech is in the summary. They are not even mentioned in the Valve article. So I read the white paper from the original developer, Tectonic, who is. And it seems like actual engineering to me.
https://www.tectonicaudiolabs.com/resources/
Tectonic_BMRWhitePaper_Rev2.0_2019-1.pdf
If you view the BMR as a radially symmetric flat panel speaker, it seems consistent with the science as explained in the Tech Ingredients Youtube video.
https://www.youtube.com/watch?v=zdkyGDqU7xA
Thank you for the link. What I understood from it, is that BMR is basically a way to control cone breakup. Isn’t this exactly what every speaker manufacturer is trying to do? Certainly I have seen speakers with dotted cones and cones with rings on them before. Hard to see why this particular implementation needs to get a new name.
The idea is interesting, but I am skeptic on their asseveration that this setup “[…] allow the ears and head to imprint their own tonal coloration onto the sound”. For the ears, OK. For the head, let me highly doubt it will make any difference.
You can test it easily- send a signal to the left driver and plug your left ear. Anything you hear with your right ear has been modified by the shape and materials (hair, flesh, cartilage) of your head.
If you hold a speaker 2 ft from your ear and move it towards your ear, the sound you hear will change as it interferes with the reflections form your ear and head.
I think they are overlooking one very important factor in gaming immersion: Isolation. Normal full-sized headphones might not deliver an ultra-realistic experience, but they do tend to block out much of the annoying background noise that these off-ear speakers definitely won’t.
In other words, it’s hard to feel like you’ve just landed on the surface of Mars when you can hear the laundry going, the neighbor’s crappy music playing through thin apartment walls, and cars driving by outside.
If you just crank the volume high enough, those pesky background noises will immediately become less intrusive, and over time eventually fade away completely.
Yeah, but then the ambient noises in the experience that are supposed to be subtle and soft will be blaring, drowning out external sounds yes but in a very unnatural way. And sounds meant to be loud won’t have any dynamic range. The speakers aren’t capable of infinite loudness, nor are your ears capable of sensing that. It wouldn’t help immersion to make all the sound in the experience literally deafening, that’s obviously a dead end. That would be incredibly jarring.
They block out more noise than you would think, they are extremely directional. Unlike the Oculus Quest. I’ve had the volume at comfortable levels and I’ve been unable to hear others talking too me unless there right next to me.
I was worried about noise bleed too but again the directional audio is impressive. With the head set off and right next to me I can barely hear the speakers.
The Oculus quest in comparison also has built in speakers but blocks out zero noise and the audio bleed is horrific. I’m reluctant for using it in my apartment for fear of disturbing my neighbors.
I’d second this, it’s surprising how little outside audio you get. By far and away the most immersion breaking thing I’ve experienced with mine is the headset hanging off the front of my face and the cable I occasionally get tangled in, or slight weirdness with the controllers. I’ve zero complaints about the audio.
That is not necessarily true, they could do active noise control/active noise reduction and at the very least attenuate any external sounds. But that may not be desirable feature for safety, but hopefully it is designed to deliberately fail for high dB sounds like fire alarms.
Well if you want isolation you could do a 3D scan of each ear/side of head and from that figure out an algorithm to modify the sound coming out of headphones to make the sound more “real”, of course you would need a 3D scan for everyones head.
Valve are not the first to try this…. not an innovation, or really even worth talking about to be honest.
You obviously never used it then lol
Back in the day, I had a set of these…
https://www.ebay.com/c/1823579540
Which intended use was to stick a walkman in the middle of the case there and have a portable boombox.
Anyway, latter half of the 90s, I was playing Heretic or Hexen, maybe a little Descent too. I had that thing on a high back chair, rigged as PC speakers/neck-rest. At even quite low volume levels it was really immersive, and sounds seemed to locate accurately in the first person environment. Seemed creepily accurate when you were feeling your way in a new level and in that nervous alert state anyway.
Not really had that sensation since with other speakers or headphones, so I know what they’re going for with this near-ear device.
I’d like to know if they get much low frequency sound out of these. I don’t see how it would be possible due to the short path from the front to the back side of the drivers. The low frequencies are 180 degrees out of phase at front and back so they should cancel leaving these things sounding tinny as an old transistor radio.
Cross-fading was invented for a reason.
Now that Windows 10 has Audio Processing Objects you can load up yourself, it’s simple to make your own 3D headphones. Just load up Peace and copy some of the left channel to the right and vice versa with a sub-millisecond delay and damp out the highest frequencies. You can project your headphones to sound like the sound is coming from invisible speakers in front of you.
One thing I want to mention but wasn’t able to fit into the post is how much an off-ear speaker works great with people (like myself) who need hearing aids.
Headphones and earbuds aren’t really compatible with needing hearing aids (and hearing loss isn’t really fixed by just cranking the volume.) An off-ear speaker lets me hear sounds via my hearing aids, and it’s about as good of an audio experience that I can get with broken ears in the first place.
30 some years ago I did something similar to what RW ver 0.0.1 described.
In my case it was using a pair of Radio Shack Minmus-7.
I mounted them, to a large chair, on stalks, a bit forward of my head (as when seated).
With a bit of tweaking the “toe in” angle, before gluing everything ridged, the sound became nicely immersive.
Even when keeping the volume below the level of bothering others in the house, it still had the effect of your brain soon ignoring much of the, quieter, worldly noises around you.
This is why I would be curious about a reeaaly low priced set of the “obsolete” sonos speakers, to use as passive units with my own amps. Needs to be some nice ones though. Can I dream a little?
Would be fun to do a personal speaker rig again!
I cannot even BEGIN to tell you how much more immersive using IEMs with my Index is compared to these mediocre off ear speakers… i plug a cyrus soundkey into the ‘frunk’ and plug CA12s into that.
While sticking the drivers off of your ears allows for not bothering to compute the fixed Head Related Transfer Function, it does nothing for the dynamic HRTF caused by moving sound sources (indeed it makes it harder, as you now need to perform a dynamic HRTF while also applying the inverse of the fixed hRTF which you need to isolate from the dynamic HRTF!). It’s that dynamic HRTF that is the real challenge to computer (or more accurately, to measure properly from the user’s head and pinnae geometry) that gives impressive results in audio simulation fidelity. The fixed HRTF is much simpler, to the point it’s been commonplace for motherboard audio for a decade or so (remember that it only works well with IEMs/’canalphones’, not on-ear or earbud headphones).
The other half of the puzzle is environmental audio simulation, which your chosen output arrangemnet will have some bearing on (e.g. environmental simulation for a multichannel emitter arrangement will be different from one performing full HRTF for a dual-driver setup, which will be different from one for off-ear stereo drivers).