Bone Vibration Brings Typing Into VR

March 16, 2021 by Lewin Day 17 Comments

Virtual reality is becoming more of a thing, now that we have high quality headsets and the computing power to generate attractive environments. Many VR systems use controllers held in mid air, or camera-based systems that track limbs and hands for interaction. However, productivity scenarios often require prolonged interactions over a long period of time, which typically necessitates working at surfaces that allow the body to rest intermittently. To help facilitate this, a group of researchers at ETH Zurich developed TapID, including a preprint paper (PDF) that will be presented at IEEE VR 2021 later this month.

TapID consists of a wristband that carries two motion sensors, with one worn on each wrist. This allows TapID to detect taps from each of the user’s fingers individually, thanks to a machine learning algorithm that analyses the unique vibrations through your skeletal system. This is demonstrated as being useful for VR environments, where the user can type into a virtual keyboard, or interact with virtual objects on a surface, using their fingers as they would in the real world. This is a sensor fusion with the features of modern VR headsets that include hand tracking. The TapID wristbands deliver granularity and detection of small motions that is not nearly as accurate through headset-mounted senors and camera-based detection.

Test hardware includes 4 accelerometers. Two on flexible PCBs are the sensing hardware used by the system, the other two on the rigid PCB are used as a baseline during testing but do not contribute to the tap detection.

We’re not entirely convinced of the utility of sitting down in a virtual environment to type at a fake keyboard when monitors and real keyboards are more tactile and cheaper. However, having a device that can accurately determine individual finger interactions is sure to have applications in VR. And whether or not the demonstrated use cases are viable, the technology does indeed work.

It’s exciting to see the wrist-band form factor. It brings to mind the possibility of improving tap interactions in smart watches for non-VR uses. We envision chorded keyboard type gestures that detect which fingers are tapping but don’t need positional accuracy.

Those experimenting in VR interfaces may find it useful to reverse engineer what’s already out on the market, as we’ve featured before. Or, you can simply build your own! Video after the break.

Continue reading “Bone Vibration Brings Typing Into VR” →

All The Good VR Ideas Were Dreamt Up In The 60s

January 4, 2021 by Donald Papp 39 Comments

Virtual reality has seen enormous progress in the past few years. Given its recent surges in development, it may come as a bit of a surprise to learn that the ideas underpinning what we now call VR were laid way back in the 60s. Not all of the imagined possibilities have come to pass, but we’ve learned plenty about what is (and isn’t) important for a compelling VR experience, and gained insights as to what might happen next.

If virtual reality’s best ideas came from the 60s, what were they, and how did they turn out?

Interaction and Simulation

First, I want to briefly cover two important precursors to what we think of as VR: interaction and simulation. Prior to the 1960s, state of the art examples for both were the Link Trainer and Sensorama.

The Link Trainer was an early kind of flight simulator, and its goal was to deliver realistic instrumentation and force feedback on aircraft flight controls. This allowed a student to safely gain an understanding of different flying conditions, despite not actually experiencing them. The Link Trainer did not simulate any other part of the flying experience, but its success showed how feedback and interactivity — even if artificial and limited in nature — could allow a person to gain a “feel” for forces that were not actually present.

Sensorama was a specialized pod that played short films in stereoscopic 3D while synchronized to fans, odor emitters, a motorized chair, and stereo sound. It was a serious effort at engaging a user’s senses in a way intended to simulate an environment. But being a pre-recorded experience, it was passive in nature, with no interactive elements.

Combining interaction with simulation effectively had to wait until the 60s, when the digital revolution and computers provided the right tools.

The Ultimate Display

In 1965 Ivan Sutherland, a computer scientist, authored an essay entitled The Ultimate Display (PDF) in which he laid out ideas far beyond what was possible with the technology of the time. One might expect The Ultimate Display to be a long document. It is not. It is barely two pages, and most of the first page is musings on burgeoning interactive computer input methods of the 60s.

The second part is where it gets interesting, as Sutherland shares the future he sees for computer-controlled output devices and describes an ideal “kinesthetic display” that served as many senses as possible. Sutherland saw the potential for computers to simulate ideas and output not just visual information, but to produce meaningful sound and touch output as well, all while accepting and incorporating a user’s input in a self-modifying feedback loop. This was forward-thinking stuff; recall that when this document was written, computers weren’t even generating meaningful sounds of any real complexity, let alone visual displays capable of arbitrary content. Continue reading “All The Good VR Ideas Were Dreamt Up In The 60s” →

Fans Add Reality To Virtual Driving

December 5, 2020 by Al Williams 16 Comments

A few decades ago you might have been satisfied with a crude wireframe flight simulator or driving a race car with the WASD keys. Today, gamers expect more realism, and [600,000 milliliters] is no different. At first, he upgraded his race car driving chair and put on VR goggles. But watching the world whiz by in VR is you can’t feel the wind on your face. Armed with a 3D printer, some software, and some repurposed PC fans, he can now feel the real wind in virtual reality. You can see the build in the video, below.

The electronics are relatively straightforward and there is already software available. The key, though, is the giant 3D printed ducts that direct the airflow. These are big prints, so probably not for some printers, but printers are getting bigger every day. The fan parts are from Thingiverse, but the enclosures are custom and you can download them from the blog post.

Continue reading “Fans Add Reality To Virtual Driving” →

Real Spectrum Analysis Goes Virtual

November 24, 2020 by Al Williams 4 Comments

One of the hard things about electronics is that you can’t really see the working parts without some sort of tool. If you work on car engines, fashion swords, or sculpt clay, you can see with your unaided eye what’s going on. Electronic components are just abstract pieces and the real action requires a meter or oscilloscope to understand. Maybe that’s what [José] was thinking of when he built a-radio. This “humble experiment” pipes a scan from a software-defined radio into VR goggles, which can be as simple as a smartphone and some cardboard glasses.

The resulting image shows you what the radio spectrum looks like. Granted, so will a spectrum analyzer, but perhaps the immersion will provide a different kind of insight into radio frequency analysis.

Continue reading “Real Spectrum Analysis Goes Virtual” →

Tired Of The Cat-and-Mouse

November 7, 2020 by Elliot Williams 49 Comments

Facebook just announced their plans for the Oculus Quest 2 VR headset. You probably won’t be surprised, but they want more of your user data, and more control over how you use the hardware. To use the device at all, you’ll need a verified Facebook account. Worse, they’re restricting access to the wide world of community-developed applications by requiring a developer account to be able to “sideload” non-Facebook software onto the device. Guess who decides who gets to be a developer. Hint: it’s not the people developing software.

Our article suggests that this will be the beginning of a race to jailbreak the headset on the community’s part, and to get ahead of the hackers on Facebook’s. Like every new release of iOS gets a jailbreak within a week or two, and then Apple patches it up as fast as they can, are we going to see a continual game of hacker cat-and-mouse with Facebook?

I don’t care. And that’s not because I don’t care about open hardware or indie VR developers. Quite the opposite! But like that romance you used to have with the girl who was absolutely no good for you, the toxic relationship with a company that will not let you run other people’s games on their hardware is one that you’re better off without. Sure, you can try to fix it, or hack it. You can tell yourself that maybe Facebook will come around if you just give them one more chance. It’s going to hurt at first.

But in the end, there is going to be this eternal fight between the user and the company that wants to use them, and that’s just sad. I used to look forward to the odd game of cat and mouse, but nowadays the cats are just too well bankrolled to make it a fair fight. If you’re buying a Quest 2 today with the intent of hacking it, I’d suggest you spend your time with someone else. You’re signing up for a string of heartbreaks. Nip it in the bud. You deserve better. There are too many fish in the sea, right?

What are our options?

As Facebook Tightens Their Grip On VR, Jailbreaking Looks More Likely

November 3, 2020 by Donald Papp 57 Comments

The Quest 2 wireless VR headset by Oculus was recently released, and improves on the one-and-a-half year old Quest mainly in terms of computing power and screen resolution. But Oculus is owned by Facebook, a fact that Facebook is increasingly keen on making very clear. The emerging scene is one that looks familiar: a successful hardware device, and a manufacturer that wants to keep users in a walled garden while fully controlling how the device can be used. Oculus started out very differently, but the writing has been on the wall for a while. Rooting and jailbreaking the Quest 2 seems inevitable, but what will happen then? Continue reading “As Facebook Tightens Their Grip On VR, Jailbreaking Looks More Likely” →

Light Fields: Missing Ingredient For Immersive 3D Video Gets Improved

October 20, 2020 by Donald Papp 13 Comments

46 time-synchronized action cameras make up the guts of the capture device.

3D video content has a significant limitation, one that is not trivial to solve. Video captured by a camera — even one with high resolution and a very wide field of view — still records a scene as a flat plane, from a fixed point of view. The limitation this brings will be familiar to anyone who has watched a 3D video (or “360 video”) in VR and moved their head the wrong way. In these videos one is free to look around, but may not change the position of their head in the process. Put another way, pivoting one’s head to look up, down, left, or right is fine. Moving one’s head higher, lower, closer, further, or to the side? None of that works. Natural movements like trying to peek over an object, or moving slightly to the side for a better view simply do not work.

Light field video changes that. It is captured using a device like the one in the image above, and Google has a resource page giving an excellent overview of what light field video is, what it can look like, and how they are doing it. That link covers recent improvements to their camera apparatus as well as to video encoding and rendering, but serves as a great show-and-tell of what light fields are and what they can do.

Continue reading “Light Fields: Missing Ingredient For Immersive 3D Video Gets Improved” →