Quest

6 Articles

VR Headset With HDMI Input Invites A New Kind Of Cyberdeck

September 1, 2024 by 30 Comments

Meta’s Quest VR headset recently got the ability to accept and display video over USB-C, and it’s started some gears turning in folks’ heads. [Ian Hamilton] put together a quick concept machine consisting of a Raspberry Pi 400 that uses a VR headset as its monitor, which sure seems like the bones of a new breed of cyberdeck.

With passthrough on, one still sees the outside world.

The computer-in-a-keyboard nature of the Pi 400 means that little more than a mouse and the VR headset are needed to get a functional computing environment. Well, that and some cables and adapters.

What’s compelling about this is that the VR headset is much more than just a glorified monitor. In the VR environment, the external video source (in this case, the Raspberry Pi) is displayed in a window just like any other application. Pass-through can also be turned on, so that the headset’s external cameras display one’s surroundings as background. This means there’s no loss of environmental awareness while using the rig.

[Note: the following has been updated for clarity and after some hands-on testing] Video over USB-C is technically DisplayPort altmode, and both the video source and the USB-C cable have to support it. In [Ian]’s case, the Raspberry Pi 400 outputs HDMI and he uses a Shadowcast 2 capture card to accept HDMI on one end and outputs video over USB-C on the other.

Here’s how it works: the Quest has a single USB-C port on the side, and an app (somewhat oddly named “Meta Quest HDMI link”) running on the headset takes care of accepting video over USB and displaying it in a window within the headset. The video signal expected is UVC (or USB Video Class), which is what most USB webcams and other video devices output. (There’s another way to do video over USB-C which is technically DisplayPort altmode, and both the video source and the USB-C cable have to support it. That is not what’s being used here; the Quest does not support this format. Neither is it accepting HDMI directly.) In [Ian]’s case, the Raspberry Pi 400 outputs HDMI and he uses a Shadowcast 2 capture card to accept HDMI on one end and output UVC video on the other, which is then fed into the Quest over a USB-C cable.

As a concept it’s an interesting one for sure. Perhaps we’ll see decks of this nature in our next cyberdeck contest?

Meta Doesn’t Allow Camera Access On VR Headsets, So Here’s A Workaround

August 18, 2024 by 13 Comments

The cameras at the front of Meta’s Quest VR headsets are off-limits to developers, but developer [Michael Gschwandtner] created a workaround (Linkedin post) and shared implementation details with a VR news site.

The view isn’t a pure camera feed (it includes virtual and UI elements) but it’s a clever workaround.

The demo shows object detection via MobileNet V2, which we’ve seen used for machine vision on embedded systems like the Raspberry Pi. In this case it is running locally on the VR headset, automatically identifying objects even though the app cannot directly access the front-facing cameras to see what’s in front of it.

The workaround is conceptually simple, and leverages the headset’s ability to cast its video feed over Wi-Fi to other devices. This feature is normally used for people to share and spectate VR gameplay.

First, [Gschwandtner]’s app sets up passthrough video, which means that the camera feed from the front of the headset is used as background in VR, creating a mixed-reality environment. Then the app essentially spawns itself a Chromium browser, and casts its video feed to itself. It is this video that is used to — in a roundabout way — access what the cameras see.

The resulting view isn’t really direct from the cameras, it’s akin to snapshotting a through-the-headset view which means it contains virtual elements like the UI. Still, with passthrough turned on it is a pretty clever workaround that is contained entirely on-device.

Meta is hesitant to give developers direct access to camera views on their VR headset, and while John Carmack (former Meta consulting CTO) thinks it’s worth opening up and can be done safely, it’s not there yet.

Quest 3 VR Headset Can Capture 3D Video (Some Tampering Required)

November 24, 2023 by 13 Comments

The Quest 3 VR headset is an impressive piece of hardware. It is also not open; not in the way most of us understand the word. One consequence of this is the inability in general for developers or users to directly access the feed of the two color cameras on the front of the headset. However, [Hugh Hou] shares a method of doing exactly this to capture 3D video on the Quest 3 headset for later playback on different devices.

The Quest 3 runs Android under the hood, and Developer Mode plus some ADB commands does the trick.

There are a few steps to the process and it involves enabling developer mode on the hardware then using ADB (Android Debug Bridge) commands to enable the necessary functionality, but it’s nothing the average curious hacker can’t handle. The directions are written out in the video’s description, along with a few handy links. (The video is embedded below just under the page break, but view it on YouTube to access the description and all the info in it.)

He also provides some excellent guidance on practical things like how to capture stable shots, editing the videos, and injecting the necessary metadata for optimal playback on different platforms, including hassle-free uploading to a service like YouTube. [Hugh] is no stranger to this kind of video and camera handling and really knows his stuff, and it’s great to see someone provide detailed instructions.

This kind of 3D video comes down to recording two different views, one for each eye. There’s another way to approach 3D video, however: light fields are also within reach of enterprising hackers, and while they need more hardware they yield far more compelling results.

Continue reading “Quest 3 VR Headset Can Capture 3D Video (Some Tampering Required)” →

Beautifully Rebuilding A VR Headset To Add AR Features

July 25, 2023 by 12 Comments

[PyottDesign] recently wrapped up a personal project to create himself a custom AR/VR headset that could function as an AR (augmented reality) platform, and make it easier to develop new applications in a headset that could do everything he needed. He succeeded wonderfully, and published a video showcase of the finished project.

Getting a headset with the features he wanted wasn’t possible by buying off the shelf, so he accomplished his goals with a skillful custom repackaging of a Quest 2 VR headset, integrating a Stereolabs Zed Mini stereo camera (aimed at mixed reality applications) and an Ultraleap IR 170 hand tracking module. These hardware modules have tons of software support and are not very big, but when sticking something onto a human face, every millimeter and gram counts.

Continue reading “Beautifully Rebuilding A VR Headset To Add AR Features” →

As Facebook Tightens Their Grip On VR, Jailbreaking Looks More Likely

November 3, 2020 by 57 Comments

The Quest 2 wireless VR headset by Oculus was recently released, and improves on the one-and-a-half year old Quest mainly in terms of computing power and screen resolution. But Oculus is owned by Facebook, a fact that Facebook is increasingly keen on making very clear. The emerging scene is one that looks familiar: a successful hardware device, and a manufacturer that wants to keep users in a walled garden while fully controlling how the device can be used. Oculus started out very differently, but the writing has been on the wall for a while. Rooting and jailbreaking the Quest 2 seems inevitable, but what will happen then? Continue reading “As Facebook Tightens Their Grip On VR, Jailbreaking Looks More Likely” →

Ask Hackaday: Is Anyone Sad Phone VR Is Dead?

November 13, 2019 by 92 Comments

It’s official: smartphone-based VR is dead. The two big players in this space were Samsung Gear VR (powered by Oculus, which is owned by Facebook) and Google Daydream. Both have called it quits, with Google omitting support from their newer phones and Oculus confirming that the Gear VR has reached the end of its road. Things aren’t entirely shut down quite yet, but when it does it will sure leave a lot of empty headsets laying around. These things exist in the millions, but did anyone really use phone-based VR? Are any of you sad to see it go?

Google Cardboard, lowering cost and barrier to entry about as low as it could go.

In case you’re unfamiliar with phone-based VR, this is how it works: the user drops their smartphone into a headset, puts it on their head, and optionally uses a wireless controller to interact with things. The smartphone takes care of tracking motion and displaying 3D content while the headset itself takes care of the optics and holds everything in front of the user’s eyeballs. On the low end was Google Cardboard and on the higher end was Daydream and Gear VR. It works, and is both cheap and portable, so what happened?

In short, phone-based VR had constraints that limited just how far it could go when it came to delivering a VR experience, and these constraints kept it from being viable in the long run. Here are some of the reasons smartphone-based VR hit the end of the road: Continue reading “Ask Hackaday: Is Anyone Sad Phone VR Is Dead?” →