Building Cameras For The Immersive Future

February 28, 2020 by Lewin Day 6 Comments

Thus far, the vast majority of human photographic output has been two-dimensional. 3D displays have come and gone in various forms over the years, but as technology progresses, we’re beginning to see more and more immersive display technologies. Of course, to use these displays requires content, and capturing that content in three dimensions requires special tools and techniques. Kim Pimmel came down to Hackaday Superconference to give us a talk on the current state of the art in advanced AR and VR camera technologies.

[Kim]’s interest in light painting techniques explored volumetric as well as 2D concepts.

Kim has plenty of experience with advanced displays, with an impressive resume in the field. Having worked on Microsoft’s Holo Lens, he now leads Adobe’s Aero project, an AR app aimed at creatives. Kim’s journey began at a young age, first experimenting with his family’s Yashica 35mm camera, where he discovered a love for capturing images. Over the years, he experimented with a wide variety of gear, receiving a Canon DSLR from his wife as a gift, and later tinkering with the Stereorealist 35mm 3D camera. The latter led to Kim’s growing obsession with three-dimensional capture techniques.

Through his work in the field of AR and VR displays, Kim became familiar with the combination of the Ricoh Theta S 360 degree camera and the Oculus Rift headset. This allowed users to essentially sit inside a photo sphere, and see the image around them in three dimensions. While this was compelling, [Kim] noted that a lot of 360 degree content has issues with framing. There’s no way to guide the observer towards the part of the image you want them to see.

Continue reading “Building Cameras For The Immersive Future” →

Tricking The Brain Into Seeing Boosted Contrast In Stereo Imagery

February 1, 2020 by Donald Papp 20 Comments

Last year a team of researchers published a paper detailing a method of boosting visual contrast and image quality in stereoscopic displays. The method is called Dichoptic Contrast Enhancement (DiCE) and works by showing each eye a slightly different version of an image, tricking the brain into fusing the two views together in a way that boosts perceived image quality. This only works on stereoscopic displays like VR headsets, but it’s computationally simple and easily implemented. This trick could be used to offset some of the limitations of displays used in headsets, for example making them appear capable of deeper contrast levels than they can physically deliver. This is good, because higher contrasts are generally perceived as being more realistic and three-dimensional; important factors in VR headsets and other stereoscopic displays.

Stereoscopic vision works by having the brain fuse together what both eyes see, and this process is called binocular fusion. The small differences between what each eye sees mostly conveys a sense of depth to us, but DiCE uses some of the quirks of binocular fusion to trick the brain into perceiving enhanced contrast in the visuals. This perceived higher contrast in turn leads to a stronger sense of depth and overall image quality.

Example of DiCE-processed images, showing each eye a different dynamic contrast range. The result is greater perceived contrast and image quality when the brain fuses the two together.

To pull off this trick, DiCE displays a different contrast level to both eyes in a way designed to encourage the brain to fuse them together in a positive way. In short, using a separate and different dynamic contrast range for each eye yields an overall greater perceived contrast range in the fused image. That’s simple in theory, but in practice there were a number of problems to solve. Chief among them was the fact that if the difference between what each eyes sees is too great, the result is discomfort due to binocular rivalry. The hard scientific work behind DiCE came from experimentally determining sweet spots, and pre-computing filters independent of viewer and content so that it could be applied in real-time for a consistent result.

Things like this are reminders that we experience the world only through the filter of our senses, and our perception of reality has quirks that can be demonstrated by things like this project and other “sensory fusion” edge cases like the Thermal Grill Illusion, which we saw used as the basis for a replica of the Pain Box from Dune.

A short video overview of the method is embedded below, and a PDF of the publication can be downloaded for further reading. Want a more hands-on approach? The team even made a DiCE plugin (freely) available from the Unity asset store.

Continue reading “Tricking The Brain Into Seeing Boosted Contrast In Stereo Imagery” →

A Pair Of CRTs Drive This Virtual Reality Headset

January 23, 2020 by Dan Maloney 12 Comments

With the benefit of decades of advances in miniaturization, looking back at the devices of yore can be entertaining. Take camcorders; did we really walk around with these massive devices resting on our shoulders just to record the family trip to Disneyworld? We did, but even if those days are long gone, the hardware remains for the picking in closets and at thrift stores.

Those camcorders can be turned into cool things such as this CRT-based virtual reality headset. [Andy West] removed the viewfinders from a pair of defunct Panasonic camcorders from slightly after the “Reggievision” era, leaving their housings and optics as intact as possible. He reverse-engineered the connections and hooked up the composite video inputs to HDMI-to-composite converters, which connect to the dual HDMI ports on a Raspberry Pi 4. An LM303DLHC accelerometer provides head tracking, and everything is mounted to a bodged headset designed to use a phone for VR. The final build is surprisingly neat for the number of thick cables and large components used, and it bears a passing resemblance to one of those targeting helmets attack helicopter pilots use.

The software is an amalgam of whatever works – Three.js for browser-based 3D animation, some off-the-shelf drivers for the accelerometers, and Python and shell scripts to glue it all together. The video below shows the build and a demo; we don’t get the benefit of seeing what [Andy] is seeing in glorious monochrome SD, but he seems suitably impressed. As are we.

We’ve seen an uptick in projects using CRT viewfinders lately, including this tiny vector display. Time to scour those thrift stores before all the old camcorders are snapped up.

Continue reading “A Pair Of CRTs Drive This Virtual Reality Headset” →

Building A Limitless VR Desktop

December 21, 2019 by Gerrit Coetzee 44 Comments

[Gabor Horvath] thinks even two monitors is too little space to really lay out his windows properly. That’s why he’s building a VR Desktop straight out of our deepest cyberpunk fantasies.

The software runs on Windows and Android at the moment. The user can put up multiple windows in a sphere around them. As their head moves, the window directly in front grows in focus. Imagine how many stack overflow windows you could have open at the same time!

Another exciting possibility is that the digital work-spaces can be shared among multiple users. Pair programming isn’t so bad, and now the possibility of doing it effectively while remote seems a little more possible. Even pair CAD might be possible depending on how its done. Imagine sharing your personal CAD session on another user’s screen and seeing theirs beside yours, allowing for simultaneous design.

Overall it’s a very cool tech demo that could turn into something more. It makes us wonder how long it is before tech workers on their way to lunch are marked by a telltale red circle on their face.

Ask Hackaday: Is Anyone Sad Phone VR Is Dead?

November 13, 2019 by Donald Papp 92 Comments

It’s official: smartphone-based VR is dead. The two big players in this space were Samsung Gear VR (powered by Oculus, which is owned by Facebook) and Google Daydream. Both have called it quits, with Google omitting support from their newer phones and Oculus confirming that the Gear VR has reached the end of its road. Things aren’t entirely shut down quite yet, but when it does it will sure leave a lot of empty headsets laying around. These things exist in the millions, but did anyone really use phone-based VR? Are any of you sad to see it go?

Google Cardboard, lowering cost and barrier to entry about as low as it could go.

In case you’re unfamiliar with phone-based VR, this is how it works: the user drops their smartphone into a headset, puts it on their head, and optionally uses a wireless controller to interact with things. The smartphone takes care of tracking motion and displaying 3D content while the headset itself takes care of the optics and holds everything in front of the user’s eyeballs. On the low end was Google Cardboard and on the higher end was Daydream and Gear VR. It works, and is both cheap and portable, so what happened?

In short, phone-based VR had constraints that limited just how far it could go when it came to delivering a VR experience, and these constraints kept it from being viable in the long run. Here are some of the reasons smartphone-based VR hit the end of the road: Continue reading “Ask Hackaday: Is Anyone Sad Phone VR Is Dead?” →

VR On The 6502

June 23, 2019 by Lewin Day 6 Comments

The MOS Technology 6502 was one of the more popular processors of the 1980s. It ran the Commodore 64, the NES in a modified form, and a whole bunch of other hardware, too. By modern standards, it’s barely fit to run a calculator, but no matter – [Nick Bild] built a VR game that runs on the retro CPU anyway!

[Nick]’s project is built on his 6502 computer, the Vectron 64. Being a breadboard build, it’s easy to modify things and add additional hardware, and that’s precisely what he did. The VR system uses two 320 x 240 LCD screens, one for each eye. These are controlled over SPI, but the humble 6502 simply doesn’t have the speed to clock out enough bits fast enough for a video game. Instead, additional hardware is added to generate pulses to run the screens. There’s a bunch of other neat hacks as well that help make the game playable, like overclocking the CPU to 1.75 MHz and drawing common elements to both screens at the same time.

To test out the VR system, [Nick] coded a basic Asteroids VR game. It’s not really practical to demonstrate the game without the hardware, but we’d love to try it out. There’s something compelling about a low-resolution VR game with 8-bit graphics, and we hope to see the concept further developed in future.

More grunt would make this project even more capable, and for that, a 6502 running at 20MHz could come in handy. Video after the break.

[Thanks to Fred Gimble for the tip!]

Continue reading “VR On The 6502” →

Everything You Probably Didn’t Know About FOV In HMDs

June 21, 2019 by Donald Papp 8 Comments

VR headsets have been seeing new life for a few years now, and when it comes to head-mounted displays, the field of view (FOV) is one of the specs everyone’s keen to discover. Valve Software have published a highly technical yet accessibly-presented document that explains why Field of View (FOV) is a complex thing when it pertains to head-mounted displays. FOV is relatively simple when it comes to things such as cameras, but it gets much more complicated and hard to define or measure easily when it comes to using lenses to put images right up next to eyeballs.

Simulation of how FOV can be affected by eye relief [Source: Valve Software]

The document goes into some useful detail about head-mounted displays in general, the design trade-offs, and naturally talks about the brand-new Valve Index VR headset in particular. The Index uses proprietary lenses combined with a slight outward cant to each eye’s display, and they explain precisely what benefits are gained from each design point. Eye relief (distance from eye to lens), lens shape and mounting (limiting how close the eye can physically get), and adjustability (because faces and eyes come in different configurations) all have a role to play. It’s a situation where every millimeter matters.

If there’s one main point Valve is trying to make with this document, it’s summed up as “it’s really hard to use a single number to effectively describe the field of view of an HMD.” They plan to publish additional information on the topics of modding as well as optics, so keep an eye out on their Valve Index Deep Dive publication list.

Valve’s VR efforts remain interesting from a hacking perspective, and as an organization they seem mindful of keen interest in being able to modify and extend their products. The Vive Tracker was self-contained and had an accessible hardware pinout for the express purpose of making hacking easier. We also took a look at Valve’s AR and VR prototypes, which give some insight into how and why they chose the directions they did.