Simple Cubes Show Off AI-Driven Runtime Changes In VR

May 30, 2023 by Donald Papp 6 Comments

AR and VR developer [Skarredghost] got pretty excited about a virtual blue cube, and for a very good reason. It marked a successful prototype of an augmented reality experience in which the logic underlying the cube as a virtual object was changed by AI in response to verbal direction by the user. Saying “make it blue” did indeed turn the cube blue! (After a little thinking time, of course.)

It didn’t stop there, of course, and the blue cube proof-of-concept led to a number of simple demos. The first shows off a row of cubes changing color from red to green in response to musical volume, then a bundle of cubes change size in response to microphone volume, and cubes even start moving around in space.

The program accepts spoken input from the user, converts it to text, sends it to a natural language AI model, which then creates the necessary modifications and loads it into the environment to make runtime changes in Unity. The workflow is a bit cumbersome and highlights many of the challenges involved, but it works and that’s pretty nifty.

The GitHub repository is here and a good demonstration video is embedded just under the page break. There’s also a video with a much more in-depth discussion of what’s going on and a frank exploration of the technical challenges.

If you’re interested in this direction, it seems [Skarredghost] has rounded up the relevant details. And should you have a prototype idea that isn’t necessarily AR or VR but would benefit from AI-assisted speech recognition that can run locally? This project has what you need.

Continue reading “Simple Cubes Show Off AI-Driven Runtime Changes In VR” →

Jump Like Mario With This Weighted Wearable

May 11, 2023 by Bryan Cockfield 5 Comments

Virtual reality has come a long way in the past decade, with successful commercial offerings for gaming platforms still going strong as well as a number of semi-virtual, or augmented, reality tools that are proving their worth outside of a gaming environment as well. But with all this success they still haven’t quite figured out methods of locomotion that feel natural like walking or running. One research group is leaping to solve one of these issues with JumpMod: a wearable device that enhances the sensation of jumping.

The group, led by [Pedro Lopes] at the University of Chicago, uses a two-kilogram weight worn on the back to help provide the feeling of jumping or falling. By interfacing it with the virtual reality environment, the weight can quickly move up or down its rails when it detects that the wearer is about to commit to an action that it thinks it can enhance. Wearers report feeling like they are jumping much higher, or even smashing into the ground harder. The backpack offers a compact and affordable alternative to the bulky and expensive hardware traditionally used for this purpose.

With builds like these, we would hope the virtual reality worlds that are being created become even more immersive and believable. Of course that means a lot more work into making other methods of movement in the virtual space feel believable (like walking, to start with) but it’s an excellent piece of technology that shows some progress. Augmenting the virtual space doesn’t always need bulky hardware like this, though. Take a “look” at this device which can build a believable virtual reality space using nothing more than a webcam.

Continue reading “Jump Like Mario With This Weighted Wearable” →

Tactile Feedback In VR, No Cumbersome Gloves Or Motors Required

April 26, 2023 by Donald Papp 9 Comments

This clever research from the University of Chicago’s Human Computer Integration Lab demonstrates a fascinating way to let users “feel” objects in VR, without anything getting in the way of using one’s hands and fingers normally. Certainly, the picture here shows hands with a device attached to them, but look closely and you’ll see that it’s on the back of the hand only.

There’s hardware attached to the hands, yes, but only to the backs. Hands and fingers can be used entirely normally while receiving tactile feedback.

The unique device consists of a control box, wires, and some electrodes attached to different spots on the back of the hand and fingers. Carefully modulated electrical signals create tactile sensations on the front, despite originating from electrodes on the back. While this has clear applications for VR, the team thinks the concept could also have applications in rehabilitation, or prosthetics.

Continue reading “Tactile Feedback In VR, No Cumbersome Gloves Or Motors Required” →

Supercon 2022: Aedan Cullen Is Creating An AR System To Beat The Big Boys

April 10, 2023 by Maya Posch 10 Comments

There’s something very tantalizing about an augmented reality (AR) overlay that can provide information in daily life without having to glance at a smartphone display, even if it’s just for that sci-fi vibe. Creating a system that is both practical and useful is however far from easy, which is where Aedan Cullen‘s attempt at creating what he terms a ‘practical augmented reality device’.

In terms of requirements, this device would need to have a visual resolution comparable to that of a smartphone (50 pixels/degree) and with a comparable field of view (20 degrees diagonal). User input would need to be as versatile as a touchscreen, but ‘faster’, along with a battery life of at least 8 hours, and all of this in a package weighing less than 50 grams.

Continue reading “Supercon 2022: Aedan Cullen Is Creating An AR System To Beat The Big Boys” →

Blinks Are Useful In VR, But Triggering Blinks Is Tricky

April 9, 2023 by Donald Papp 8 Comments

In VR, a blink can be a window of opportunity to improve the user’s experience. We’ll explain how in a moment, but blinks are tough to capitalize on because they are unpredictable and don’t last very long. That’s why researchers spent time figuring out how to induce eye blinks on demand in VR (video) and the details are available in a full PDF report. Turns out there are some novel, VR-based ways to reliably induce blinks. If an application can induce them, it makes it easier to use them to fudge details in helpful ways.

It turns out that humans experience a form of change blindness during blinks, and this can be used to sneak small changes into a scene in useful ways. Two examples are hand redirection (HR), and redirected walking (RDW). Both are ways to subtly break the implicit one-to-one mapping of physical and virtual motions. Redirected walking can nudge a user to stay inside a physical boundary without realizing it, leading the user to feel the area is larger than it actually is. Hand redirection can be used to improve haptics and ergonomics. For example, VR experiences that use physical controls (like a steering wheel in a driving simulator, or maybe a starship simulator project like this one) rely on physical and virtual controls overlapping each other perfectly. Hand redirection can improve the process by covering up mismatches in a way that is imperceptible to the user.

There are several known ways to induce a blink reflex, but it turns out that one novel method is particularly suited to implementing in VR: triggering the menace reflex by simulating a fast-approaching object. In VR, a small shadow appears in the field of view and rapidly seems to approach one’s eyes. This very brief event is hardly noticeable, yet reliably triggers a blink. There are other approaches as well such as flashes, sudden noise, or simulating the gradual blurring of vision, but to be useful a method must be unobtrusive and reliable.

We’ve already seen saccadic movement of the eyes used to implement redirected walking, but it turns out that leveraging eye blinks allows for even larger adjustments and changes to go unnoticed by the user. Who knew blinks could be so useful to exploit?

Continue reading “Blinks Are Useful In VR, But Triggering Blinks Is Tricky” →

Mice Play In VR

March 14, 2023 by Bryan Cockfield 9 Comments

Virtual Reality always seemed like a technology just out of reach, much like nuclear fusion, the flying car, or Linux on the desktop. It seems to be gaining steam in the last five years or so, though, with successful video games from a number of companies as well as plenty of other virtual reality adjacent technology that seems to be picking up steam as well like augmented reality. Another sign that this technology might be here to stay is this virtual reality headset made for mice. Continue reading “Mice Play In VR” →

Immersive Virtual Reality From The Humble Webcam

March 12, 2023 by Joseph Long 12 Comments

[Russ Maschmeyer] and Spatial Commerce Projects developed WonkaVision to demonstrate how 3D eye tracking from a single webcam can support rendering a graphical virtual reality (VR) display with realistic depth and space. Spatial Commerce Projects is a Shopify lab working to provide concepts, prototypes, and tools to explore the crossroads of spatial computing and commerce.

The graphical output provides a real sense of depth and three-dimensional space using an optical illusion that reacts to the viewer’s eye position. The eye position is used to render view-dependent images. The computer screen is made to feel like a window into a realistic 3D virtual space where objects beyond the window appear to have depth and objects before the window appear to project out into the space in front of the screen. The resulting experience is like a 3D view into a virtual space. The downside is that the experience only works for one viewer.

Eye tracking is performed using Google’s MediaPipe Iris library, which relies on the fact that the iris diameter of the human eye is almost exactly 11.7 mm for most humans. Computer vision algorithms in the library use this geometrical fact to efficiently locate and track human irises with high accuracy.

Generation of view-dependent images based on tracking a viewer’s eye position was inspired by a classic hack from Johnny Lee to create a VR display using a Wiimote. Hopefully, these eye-tracking approaches will continue to evolve and provide improved motion-responsive views into immersive virtual spaces.