Demonstration of the PMDG 737 being controlled by a blind user using Talking Flight Monitor

Flying Blind: Taking Flight Simulation To A New Level In Accessibility

November 2, 2021 by Ryan Flowers 14 Comments

Software developers [Andy Borka] and [Jason Fayre] have a love for aviation. They are also both totally blind. They’ve developed software called Talking Flight Monitor, and it has made flight simulation possible for anyone with impaired vision or blindness, as you can experience in the blurry video below the break. What draws them to aviation and flight simulators?

This fascination with flight is not limited to the sighted, and who wouldn’t want to experience what it’s like to be in cockpit of a modern airliner? I still recall the awe that I felt when at 9 years old, I glanced the flight deck of a McDonnell Douglas MD-80 as I boarded the aircraft. The array of lights, buttons, switches, and gauges dazzled me for years to come. I wanted to know how all of it worked. I wanted to be a pilot. A few years later I discovered Flight Simulator 4 on a 286, and I was hooked for life.

For the vision impaired this presents a problem. Flight simulators are by nature extremely visual, and they lack the text based interface that would allow a screen reader to help a visually impaired person make use of the simulator. Enter Talking Flight Monitor.

[Andy] and [Jason] have worked with PMDG Simulations to create text friendly interfaces for the 737 and 777 produced by PMDG. These ultra-realistic aircraft are available for the Prepar3D flight Simulator, and they result in a combination that blurs the line between Flight Simulator and Flight Training. By modifying these aircraft with accessible control panels, Talking Flight Monitor allows a completely blind flight simulator user to take off, navigate, and even land without ever seeing the screen.

Talking Flight Monitor makes flight possible using over 70 keyboard shortcuts. Both autopilot control and full manual control of the aircraft simulation are possible. Compatibility with standard simulation software is maintained in such a way that tutorials for programming flight computers not controlled by Talking Flight Monitor will still work. It even includes its own voice, so it does not require a screen reader to use.

Our hats are off to [Andy] and [Jason] for their hard work, diligence, and true application of the Hacker spirit. Thanks to [Mike Stone] for this most excellent tip.

[Note: The images in this post are produced by a community of blind flight simulator users who are not concerned with visual quality. They have been intentionally left blurry.]

Continue reading “Flying Blind: Taking Flight Simulation To A New Level In Accessibility” →

Omni-Wheeled Cane Steers The Visually-Impaired Away From Obstacles

October 24, 2021 by Kristina Panos 21 Comments

Sure, there are smart canes out there, commercial and otherwise. We’ve seen more than a few over the years. But a group of students at Stanford University have managed to bring something novel to the augmented cane.

Theirs features a motorized omni wheel that sweeps smoothly from left to right during normal cane operation, and when the cane senses an object that turns out to be an obstacle, the omni wheel goes into active mode, pulling the user out of the path of danger.

Tied for best part of this build is the fact that they made the project with open hardware and published all the gory details in a repo, so anyone can replicate it for about $400.

The cane uses a Raspi 4 with camera to detect objects, and a 2-D LIDAR to measure the distance to those objects. There’s a GPS and a 9-DOF IMU to find the position and orientation of the user. Their paper is open, too, and it comes with a BOM and build instructions. Be sure to check it out in action after the break.

There’s more than one way to guide people around with haptic feedback. Here’s the smartest pair of shoes we’ve seen lately.

Continue reading “Omni-Wheeled Cane Steers The Visually-Impaired Away From Obstacles” →

Seeing-Eye Shoes Pair Computer Vision With Haptic Feedback

May 12, 2021 by Kristina Panos 21 Comments

Over the years, we’ve seen plenty of projects that use ultrasonic or time-of-flight sensors as object detection methods for the visually impaired. Ultrasonic sensors detect objects like sonar — they send sound pulses and measure the time it takes for the signal to bounce off the object and come back. Time-of-flight sensors do essentially the same thing, but with infrared light. In either case, the notifications often come as haptic feedback on the wrist or head or whatever limb the ultrasonic module is attached to. We often wonder why there aren’t commercially-made shoes that do this, but it turns out there are, and they’re about to get even better.

The shoe’s view — safe walking space is delimited with color. Image via TU Graz

Today, Tec-Innovation makes shoes with ultrasonic sensors on the toes that can detect objects up to four meters away. The wearer is notified of obstacles through haptic feedback in the shoes as well as an audible phone notification via Bluetooth. The company teamed up with the Graz University of Technology in Austria to give the shoes robot vision that provides even better detail.

Ultrasonic is a great help, but it can’t detect the topography of the obstacle and tell a pothole from a rock from a wall. But if you have a camera on both feet, you can use the data to determine obstacle types and notify the user accordingly. These new models will still have the ultrasonic sensors to do the initial object detection, and use the cameras for analysis.

Whenever they do come out, the sensors will all be connected through the app, which paves the way for crowdsourced obstacle maps of various cities. The shoes will also be quite expensive. Can you do the same thing for less? Consider the gauntlet thrown!

We could all use some navigational help sometimes. Don’t want to look like a tourist? Get turn-by-turn directions in the corner of your eye.

Thanks for the tip, [Qes]!

Giving Sight To The Blind With A Wave Of The Hand

November 6, 2019 by Gerrit Coetzee 1 Comment

[Jakob Kilian] is working on a glove that he hopes will let the blind “see” their surroundings.

One of the most fascinating examples of the human brain’s plasticity is in its ability to map one sense to another. Some people, for example, report being able to see sound, giving them a supernatural ability to distinguish tones. This effect has also been observed in the visually impaired. There are experiments where grids of electrodes were placed on the tongue or mechanical actuators were placed on the lower back. The signals from a camera were fed into these grids and translated in to shocks or movement. The interesting effect is that the users quickly learned to distinguish objects from this low resolution input. As they continued to use these devices they actually reported seeing the objects as their visual centers took over interpreting this input.

Most of these projects are quite bulky and the usual mess you’d expect from a university laboratory. [Jakob]’s project appears to trend to a much more user-friendly product. A grid of haptics are placed on the back of the user’s hand along with a depth camera. Not only is it somewhat unobtrusive, the back of the hand is very sensitive to touch and the camera is in a prime position to be positioned for a look around the world.

[Jakob] admits that, as an interaction designer, his hardware hacking skills are still growing. To us, the polish and thought that went into this is already quite impressive, so it’s no wonder he’s one of the Hackaday Prize Finalists.

Hackaday Prize Entry: A Bracelet For The Blind

August 15, 2017 by Kristina Panos 9 Comments

The World Health Organization estimates that around 90% of the 285 million or so visually impaired people worldwide live in low-income situations with little or no access to assistive technology. For his Hackaday Prize entry, [Tiendo] has created a simple and easily reproducible way-finding device for people with reduced vision: a bracelet that detects nearby objects and alerts the wearer to them.

It does its job using an ultrasonic distance sensor and an Arduino Pro Mini. The bracelet has two feedback modes: audio and haptic. In audio mode, the bracelet will begin to beep when an object is within 2.5 meters. And it behaves the way you’d expect—get closer to the object and the beeping increases; back away and it decreases. Haptic mode involves two tiny vibrating disk motors attached to small PVC cuffs that fit on the thumb and pinky. These motors will buzz differently based on the person’s proximity to a given object. If an object is 1 to 2.5 meters away, the pinky motor will vibrate. Closer than that, and it switches over to the thumb motor.

To add to the thriftiness of this project, [Tiendo] re-used other objects where he could. The base of the bracelet is a cuff made from PVC. The nylon chin strap and plastic buckle from a broken bike helmet make it adjustable to fit any wrist. To keep the PVC cuff from chafing, he slipped small pieces from an old pair of socks on to the sides.

It’s easy to see why this project is a finalist in our Best Product contest. It’s a simple, low-cost assistive device made from readily available and recycled materials, and it can be built by anyone who knows a little bit about electronics. Add in the fact that it’s lightweight and frees up both hands, and you have a great product that can help a lot of people. Watch it beep and buzz after the break. Continue reading “Hackaday Prize Entry: A Bracelet For The Blind” →

The Music Of A Sunset

July 21, 2016 by James Hobson 12 Comments

What would you do if you suddenly went blind and could never again see the sun set? How would you again experience this often breathtaking phenomenon? One answer is music, orchestrated by the sun and the Weather Warlock.

Built by the musician [Quintron] (builder and inventor of insane electronic instruments), the Weather Warlock is an analog synthesizer controlled by — you guessed it — the weather. It translates temperature, moisture, wind and sunlight into tones and harmonics with an E major root chord. UV, light, moisture, and temperature sensors combined with an anemometer set up outside feed the weather data to a synthesizer that has [Quintron] dialing knobs and toggling switches. The Weather Warlock steams 24/7 to the website weatherfortheblind.org so that the visually impaired are able to tune in and experience the joy of sunrise and sunset through music. Continue reading “The Music Of A Sunset” →

See Who’s Calling With Caller Pi-D

October 11, 2015 by Kristina Panos 13 Comments

One of the hardest things in life is watching your parents grow old. As their senses fail, the simplest things become difficult or even impossible for them to do.

[kjepper]’s mom is slowly losing her sight. As a result, it’s hard for her to see things like the readout on the caller ID. Sure, there are plenty of units and phones she could get that have text-to-speech capabilities, but the audio on those things is usually pretty garbled. And yes, a smartphone can natively display a picture of the person calling, but [kjepper]’s mom isn’t technologically savvy and doesn’t need everything else that comes with a smartphone. What she needs is a really simple interface which makes it clear who’s calling.

Initially, [kjepper] tried to capture the caller ID data using only a USB modem. But for whatever reason, it didn’t work until he added an FSK–DTMF converter between the modem and the Pi. He wrote some Node.js in order to communicate with the Pi and send the information to the screen, which can display up to four calls at once. To make a mom-friendly interface, he stripped an old optical mouse down to the scroll wheel and encased it in wood. Mom can spin the wheel to wake the system up from standby, and click it to mark the calls as read. Now whenever Aunt Judy calls the landline, it’s immediately obvious that it’s her and not some telemarketer.

[via r/DIY]