Adventure travel can be pretty grueling, what with the exotic locations and potential for disaster that the typical tourist destinations don’t offer. One might find oneself dangling over a cliff for that near-death-experience selfie or ziplining through a rainforest canopy. All this is significantly complicated by being blind, of course, so a tool like this Raspberry Pi low-vision system would be a welcome addition to the nearly-blind adventurer’s well-worn rucksack.
[Dan] has had vision problems since childhood, but one look at his YouTube channel shows that he doesn’t let that slow him down. When [Dan] met [Ben] in Scotland, [Ben] noticed that he was using his smartphone as a vision aid, looking at the display up close and zooming in to get as much detail as possible from his remaining vision. [Ben] thought he could help, so he whipped up a heads-up display from a Raspberry Pi and a Pi Camera. Mounted to a 3D-printed frame holding a 5″ HDMI display and worn from a GoPro head mount, the camera provides enough detail to help [Dan] navigate, as seen in the video below.
The rig is a bit unwieldy right now, but as proof of concept (and proof of friendship), it’s a solid start. We think a slimmer profile design might help, in which case [Ben] might want to look into this Google Glass-like display for a multimeter for inspiration on version 2.0.
It would be useful to have a map of his still functional retina areas, then a transform could be calculated to map the rectangular coordinates of a video feed to the nonlinear field of his vision, his brain would then do the rest of the remapping work. The idea being to get as much of the image pixels into function areas in a way that enhances the eye-brain’s ability to fill in gaps.
and better display – Jeri was working with LCOS projectors and back reflecting glasses, but she is in a failed VC fueled startup mode, so wont produce anything :(
I’d get rid of all of that design, an Android phone with HD and a VR headset that has been modified to have cameras for mix reality would be ideal, them both the real world and the virtual world’s accessibility would be maximised for him.
Mapping the remaining sensors in his retina should not be that hard just put randomly positioned R G and B dots on a dark field and read the electrical response of his visual cortex with scalp pickups. In fact if you put a neural network into the loop it can calculate the transform at the same time. Let me know if you need the details, but that should give you a clear idea of the direction to take things.
So with the iPhone and Android platform we tested on, the camera preview latency was much worse than the raspberry pi setup. Small differences here go a long way. Mapping vision sounds like a really interesting idea, I’ll keep it on the backburner, and would absolutely welcome the contribution if you’re interested!
You need to talk to somebody at a university or associated hackerspace, there is not a lot I can do from where I am, other that give you a valuable idea. :-)
I wonder if you can connect a MIPI camera directly to a MIPI screen and just bypass the CPU altogether?
Touched an exposed and white hot burning nerve….
… you need to know what helping the handicapped means…. so you know if you are one to take it on or should leave it alone. You can only go one of two ways. Either is ok… just choose. I’ll explain…
Assistive Technologies! With great reverence I do state; “The highest calling a Hacker can answer to, and the one place where your hacking can make the greatest difference to the world… is to the physically handicapped.”, but once you take it on it is a commitment without end. There is NOBODY like a hacker to be innovative…. no engineer is worth a toot compared to a motivated hacker! But….. When you alone hold the designs and knowledge supporting the handicapped you must continue support of the technology for life. If you touch this world take it seriously! Electronics can do a great deal but is susceptible to single component failure that must be repaired or a whole electronics dependent assisted lifestyle comes to a tragic end at little or no notice, bringing many tears and anguish as well as a tragically and suddenly crushed life that goes on suffering for years after. A whole world ends. Keep your records, provide lifetime support or stay away to begin with! Dying? Print it all out quick!
You remember and perhaps now use the MS Magnifier and other assistive technology works added to Windows for the handicapped? Close friend of mine, greatly disabled yet making highly effective and positive contributions out of his life to local govt AND society was the one that introduced MS to these and saw them through the complications of licensing till these were legally and permanently added to Windows, and his effort was made simply because it made a positive difference to the disabled of the world. And he was working for a blind man running a non-profit to convert college math textbooks to braille. These are widely used…. the blind learn college level math… no mind should be denied learning…
You all know of Stephen Hawkings and his hacked together voice system, and the later complications rebuilding it yet making it still sound the same, it was well televised. Contributing to him was far more than worthwhile and it was 100% hacking, and I am totally sure any one of us would have been grossly honored to be asked to assist. I suspect most have also seen an Epson HX-20 mounted on a wheelchair providing it’s user with ability to guide the wheelchair through puffs of air or a tongue activated joystick (I have an HX-20 on hand should anyone have such a verifiable wheelchair need of replacement cpu.). All of this is hacking. If you do hacking and apply it to the handicapped, you are making a serious difference to a life and society as a whole, all positive.
I became fully disabled just short of Y2k. That placed me in the care of these disabled people that helped me back up by rehab and WORK, while I learned to control my condition. I always had a belief in the handicapped from long before, and from an early age, knew they are still really people. No mind should go to waste. I did come back and kick ***! And I’m down again. It gets worse with age… yet it will be that I rise again!
Help any handicapped person that asks. Imagine what it takes of them to ask. Imagine what it adds to you to give. Imagine what it adds to the world to have lent that hand.
Imagine what it takes a handicapped person to ask… imagine what it takes to respond with giving… imagine what response there is to refusal.
And now I do seek you accept my apologies… not tempt me to respond. We each deal with our own difficulties and some may have talents to apply…. if you have them to apply it will confront you personally, not here.
Post Comment button is a ‘lil bit hard to push for some reason…
You have succinctly stated a number of completely true points about creative use of technology to help the handicapped. If you’re not “all in” it is best to “stay out” – and pass your ideas to those who can make the commitment to follow through and maintain what is “hacked” together to help them in their daily struggles. I salute you for your efforts, and continued struggle not to accept your situation but to strive to improve it. If only more able bodied folks were as driven to improve life’s conditions we would all be better for it.
Thank You. But you said something even greater…. folks should collaborate and pass on their ideas to those that are making the commitment. We are ALL needed. There was no intent to make warning to keep assistance and ideas away yet reading what I wrote it may have dissuaded those with better ideas. Everyone should still make contributions, fires are started with just a spark. Note whom it is that is committed, assist those persons, and be proud of your part. Be a leader if you are such to take the responsibility.
An update for your interest… The braille math translation work was by a long lived non-profit that came to a natural end not long ago. The originator is both blind and deaf. The State took up the flag and transferred the idea to even greater good from those roots. Simply too good an idea to let pass, they trained prison inmates to do the work. It has succeeded, giving these great competence with a solid career outlook on the outside in a uniquely specialized field that worldwide was a near vacuum (to the best of my knowledge) and hindering the blind.
I’ve never seen anything result in so many positives during it’s life, and again bloom to so much more at it’s ending. Sounds like I’m being foolishly romantic… but actuality is the case give has been grossly understated.
http://www.sleepingangel.com/ebworld/ebkdaniel.htm
Daniel taught me all the rest about disability that I need to know. I was his scout leader and neighbor and babysitter 4 or more days most weeks…
“A white hot open wound.”. Not a bleeding heart at all. Nobody looking at these in this world would be unaffected. Quite happy to be of service….
Everyone’s help is needed.
Hi folks, I had some contact with the guy who invented vOICe way back when, he seems to think a Pi might work but a custom hardware would be better due to the relatively large processing overhead.
Maybe 4* Zero 1.3’s would work, two per eye processing odd/even frames separately then combining audio channels through a multiplexer?
But yes, the basic idea is indeed sound and as someone who has vision problems already (38) need to invest in some sort of backup system in case things get any worse. As it is I can’t read SMD markings any more so something is obviously deteriorating.
I have non progressive optic nerve atrophy and see roughly 20/70. For the first 13 years of my life I saw 20/400 due to a UCLA quirk and I’ve found developmentally a stranger journey… It works but recently I went into a vocational agency (25 years old, BSBA degree being done, small decentish business I started, no prior work experience) and carrying that “low vision” title makes me realize how I struggle in some ways. I’m personally interested in a rather small screen attached to the edge of one’s glasses and a camera to augment things.. EI a digital form of a bioptic with many more features. I can drive but I have issues determining the light colour but am otherwise safe in the rural areas. Distance is a pain and when i see all these other folks who have it harder off-I wish I could do something about it. This is a nice step in the right direction, however, it must be rather small and practical to gain traction. AVE’s philosophy works for prototypes but the final spit and polish will be needed for adoption.
Anyways are there any collaborative projects cooking on a larger scale? I am keen on various technologies and I suppose I want something to invest into which the chance of it producing something highly useful for numerous people.
This is awesome!!! So impressed with his adventurous spirit, as well as the technology behind it! So incredible to see others with low vision making such great strides and taking on the world. Are there any other particular options for assistive technology for visually impaired, such as this one, that he would recommend for everyday use? It looks like they’re a great option for partially sighted people, but I’m always looking for as much help/opinions as I can get. Thanks in advance! :)