Hackaday Prize Entry: LipSync, Smartphone Access For Quadriplegic People

For most of us, our touch-screen smartphones have become an indispensable accessory. Without thinking we tap and swipe our way through our digital existence, the promise of ubiquitous truly portable computing has finally been delivered.

Smartphones present a problem though to some people with physical impairments. A touchscreen requires manual dexterity on a scale we able-bodied people take for granted, but remains a useless glass slab to someone unable to use their arms.

LipSync is a project that aims to address the problem of smartphone usage for one such group, quadriplegic people. It’s a mouth-operated joystick for the phone’s on-screen cursor, with sip-and-puff vacuum control for simulating actions such as screen taps and the back button.

To the smartphone itself, the device appears as a standard Bluetooth pointing device, while at its business end the joystick and pressure sensor both interface to a Bluetooth module through an Arduino Micro. The EAGLE board and schematic files are available on the project’s hackaday.io page linked above, and there is a GitHub repository for the code.

Technology is such a part of our lives these days, and it’s great to see projects like this bridge the usability gaps for everyone.  Needless to say, it’s a perfect candidate for the Assistive Technology round of the Hackaday Prize.



Hand Waving Unlocks Door

Who doesn’t like the user interface in the movie Minority Report where [Tom Cruise] manipulates a giant computer screen by just waving his hands in front of it? [AdhamN] wanted to unlock his door with hand gestures. While it isn’t as seamless as [Tom’s] Hollywood interface, it manages to do the job. You just have to hold on to your smartphone while you gesture.

The project uses an Arduino and a servo motor to move a bolt back and forth. The gesture part requires a 1sheeld board. This is a board that interfaces to a phone and allows you to use its capabilities (in this case, the accelerometer) from your Arduino program.

The rest should be obvious. The 1sheeld reads the accelerometer data and when it sees the right gesture, it operates the servo. It would be interesting to do this with a smart watch, which would perhaps look a little less obvious.

We covered the 1sheeld board awhile back. Of course, you could also use NFC or some other sensor technology to trigger the mechanism. You can find a video that describes the 1sheeld below.

Continue reading “Hand Waving Unlocks Door”

Abusing a Cellphone Screen with Solenoids Posts High Score

This Raspberry Pi 2 with computer vision and two solenoid “fingers” was getting absurdly high scores on a mobile game as of late 2015, but only recently has [Kristian] finished fleshing the project out with detailed documentation.

Developed for a course in image analysis and computer vision, this project wasn’t really about cheating at a mobile game. It wasn’t even about a robotic interface to a smartphone screen; it was a platform for developing and demonstrating the image analysis theory he was learning, and the computer vision portion is no hack job. OpenCV was used as a foundation for accessing the camera, but none of the built-in filters are used. All of the image analysis is implemented from scratch.

The game is a simple. Humans and zombies move downward in two columns. Zombies (green) should get a screen tap but not humans. The Raspberry Pi camera takes pictures of the smartphone’s screen, to which a HSV filter is applied to filter out everything except green objects (zombies). That alone would be enough to get you some basic results, but not nearly good enough to be truly reliable and repeatable. Therefore, after picking out the green objects comes a whole chain of additional filtering. The details of that are covered on [Kristian]’s blog post, but the final report for the project (PDF) is where the real detail is.

If you’re interested mainly in seeing a machine pound out flawless victories, the video below shows everything running smoothly. The pounding sounds make it seem like the screen is taking a lot of abuse, but [Kristian] mentions that’s actually noise from the solenoids and not a product of them battling the touchscreen. This setup can be easily adapted to test out apps on different models of phones — something that has historically cost quite a bit of dough.

If you’re interested in the nitty-gritty details of the reasons and methods used for the computer vision portions, be sure to go through [Kristian]’s github repository where everything about the project lives (including the aforementioned final report.)

Continue reading “Abusing a Cellphone Screen with Solenoids Posts High Score”

Smartphone-based Robotic Rover Project goes Open Source

[Aldric Négrier] wrote in to let us know that his DriveMyPhone project has been open sourced. The project is a part telepresence, part remote-controlled vehicle, part robotic rover concept on which he says “I spent more time […] than I should have.” He has shared not just the CAD files, but every detail including tips on assembly. He admits that maybe a robotic chassis for a smartphone might not seem like a particularly new idea today, but it was “an idea with more potential” back in 2010 when he first started.

The chassis is made to cradle a smartphone. Fire up your favorite videoconferencing software and you have a way to see where you’re going as well as hear (and speak to) your surroundings. Bluetooth communications between the phone and the chassis provides wireless control. That being said, this unit is clearly designed to be able to deal with far more challenging terrain than the average office environment, and has been designed to not only be attractive, but to be as accessible and open to repurposing and modification as possible.

Continue reading “Smartphone-based Robotic Rover Project goes Open Source”

Smartphone and IR Line Laser Measure Distance

Measuring the distance using lasers is a mainstay of self-driving vehicles and ambitious robotics projects. The fine folks at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) decided to tackle the problem in an innovative way. [Jason H. Gao] and [Li-Shiuan Peh] used an infra-red (IR) line laser and the camera on a smartphone. Their prototype cost only $49 since they used a smartphone that was on hand. The article reports good results using the device outdoors in direct sunlight which is often a challenge for inexpensive lidars.

The line laser creates a horizontal line that is reflected back to the camera on the phone. The vertical position of the laser on the camera image lets the phone calculate the distance by parallax. To bring out a faint laser reflection, the algorithm compares four images – two with the laser on and two with it off – and subtracts the background. Using a smartphone for this is ideal since it automatically adjusts for light level and can easily be upgraded to a newer phone with a better camera later.

This should be a cheap and easily replicable setup. If you make one of these, let us know. If you need something more refined, check out this post on interfacing the Neato vacuum cleaner’s XV-11a lidar with the Raspberry Pi.

Continue reading “Smartphone and IR Line Laser Measure Distance”

Dry Ice is Nice for Separating Broken Phone Screens

Smartphones are the opium of the people. If you need proof, just watch the average person’s reaction when they break “their precious”. Repairing smartphones has become a huge business. The most often broken item on phones is of course the front glass. In most cases, the screen itself doesn’t break. On newer smartphones, even the touchscreen is safe. The front glass is only a protective lens.

The easiest way to repair a broken front glass is to swap the entire LCD assembly. For an iPhone 6 plus, this will run upwards of $120 USD. However, the glass lens alone is just $10. The problem is that the LCD, digitizer and front glass are a laminated package. Removing them without breaking the wafer thin LCD glass requires great care. The hardest part is breaking down the optical glue securing the glass to the LCD. In the past that has been done with heat. More recently, companies from China have been selling liquid-nitrogen-based machines that cool the assembly. Now immersing a phone screen in -196° C liquid nitrogen would probably destroy the LCD. However, these machines use a temperature controller to keep a surface at -140° C. Just enough to cause the glue to become brittle, but not kill the LCD.

[JerryRigEverything] doesn’t have several thousand dollars for a liquid nitrogen machine, but he does have a $5 block of dry ice. Dry ice runs at -78.5°C. Balmy compared to liquid nitrogen, but still plenty cold. After laying the phone screens down on the ice for a few minutes, [Jerry] was able to chip away the glass. It definitely takes more work than the nitrogen method. Still, if you’re not opening your own phone repair shop, we think this is the way to go.

Broken phones are a cheap and easy way to get high-resolution LCD screens for your projects. The problem is driving them. [Twl] has an awesome project on Hackaday.io for driving phone screens using an FPGA. We haven’t seen it done with iPhone 6 yet though. Anyone up for the challenge?

Continue reading “Dry Ice is Nice for Separating Broken Phone Screens”

Building One Thing In China

Conventional wisdom dictates that if you need to make a million of something, you go to China. China is all about manufacturing, and there aren’t many other places on the planet that have the industry and government-subsidized shipping that will bring your product from China to people around the world. Building a million things in China is one thing, but what about building one thing? How do you create a working prototype of your latest product, and how do you make that prototype look like something that isn’t held together with zip ties and hot glue? The folks at Hatch Manufacturing have a guide for doing just that, and lucky for us, it’s a process that’s easy to replicate in any well-equipped shop.

In this tutorial/case study/PR blitz, Hatch Manufacturing takes on constructing a one-off smartphone. The Huaqiangbei markets in Shenzhen are filled with vendors selling smartphones of all shapes and sizes. If you want a miniature iPhone running Android, that’s no problem. If you want a phone that looks like a 1969 Dodge Charger with the Stars and Bars on top, you can find it in China. But how are all these phones made, and how do you show off a prototype to factories begging for business?

The answer, as is always the case, comes from one-off manufacturing. Building, assembling and reworking PCBs is a well-trodden path whose process could fill several volumes, but for this post, Hatch Manufacturing decided to focus on the plastics that go into a smartphone or tablet.

Once the case or enclosure is designed with a few CAD tools, a block of plastic is run through a mill. After that, it’s a matter of painting and finishing the latest smartphone that will show up in the Chinese market. Putting a professional finish on a block of plastic is something that will look familiar to anyone who has ever assembled a miniature plastic model. There’s priming, airbrushing, sanding, more painting, sanding, wet sanding, and still more sanding. After that comes polishing the plastic part to a fine finish. It is extraordinarily labor intensive work even for a skilled hand with the right equipment.

Once the plastics are done, the PCB, display, battery, and everything else comes together in a completely custom one-off prototype. It’s very similar to how this would be done in any small shop with a benchtop mill and a dozen grades of wet/dry sandpaper. It’s also something anyone can do, provided they have enough practice and patience.