While it seems that many people are wise to shoulder surfing, keeping a lookout for anyone spying on their passwords, [Haroon] wrote in to remind us that the threat is just as real today as it ever was.
The subjects of his research are touch screen phones and tablets, which utilize on-screen keyboards for data entry. He says that while nearly all password entry boxes on these devices are obscured with the traditional line of asterisks, the keyboards themselves are quite an interesting vulnerability.
Since touch screen technology can be finicky at times, most vendors ship their devices with some sort of key press verification system. On the iPhone and iPad, for instance, each key is highlighted in blue following a button press. This functionality makes it quite easy for shoulder surfers to casually steal your password if you’re not paying attention.
But what if you are well aware of your surroundings? [Haroon] has developed a piece of software he calls shoulderPad, which is based on openCV that does the surfing for him. The application can monitor a video stream, live or recorded, extracting the user’s password from the highlighted button presses. His demonstrations show the recording taking place at a relatively close distance, but he says that it would be quite easy to use surveillance footage or zoom lenses to capture key presses from afar.
He does say that the button highlighting can be easily disabled in the iPhone’s options pane, which should negate this sort of attack for the most part.
Continue reading to see a quick video of shoulderPad in action.
[youtube=http://www.youtube.com/watch?v=RGS4q-WHTlg&w=470]
Erm, how do you disable keyboard highlighting in iOS?
Heck, how do you *enable* it? It’s certainly not turned on for my iphone running ios 4.3.3
It would be nice if it was disabled only for password fields. Since disabling it globally is unnecessary and possibly annoying.
Or maybe it does that?
And then there is the fact that phones and tablets show each character you type in the text box momentarily before obscuring it. You could scan that for passwords too in a video stream of sufficient resolution (but obviously the keyboard reading is easier to see).
-Taylor
Why there is no google+ button here;)?
as for this “hack” – great job, and certainly is not only applicable to blue hilighting keyboards – just that simple color change made it easier to do.
If we can see the password box, the finger position and when letters appear, we can take finger postion and assume that button under it was pressed at the time. Well, now you can troll your acquaintances saying that Android is more secure because there is so much disparity in keyboard apps and display sizes;)
That is not the stock iOS keyboard in the picture. iOS does not highlight key presses in blue. It also is not black.
Hi Josh.
It is indeed the stock keyboard, but with “complex passwords” enabled.
The white keyboard shows up for regular typing, and this one shows up for password prompts.
Maybe in the future password fields on touch screen devices will have their on-screen keyboard’s keys randomized?
I use a morse code keyboard on my android phone, that ought to stop ’em.
I think you could steal anybody’s password from any keyboard (SW or HW) if you get to film them as they do it (and you know the layout of the keyboard, or can discern it in the footage). I really see no protection against that, unless we really start typing in passwords in Morse code…
Because computers can’t decipher morsecode?
@Anonymous: I’m with you on this one. A randomised keyboard layout for password fields (different each time) would not only negate shoulder surfing, but also prevent build up of fingerprints in certain positions on the screen, which, while it may not be such a big deal for occasionally used online passwords, definitely is an issue for some unlock codes.
there’s a morse code android app that replaces the standard keypad. i was able to get pretty fast sending messages with it, and i just learned the code not long ago
Interesting post!
There is also the iPhone version of this attack, which can also steal passwords and long texts while the victims move naturally during typing.
http://slashdot.org/submission/1716844/Fast-Automatic-iPhone-Shoulder-Surfing
Here is the video of the iPhone version of the attack http://www.youtube.com/playlist?list=PL81F91E404B928833