Gyroscope-based Smartphone Keylogging Attack

smartphone_keylogging_with_gyroscopes

A pair of security researchers have recently unveiled an interesting new keylogging method (PDF Research Paper) that makes use of a very unlikely smartphone component, your gyroscope.

Most smart phones now come equipped with gyroscopes, which can be accessed by any application at any time. [Hao Chen and Lian Cai] were able to use an Android phone’s orientation data to pin down what buttons were being pressed by the user. The attack is not perfect, as the researchers were only able to discern the correct keypress about 72% of the time, but it certainly is a good start.

This side channel attack works because it turns out that each button on a smart phone has a unique “signature”, in that the phone will consistently be tilted in a certain way with each keypress. The pair does admit that the software becomes far less accurate when working with a full qwerty keyboard due to button proximity, but a 10 digit pad and keypads found on tablets can be sniffed with relatively good results.

We don’t think this is anything you should really be worried about, but it’s an interesting attack nonetheless.

[Thanks, der_picknicker]

31 thoughts on “Gyroscope-based Smartphone Keylogging Attack

  1. Interesting indeed. My first thought was if handedness would affect accuracy, or would measurements simply need to be reversed.

    “The motion of the
    smartphone during keystroke is affected by many factors, such as the typing force, the resistance force of the
    holding hand, the original orientation of the device, and
    the location where the supporting hand holds the device.”

    I didn’t read the WHOLE research paper, but from this it seems they’re looking at the same hand doing the typing and holding the device. One would think the rotational forces would be the same when you touch the screen in the same spot from the opposite hand holding the device, but they would be significantly smaller.

    1. You mean like have a plank of dumb wood with letters painted on then stick a gyro to the back of it and make it a keyboard?
      Cool idea, would definitely be interesting to see how well it worked.

    2. Then how do you press Ctrl+C? And when you put the keyboard on the desk it maybe more difficult for your PC to recognize the key. It’s a cool idea, and it is just an idea.

  2. I don’t think this is using a gyroscope– linked article even uses the work accelometer. I can’t think of a single phone that has a gyroscope. The new Wii (and maybe the new Playstation) controllers have gyros. Most smartphones have accelerometers.

    Accelerometers measure linear acceleration. Gyros measure rotational acceleration.

  3. They can increase accuracy by having its output estimated letters compared as “words” usign a “hamming-distance styled” estimator with a dictionary and then those word level estimations can be grammatically parsed to estimate phrases.

    It all boils down to the enthropy of the language of the user and the total real information the gyro can get.

  4. Having successfully installed the gyroscope sniffing software, isn’t that the actual attack? I mean if you can install that, why not just install a straight-up keylogger? Anyway, I wonder if they can improve accuracy using the same auto-correct algorithm iPhone uses, or that T9word feature on some phones? Maybe that’s what Khanzerbero is talking about.

    1. Sensor data like this can generally be accessed by any app that wants to. Just have to get them to install the app in the first place.

      Keylogger, on the other hand, requires exploitz.

  5. If you’re trying to use this to get the phone’s password, the 72% isn’t such a big deal because you’ll have plenty of opportunities. Record multiple login attempts, and that should handle the error nicely.

  6. Easy enough to defeat though. Just lay the device flat on a table while typing.

    Without that, it would be interesting to see how the accuracy compares on a tablet vs. a smartphone.

  7. Then physical orientation matching and sound spectrum pattern matching are also holes.

    These ‘researchers”gotta be under some trademark or nobody would even be talking about this..

  8. So, your touch screen presses generate distinct patterns, yes, that’s a given as it’s how phone games work for things like rotation.

    I see how this works to identify individual presses (aka, you can tell you’ve pressed *a* certain button) on a phone by phone/user by user basis.

    However, matching up those distinct presses to a value so you can actually figure out *which* button has been pressed is harder if not near impossible due to the variables involved.

    Ie, is phone free standing, on a surface, is left or right hand being used, in transit (vibrating/jittering around), on a motherchuffing boat, etc…

    I think this while a nice discovery and may end up causing things like accels/gyros to be accessed only by whitelisted or active apps, I don’t think it’s much of a security problem.

    1. Basically, what I’m saying is that graph up there showing “key 1” “key 2” et all, is misleading.

      The reason? Either the app has just designated it an arbitrary value of “key 1” when really it could be number 7 on the keypad, or the reseachers will have matched up keypad numbers to the correct profile manually while testing it, the app itself won’t (unless it has some pattern matching).

  9. Seconded on the “Improve Accuracy” concept.

    @Khanzerbero: How did you arrive at the use of Hamming as opposed to several other tools for Error Detect/Correct? My background is more hardware than software and I am trying to learn :]

    This hack seems in two parts- accessing the sensor data and parsing it into the desired keystrokes. Hmnm- exploiting a market app for something innocent appearing that needs positional data+communications would serve as the judas data conduit? One part of exploit seems plausible to me. Of the many ways to hack the rest?

    One being grepping the keystroke handling of the firmware. As reputedly there are internal math models for device’s using variants of timing detection for several common categories of input error:

    berkeley.intel-research.net/~klyons/pubs/autowhiteout++chi08.pdf

    autowhiteout, in one phrase for the tl:dr version=detects timing and blanks a suspect keystroke/s which prompts the human to retype.

    The semi-related one for speech to text software:

    http://www.research.ibm.com/compsci/spotlight/hci/halverson99.pdf

    That study by IBM of speech-to-text software offers little depiction of “math tools” so it only serves to help build our understanding of Human>Machine parsing correction “overview” in a limited case. But- it’s on track to taking a 70~% confidence level character stream and someday raising the confidence level far enough to risk burning up blown access attempts.

    Oh- I updated a friend’s phone to Android 2.2x and it’s feature of “haptic by vibrate” might offer a way to totally FUBAR this not even fully formed as an exploit keylogger CONCEPT.

    Which made me contemplate having the phone begin a soft vibrate after keystroke n to frustrate this proposed exploit rather completely.

    1. Don’t think vibrate would fubar, though I think it might hamper.

      Remember Sony when they said the vibrate function would mess with the motion sensing (yes, lame excuse to avoid paying royalties ;) ?

      But seriously, the vibrate occurs only AFTER you press a key, and it moves it predictably. All you’d have to do is basically use your gyro/accel to record it vibrating without a keypress and effectively “subtract” that motion.

    2. I was thinking about they first should do principal component analysis, develop son eigen-letters and then in the principal components space they shloud do an estimate at the letter level, and then at the word level a hamming distance based estimation against a dictionary, and then at the phrase level some semantic analisys.

      Why hamming? i guess its faster.

  10. What everybody (or at least the hackaday poster) misses is that even with a statistical chance of 72% of being right, you weaken the security a lot.

    Suppose there is a 4-digit PIN that allows 10 tries, for a 1/1000 chance of breakin by random guesses. You have to steal on average 1000 phones and randomly try PINs before you hit the jackpot.

    But with say a 90% (the calculations are easier with a round number) chance of getting the numbers right from the attack means you have only 0.9^4 = 65% chance of getting the pin right on the first try. But if you didn’t get it right on the first try, you have 9 tries left. You often have a “next candidate”. So you can try the 4 “second-best” tries next. If the second-best has a 90% chance of being correct (given the first try was wrong), then we again have 65% chance of success on the next 4 tries… With 5 tries left, we can try a few two-wrong PINs but that doesn’t make a big difference. In this example with imperfect pin-stealing, the average number of phones to steal before the PIN can be guessed goes from 1000 to under two.

    With the reported 72% accuracy and only 3 tries, there is still a significant advantage over “blind guessing”. The first guess has a 27% chance of success. With the two next tries this can be raised to almost 40%. On average after stealing 2.5 phones you have guessed the PIN correctly within 3 tries.

    Now

Leave a Reply to kernelcodeCancel reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.