Using Smartphone Cameras To Make Sure Drivers Are Looking At The Road

Most of us are probably quite aware of the damage that a car can inflict when driven by a distracted driver. In an ideal world, people who are driving a car would not allow something like their phone to distract them from their primary task of being the primary navigation system for the 1+ metric ton vehicle which they are controlling.

Many smartphone apps as well as in-car infotainment systems have added features over the years that try to prevent a driver from using them, but they run into the issue that it’s hard to distinguish between passenger and driver. As it turns out, asking the human driver whether they are the driver doesn’t always get the expected result. This is where [Rushil Khurana] and his team at Carnegie Mellon University (CMU) have come up with a more fool-proof approach.

In their paper (PDF), they cover the algorithm and software implementation that uses the smartphone’s own front (selfie) and back cameras to determine from the car’s interior which side of the car the user is sitting in, and deducing from that whether the user is sitting in the driver’s seat or not.  From there it is a fairly safe assumption to make that if the user is sitting in the driver’s seat, and the car is moving, that this user should not be looking at the phone’s screen.

In a test involving 16 different cars and 33 users, they achieved an overall accuracy of 94% with the phone held in the hand, and 92.2% while docked. This is more reliable than the other approaches covered in the paper, and as a benefit does not require any extra hardware. Who knows, upcoming smartphones may include a feature like this, so that apps can easily determine what feature set should be made available to a driver, if any.

25 thoughts on “Using Smartphone Cameras To Make Sure Drivers Are Looking At The Road

  1. This is an interesting idea! While I haven’t read the paper, I assume that the software can be circumvented by covering the cameras. Presumably this would be too much of a hassle for a lot of people, but maybe not. Anyway, it’s a clever idea trying to solve a major problem. So I appreciate the article.

    1. That would be me, taping it over when it kept shutting off on a mount on the A pillar, when I was using a “non-approved” 3rd party GPS navigation app….

    1. Or Aus. That would be my next trick if taping didn’t work, install UK firmware. Then if it really wants to play me, it would end up seeing mirror images or something.

    2. Well if the software uses gps to determine if you’re moving it would know that is a lie, but the real question is what if your driving a RHD car in an LHD country or visa versa, now that would be a bitch!

  2. Maybe they should stop using smartphones and tablets as the interface for in-vehicle navigation and entertainment? Seems like the only reasonable solution, but the real issue is that they’re tired of paying the BOM for non-touchscreen controls and HID. Spy on the passengers all you like, it’s still fundamentally less safe than tactile interfaces.

    People aren’t going to put up with their devices shutting off functionality by looking at them through facing cameras. They will cover the cameras or use their phones with other apps to bypass it, which will just be more dangerous. Manufacturers want to cut costs and only develop these impotent safety measures as a PR move. They don’t care.

    1. Don’t know about you, but I don’t want to pay the price of a phone for a crappy device that has out of date road data, overpriced data updates, never gets new features, etc. Plus it can’t read my calendar or contacts list for addresses, and hasn’t got google to find businesses.

      Especially as I’ve got a perfectly good device in my pocket which can do it all.

  3. I’m not a fan of letting computers decide what people are allowed to do. All code is limited by its decision tree/training set, and the world is full of edge cases.

    The promise is always Tony Stark’s Jarvis or a non-homicidal version of HAL.. notable for being plot devices that never make any real decisions. The deliverables are autocorrect and Microsoft’s Clippy.

    1. “I’ve had an accident and am sinking in a lake while trapped in my car, call emergency services!!”

      “I’m sorry, Dave….you can’t make a phone call while your car is moving….”

      As with most pretty-well-engineered things, we don’t notice the versions of this that we take for granted from Google search suggestions to YouTube video “nexts” and mapping software route selection. This is quite apart from whether we find the results hilarious or annoying.

      There is a worthwhile read of how Clippy went so terribly wrong through incomplete implementation of Bayesian networks here (Slideshare) :

      https://www.slideshare.net/20073241/bayesian-network-backbone-of-clippy?from_action=save

      As always, code comes (mostly) from people and people succumb to shortsigtedness, committees, corporate decisions, mistakes, deadline pressure, and all the rest of the non-deterministic monkey-brain problems. In short: hominid in, hominid out. (“HIHO”)

      There’s no app for that. Things would be terribly dull if there were.

    2. Remember when device users would have already lit their pitchforks on fire and started on march towards the manufacturer’s headquarters? Those were simpler times…

      Also, here’s an idea why don’t you teach your kids to not be douche nozzles and put the damn phone down. You cannot regulate away shitty behavior. all this stuff does is you rode yet more of our rights to privacy and to make our own damn mistakes.

      I for one refuse to have a device that is constantly looking at me or scanning the room I’m in or listening for buzzwords. I’m even posting this on what would be considered an ancient Android device. Screw new technology, all it wants to do is listen for a new buzzword for its advertising algorithms. These are data pads from Star trek in our pockets and all they can do is show us the next McDonald’s sandwich in the wrong language with subtitles … still in the wrong language…..

      I swear I’m not better, but I am salty.

  4. So…you’re proposing that the nanny-state software app that spies on you through your own cameras is not going to read the GPS location off your phone too?

    My question is when we can expect the “upgrade” to the app enabling it to make the 911 call to file a police report against you?

    1. I was going to mention “nanny state” also, but even if it is not mandated by some government entity that loves intruding on your privacy, (i.e controlling your life) it will be implemented by commercial interests with the same goal (FAGAM).

      1. Just as a side note I never liked the phrase “Nanny State” as nannys tend to be loving and caring, while most governments tend to be abusive and controling, I think “Wife Beater State” would be a better term.

  5. App based: uninstall

    Hardware/firmware based: I’m just going to buy a phone from someone who doesn’t have this “feature”

    Alternative hardware/firmware based: custom os

    I appreciate your good intentions, but I’m not going to volunteer letting my phone babysit me. I don’t imagine people who willingly break the law will be volunteering any time soon either. Thanks for trying, but it’s not going anywhere

  6. I hate “nanny state” stuff as badly as some of the tinfoil hate crowd does.
    But….
    Speaking as someone who has had to brake hard and take to the rumble strips in order to avoid smartphone users. Been cut off in city traffic.
    Just see people looking down in their lap, with one hand on the wheel (as if they think no one can tell what they are doing ).
    Are they wearing a Bluetooth earpiece? yep talking non-stop.
    I would be happy to see it federally mandated that new cars triangulate a powered phone within its cabin and refuse to initiate any initial movement until you interlock it with the car ecu.
    Phone ringing? Your narcissistic ass can let it go to voice mail or the text hole, while you pull over and put the car into park.
    Unless you are driving the only Ambulance in your county or you run the suicide hot line ,
    your text or call is not too fucking important to wait the time it may take to park.

Leave a Reply to Greg ACancel reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.