As auto manufacturers have brought self-driving features to their products, we’re told about how impressive their technologies are and just how much computing power is on board to make it happen. Thus it surprised us (and it might also surprise you too) that some level of self-driving can be performed by an Android phone. [Mankaran Singh] has the full details.
It starts with the realization that a modern smartphone contains the necessary sensors to perform basic self-driving, and then moves on to making a version of openpilot that can run on more than the few supported phones. It’s not the driver-less car of science fiction but one which performs what we think is SAE level 2 self driving, which is cruise control, lane centering, and collision warning. They take it out on the road in a little Suzuki on a busy Indian dual carriageway in the rain, and while we perhaps place more trust in meat-based driving, it does seem to perform fairly well
Self driving features are codified into a set of levels for an easy reference on what each is capable of doing. We’ve taken a look at it in the past, should you be interested.
>> “They take it out on the road in a little Suzuki on a busy Indian dual carriageway in the rain…”
This… does not seem like the optimal venue to debug your software
Well….. it *is*… India
https://www.youtube.com/watch?v=Mjqeyd0d2QY
It does not, however to be fair, much of the testing in the US has been on real roads with real cars and pedestrians to dodge (sometimes poorly :( ) so it’s at least consistent with most self-driving testing environments.
To be fair(er), that’s *after* tens of thousands of hours of simulation and then testing on closed tracks and carefully controlled conditions, not just “welp, let’s see if we die or not”
Developer of flowpilot here, The torque on steering wheel is limited and made sure that it never turns fast enough and the driver has enough time to react. The car’s EPS ECU is DOES NOT allow to alter car’s trajectory too quickly for the driver to safely react. So as long as the driver is attentive, its just like any other automotive grade ADAS system.
Hi, really appreciate your work to make openpilot easier to hack on. Can you share what’s the wireless can bus interface you use?
We do not recommend or support wireless CAN anymore. You’d need a panda which also has safety built into it follows MISRA C spec.
How do you see your way forward? AFAIK Comma3 does not support phones anymore and some code runs on the DSP on their SOC. Are you still able to profit from newer AI models from Comma or do you use an old driving model from the Comma2?
Comma 3 is snapdragon 845. All the modern android phones have DSP built in and you can run models on them.
We have the ‘f2’ version that uses models from comma 2, and then there is the ‘f3’ version that runs model from comma 3.
More details here: https://flowdrive.substack.com/p/flowpilot-rebooted
The power steering’s ECU DOES NOT allow car to alter its trajectory too quickly for the driver to safely react. So as long as you are using that safety, it dosent matters if above it is some DIY autopilot system or one developed by automakers.
Scary.
It is relatively easy to run a vehicle in trained situations. Covering general use and establishing a save ride is something completely different.
I laugh that we use so much calculation power to run a phone on… driving a car by a device designed for phoning is just irresponsible, in my point of view not a good demonstration of hacking skills nor common sense. This belongs in the category: do not try this at home!
GvT
Hi GvT, please go through the safety blog post of openpilot:
https://blog.comma.ai/understanding-the-openpilot-safety-model/
This goes into detail of making a safe DIY autopilot system.
Hi Ma,
thanks for the link.
This does not alter my opinion. It describes some fundamentals but safety goes much deeper! Referring to the ISO 26262 and ASIL a lot of commonly accepted standards might not be fulfilled. I doubt that the code has been developed and tested accordingly or that the hardware it runs on fulfilled these requirements. How can you ensure that the compiler works correct or that the no bit in the memory flipped? How do you ensure cut off times in such an OS running several tasks at once?
A prove of concept? YES. A prove that the OEMs shoot with canons on sparrows? NO!
Greetings
GvT
Openpilot or flowpilot does not bypass the safety built into the steering ECU. On top of that there is panda which makes redundant checks that the commands being sent to actuators are safe and under the limits.
On top of that, anything else is a QM, let it be Tesla autopilot or mobile eye SOC
https://en.wikipedia.org/wiki/Automotive_Safety_Integrity_Level#QM
It dosent matters if code on the phone bit flips, the ASIL-D ECUs in cars are not affected.
If you don’t want to use android, what’s stopping from getting and industrial grade SOC and running this software on that ?
Its multi-platform and can run on any Linux based OS.
1. A smartphone is not a device designed for “phoning”.
2. Look up Ernst Dickmanns.
3. It’s the opposite of “don’t try it at home”. (I.e.: try it at home ONLY.)
I’ve just looked up Ernst Dickmanns, that’s very interesting! Thanks
Openpilot has very good results for an open-source project. There are many demos where it works much better than my car’s driving assistant (and probably many other ‘lane keeping’ assistants). Is it better that all – absolutely not, but it is quite good. Quite remarkable given it only works with a single camera.
This is not what one would call ‘self driving’ even though everyone likes to use the term. It is lane keeping assistance and also some cruse control like thing which maintains the distance with the car upfront. It absolutely requires a human behind the wheel. I would not trust it to chance lanes as it has no clue about any cars behind/next to you.
The article is about running open pilot on a regular phone vs, some specific models the team decided to support. In that regard it is a good hack. Granted you still need “root” access and that’s not readily available on every phone.
You don’t need root access, the non root version works out of the box.
Next thing you know your phone will be getting a ticket for talking on its human while driving
Not sure if “surprising” would have been my choice of adjective. “Alarming” maybe?
Can’t wait for the script kiddies to try this one, once they get bored with the Flipper Zero. Luckily, there still seems to be a bit of effort required to get up and driving.
Why alarming, exactly? Can you describe exactly why you’re alarmed, without insulting entire groups of people? Are you alarmed because you don’t know anything about openpilot so you instead replace fear with knowledge?
Why are you so defensive? And so what if he insults you? I can explain why i find it alarming. There is a huge difference between proper safety trials and “winging it”. I would not want to be on the road with someone trusting their safety -and my safety by proxy- to a hack. You can say what you will about openpilot, but it simply isn’t ok to make others on the road pay for your lack of sense.
I have to ask if you read the article properly. Its only SAE lvl 2 and as other comments pointed out its done slow by the car hardware so the driver can react at any moment. A highway is more dangerous on a windy dy because there you get jerked around without warning and faster.
“so the driver can react at any moment”
Like the dead Tesla drivers who were obviously over reliant on their “self driving” car?
When you are behind the wheel of a multi-ton vehicle going 100+ km/hr, your sole JOB is to pay laser focused attention to your driving and anything that tends to lure you away from that is not a good thing IMO. Cruise control and lane drift and obstacle warning radar is as far as I’d go.
Didn’t one of the early self-driving car companies open source their software before the pandemic and come out with something like this? I swear I read an article that you only needed a phone and certain models of vehicle that were drive-by-wire to make this work.
Commaai/geohot did and that’s who this is. They caught a LOT of flack legally for being very cavalier about the whole thing.
Yes, this was open sourced and is currently used by Comma.ai. You can see a lot of info about the project at their website, comma.ai. They also sell some hardware all setup to run OpenPilot in many supported cars for about $1700. You do need to install it in your car, of course, but that’s not too hard. I’ve been using it for about 4 months. It’s great as a lane keeping/cruise control on the highway. It works on surface streets also. But it is most definitely NOT a self-driving system.
A regular phone probably wouldn’t be able to comply with the regulations for software that is relied upon for safety.
Nice idea, maybe good for farming or offroad use, but among pedestrians and other drivers? Is Android the first platform you think of when you think of safe software ready for critical environments?
I miss my Suzuki, may it rest in pieces.
I have been pondering how wise it is to roll my own, homebrew, an adaptive cruise system, because regular cruise control seems practically useless nowadays until you get out into the wide open spaces where it’s more use to avoid speeding tickets. Though 50% of the frustration comes from having to actually touch the brakes then reset it, where I don’t think I could trust it that far, just auto coasting if gap closing and resuming if clear.
It’s hacked together, like the Tesla autopilot too. Only thing, here, no one denies that this system is sketchy hacked together and can do bullshit in any time.
Don’t want to drive on the same road as one of this.