Autopilots Don’t Kill Drivers, Humans Do

The US National Highway Traffic Safety Administration (NHTSA) report on the May 2016 fatal accident in Florida involving a Tesla Model S in Autopilot mode just came out (PDF). The verdict? “the Automatic Emergency Braking (AEB) system did not provide any warning or automated braking for the collision event, and the driver took no braking, steering, or other actions to avoid the collision.” The accident was a result of the driver’s misuse of the technology.

quote-not-a-true-targetThis places no blame on Tesla because the system was simply not designed to handle obstacles travelling at 90 degrees to the car. Because the truck that the Tesla plowed into was sideways to the car, “the target image (side of a tractor trailer) … would not be a “true” target in the EyeQ3 vision system dataset.” Other situations that are outside of the scope of the current state of technology include cut-ins, cut-outs, and crossing path collisions. In short, the Tesla helps prevent rear-end collisions with the car in front of it, but has limited side vision. The driver should have known this.

The NHTSA report concludes that “Advanced Driver Assistance Systems … require the continual and full attention of the driver to monitor the traffic environment and be prepared to take action to avoid crashes.” The report also mentions the recent (post-Florida) additions to Tesla’s Autopilot that help make sure that the driver is in the loop.

The takeaway is that humans are still responsible for their own safety, and that “Autopilot” is more like anti-lock brakes than it is like Skynet. Our favorite footnote, in carefully couched legalese: “NHTSA recognizes that other jurisdictions have raised concerns about Tesla’s use of the name “Autopilot”. This issue is outside the scope of this investigation.” (The banner image is from this German YouTube video where a Tesla rep in the back seat tells the reporter that he can take his hands off the wheel. There may be mixed signals here.)

cropped_shot_2017-01-23-181745There are other details that make the report worth reading if, like us, you would like to see some more data about how self-driving cars actually perform on the road. On one hand, Tesla’s Autosteer function seems to have reduced the rate at which their cars got into crashes. On the other, increasing use of the driving assistance functions comes with an increase driver inattention for durations of three seconds or longer.

People simply think that the Autopilot should do more than it actually does. Per the report, this problem of “driver misuse in the context of semi-autonomous vehicles is an emerging issue.” Whether technology will improve fast enough to protect us from ourselves is an open question.

[via Popular Science].

Self-Driving Cars Are Not (Yet) Safe

Three things have happened in the last month that have made me think about the safety of self-driving cars a lot more. The US Department of Transportation (DOT) has issued its guidance on the safety of semi-autonomous and autonomous cars. At the same time, [Geohot]’s hacker self-driving car company bailed out of the business, citing regulatory hassles. And finally, Tesla’s Autopilot has killed its second passenger, this time in China.

At a time when [Elon Musk], [President Obama], and Google are all touting self-driving cars to be the solution to human error behind the wheel, it’s more than a little bold to be arguing the opposite case in public, but the numbers just don’t add up. Self-driving cars are probably not as safe as a good sober driver yet, but there just isn’t the required amount of data available to say this with much confidence. However, one certainly cannot say that they’re demonstrably safer.

Continue reading “Self-Driving Cars Are Not (Yet) Safe”

Geohot’s Self-Driving Code On GitHub

First there was [Geohot]’s lofty goal to build a hacker’s version of the self-driving car. Then came and a whole bunch of venture capital. After that, a letter from the Feds and a hasty retreat from the business end of things. The latest development?’s openpilot project shows up on GitHub!

If you’ve got either an Acura ILX or Honda Civic 2016 Touring addition, you can start to play around with this technology on your own. Is this a good idea? Are you willing to buy some time on a closed track?

A quick browse through the code gives some clues as to what’s going on here. The board files show just how easy it is to interface with these cars’ driving controls: there’s a bunch of CAN commands and that’s it. There’s some unintentional black comedy, like a (software) crash-handler routine named

What’s shocking is that there’s nothing shocking going on. It’s all pretty much straightforward Python with sprinklings of C. Honestly, it looks like something you could get into and start hacking away at pretty quickly. Anyone want to send us an Acura ILX for testing purposes? No promises you’ll get it back in one piece.

If you missed it, read up on our coverage of the rapid rise and faster retreat of But we don’t think the game is over yet: is still hiring. Are open source self-driving cars in our future? That would be fantastic!

Via Endagadget. Thanks for the tip, [FaultyWarrior]!

[Geohot]’s Self-Driving Car Cancelled

George [Geohot] Hotz has thrown in the towel on his “comma one” self-driving car project. According to [Geohot]’s Twitter stream, the reason is a letter from the US National Highway Traffic Safety Administration (NHTSA), which sent him what basically amounts to a warning to not release self-driving software that might endanger people’s lives.

This comes a week after a post on’s blog changed focus from a “self-driving car” to an “advanced driver assistance system”, presumably to get around legal requirements. Apparently, that wasn’t good enough for the NHTSA.

When Robot Cars Kill, Who Gets Sued?

20160530_165433On one hand, we’re sorry to see the system go out like that. The idea of a quick-and-dirty, affordable, crowdsourced driving aid speaks to our hacker heart. But on the other, especially in light of the recent Tesla crash, we’re probably a little bit glad to not have these things on the road. They were not (yet) rigorously tested, and were originally oversold in their capabilities, as last week’s change of focus demonstrated.’s downgrade to driver-assistance system really begs the Tesla question. Their autopilot is also just an “assistance” system, and the driver is supposed to retain full control of the car at all times. But we all know that it’s good enough that people, famously, let the car take over. And in one case, this has led to death.

Right now, Tesla is hiding behind the same fiction that the NHTSA didn’t buy with that an autopilot add-on won’t lull the driver into overconfidence. The deadly Tesla accident proved how that flimsy that fiction is. And so far, there’s only been one person injured by Tesla’s tech, and his family hasn’t sued. But we wouldn’t be willing to place bets against a jury concluding that Tesla’s marketing of the “autopilot” didn’t contribute to the accident. (We’re hackers, not lawyers.)

Should We Take a Step Back? Or a Leap Forward?

Stepping away from the law, is making people inattentive at the wheel, with a legal wink-and-a-nod that you’re not doing so, morally acceptable? When many states and countries will ban talking on a cell phone in the car, how is it legal to market a device that facilitates taking your hands off the steering wheel entirely? Or is this not all that much different from cruise control?

What Tesla is doing, and [Geohot] was proposing, puts a beta version of a driverless car on the road. On one hand, that’s absolutely what’s needed to push the technology forward. If you’re trying to train a neural network to drive, more data, under all sorts of conditions, is exactly what you need. Tesla uses this data to assess and improve its system all the time. Shutting them down would certainly set back the progress toward actually driverless cars. But is it fair to use the general public as opt-in Guinea pigs for their testing? And how fair is it for the NHTSA to discourage other companies from entering the field?

We’re at a very awkward adolescence of driverless car technology. And like our own adolescence, when we’re through it, it’s going to appear a miracle that we survived some of the stunts we pulled. But the metaphor breaks down with driverless cars — we can also simply wait until the systems are proven safe enough to take full control before we allow them on the streets. The current halfway state, where an autopilot system may lull the driver into a false sense of security, strikes me as particularly dangerous.

So how do we go forward? Do we let every small startup that wants to build a driverless car participate, in the hope that it gets us through the adolescent phase faster? Or do we clamp down on innovation, only letting the technology on the road once it’s proven to be safe? We’d love to hear your arguments in the comment section.

Self-Driving R/C Car Uses An Intel NUC

Self-driving cars are something we are continually told will be the Next Big Thing. It’s nothing new, we’ve seen several decades of periodic demonstrations of the technology as it has evolved. Now we have real prototype cars on real roads rather than test tracks, and though they are billion-dollar research vehicles from organisations with deep pockets and a long view it is starting to seem that this is a technology we have a real chance of seeing at a consumer level.

A self-driving car may seem as though it is beyond the abilities of a Hackaday reader, but while it might be difficult to produce safe collision avoidance of a full-sized car on public roads it’s certainly not impossible to produce something with a little more modest capabilities. [Jaimyn Mayer] and [Kendrick Tan] have done just that, creating a self-driving R/C car that can follow a complex road pattern without human intervention.

The NUC's-eye view. The green line is a human's steering, the blue line the computed steering.
The NUC’s-eye view. The green line is a human’s steering, the blue line the computed steering.

Unexpectedly they have eschewed the many ARM-based boards as the brains of the unit, instead going for an Intel NUC mini-PC powered by a Core i5 as the brains of the unit. It’s powered by a laptop battery bank, and takes input from a webcam. Direction and throttle can be computed by the NUC and sent to an Arduino which handles the car control. There is also a radio control channel allowing the car to be switched from autonomous to human controlled to emergency stop modes.

They go into detail on the polarizing and neutral density filters they used with their webcam, something that may make interesting reading for anyone interested in machine vision. All their code is open source, and can be found linked from their write-up. Meanwhile the video below the break shows their machine on their test circuit, completing it with varying levels of success.

Continue reading “Self-Driving R/C Car Uses An Intel NUC”

The Predictability Problem with Self-Driving Cars

A law professor and an engineering professor walk into a bar. What comes out is a nuanced article on a downside of autonomous cars, and how to deal with it. The short version of their paper: self-driving cars need to be more predictable to humans in order to coexist.

We share living space with a lot of machines. A good number of them are mobile and dangerous but under complete human control: the car, for instance. When we want to know what another car at an intersection is going to do, we think about the driver of the car, and maybe even make eye contact to see that they see us. We then think about what we’d do in their place, and the traffic situation gets negotiated accordingly.

When its self-driving car got into an accident in February, Google replied that “our test driver believed the bus was going to slow or stop to allow us to merge into the traffic, and that there would be sufficient space to do that.” Apparently, so did the car, right before it drove out in front of an oncoming bus. The bus driver didn’t expect the car to pull (slowly) into its lane, either.

All of the other self-driving car accidents to date have been the fault of other drivers, and the authors think this is telling. If you unexpectedly brake all the time, you can probably expect to eventually get hit from behind. If people can’t read your car’s AI’s mind, you’re gonna get your fender bent.

The paper’s solution is to make autonomous vehicles more predictable, and they mention a number of obvious solutions, from “I-sense-you” lights to inter-car communication. But then there are aspects we hadn’t thought about: specific markings that indicate the AIs capabilities, for instance. A cyclist signalling a left turn would really like to know if the car behind has the new bicyclist-handsignal-recognition upgrade before entering the lane. The ability to put your mind into the mind of the other car is crucial, and requires tons of information about the driver.

All of this may require and involve legislation. Intent and what all parties to an accident “should have known” are used in court to apportion blame in addition to the black-and-white of the law. When one of the parties is an AI, this gets murkier. How should you know what the algorithm should have been thinking? This is far from a solved problem, and it’s becoming more relevant.

We’ve written on the ethics of self-driving cars before, but simply in terms of their decision-making ability. This paper brings home the idea that we also need to be able to understand what they’re thinking, which is as much a human-interaction and legal problem as it is technological.

[Headline image: Google Self-Driving Car Project]

CES: Self-Flying Drone Cars

CES, the Consumer Electronics Show, is in full swing. Just for a second, let’s take a step back and assess the zeitgeist of the tech literati. Drones – or quadcopters, or UAVs, or UASes, whatever you call them – are huge. Self-driving cars are the next big thing. Flying cars have always been popular. On the technical side of things, batteries are getting really good, and China is slowly figuring out aerospace technologies. What could this possibly mean for CES? Self-flying drone cars.

The Ehang 184 is billed as the first autonomous drone that can carry a human. The idea is a flying version of the self-driving cars that are just over the horizon: hop in a whirring deathtrap, set your destination, and soar through the air above the plebs that just aren’t as special as you.

While the Ehang 184 sounds like a horrendously ill-conceived Indiegogo campaign, the company has released some specs for their self-flying drone car. It’s an octocopter, powered by eight 106kW brushless motors. Flight time is about 23 minutes, with a range of about 10 miles. The empty weight of the aircraft is 200 kg (440 lbs), with a maximum payload of 100 kg (220 lbs). This puts the MTOW of the Ehang 184 at 660 lbs, far below the 1,320 lbs cutoff for light sport aircraft as defined by the FAA, but far more than the definition of an ultralight – 254 lbs empty weight.

In any event, it’s a purely academic matter to consider how such a vehicle would be licensed by the FAA or any other civil aviation administration. It’s already illegal to test in the US, authorities haven’t really caught up to the idea of fixed-wing aircraft powered by batteries, and the idea of a legal autonomous aircraft carrying a passenger is ludicrous.

Is the Ehang 184 a real product? There is no price, and no conceivable way any government would allow an autonomous aircraft fly with someone inside it. It is, however, a perfect embodiment of the insanity of CES.