Neural networks control a toy car

After taking the Stanford Machine Learning class offered over the Internet last year, [David Singleton] thought he could build something really cool. We have to admit that he nailed it with his neural network controlled car. There’s not much to the build; it’s just an Android phone, an Arduino and a toy car. The machine learning part of this build really makes it special.

A neural network takes a whole bunch of inputs and represents them as a node in a network. Each node in [Davids]‘s input layer corresponds to a pixel retrieved from his phone’s camera. All the inputs of the input layer are connected to 64 nodes in the ‘hidden layer’. The nodes in the hidden layer are connected to the four output nodes, namely left, right, forward and reverse.

After training the network and weighting all the connections, [David] got a toy car to drive around a track. Weird, but it works. All the code is up on github, so feel free to take a look behind the inner machinations of a neural net. Of course, you could check out the video of [David]‘s car in action after the break.

EDIT: We originally credited [icebrain] as the author. Our bad, and we hope [David] doesn’t hate us now.


  1. Philippe says:

    Neural maybe?

  2. icebrain says:

    Sorry, but as I pointed out, I just stumbled upon it on Hacker News, I’m not the author ;) All the credits go to David Singleton (dps on Github).

  3. jAMES says:

    That is really awesome. I wonder how long it took to train?

  4. randomdude says:

    awesome… but why does it stop every half a second or so ??

    is it because of:
    A)framerate of the camera
    B)computing requirements
    C)the way it was trained

    • ChalkBored says:

      Most likely B and C.

      You could make it not look as obvious by having the car move forward a little while it thinks about the next move. You’d increase the chance of an accident, but it’s what people do while driving and talking on the cellphone.

    • tehDorf says:

      From the link:

      “You may also notice in the source that I chose to pulse the forward direction for 250 ms followed by a 500 ms pause – this was done simply because the car I used was very fast and difficult to drive round a small circuit – you might like to experiment with different values or remove this altogether if you try with a slower car.”

  5. Kyle says:

    So can someone explain to me the difference between a neural network and a graph? Is it that a neural network has a pointer to 64 other nodes?

    • Glen says:

      A neural network is based around a Directed Acyclic Graph with some extra guff for determining whether the strength of all of a nodes input is sufficient for that node to fire a signal itself.

    • Truth says:

      The difference between a NN and a DAG, as I see it, is that there is no feedback (cycles) allowed in a NN. Data is only allowed to flow from inputs to the hidden layer (and possibly additional hidden layers) to the outputs.

      The reasons for this, I suspect, is primarily for stability. And also to ease the calculation of the neuron input weights and optional trigger levels, during the offline backpropagation training. e.g. These are the outputs I WANT this is the inputs I HAVE, calculate the weights and trigger levels. Having feedback would make this much much much much harder and take considerably longer to do.

      • Truth says:

        Actually, I am thinking of what are called “Feedforward neural network” (the only ones that I have ever used). There are <href= which are similar in some ways to DAG.

      • Glen says:

        the Directed and Acyclic in DAG means no feed back. Assuming you’re strictly adhering to the direction indicated on each edge, there should be no way when walking a DAG to get back to any of the nodes you visited previously.

  6. J. Peterson says:

    Rebuilding the “track” on the fly…reminds me of the wonderful scene in The Wrong Trousers where Grommit steers the toy train he’s riding on by laying down the track in front of it on the fly.

  7. Jeff G says:

    So, you like neural-network controlled RC cars, do you?

    take a look at a paper we had recently published; however to make it more like “human cognition”, we made it a FUZZY-Neural-network. Built using two Arduino Megas (and not a laptop, because that defeats the purpose of our paper) over 3 summers ago; all arduino code was created and compiled using the method described in my WikiHow on the subject:

    I’ll arrange to have everything put online(source code, etc.) if there is enough interest.

  8. MrX says:

    “Once I got it working, I chose to remove the PCB from the original controller housing altogether and instead of powering it with 2 x AA batteries, I fed it 3.3V from the Arduino board (so all power for this unit comes over USB from the computer).”

    Careful because the maximum current limit of the FTDI chip (3.3V source) is only 50mA!

  9. MrX says:

    Correction, looks like he is using a Arduino Uno, so the source is not the FTDI chip but a onboard regulator. Still, the max current draw is still 50mA.

  10. Mike Edwards says:

    He said the car was very fast so he pulsed the mechanical operation, why not just limit the power to the motor so it travels slower but continuously (by reducing motor voltage, inserting power resistor in series with the motor, you get the idea!)

    • Truth says:

      That Resistor is going to convert some power into heat. Pulsing it is probably way more efficient – the battery will last longer.

      • Truth says:

        Also keeping the motor pulsing would allow the possibility for additional future outputs from the neural network “faster” (long straight) and “slower” (bends). But then there would be extra neurons required in the hidden layer.

  11. dfdfsd says:

    good job

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Get every new post delivered to your Inbox.

Join 96,671 other followers