Neural Networks Control A Toy Car

After taking the Stanford Machine Learning class offered over the Internet last year, [David Singleton] thought he could build something really cool. We have to admit that he nailed it with his neural network controlled car. There’s not much to the build; it’s just an Android phone, an Arduino and a toy car. The machine learning part of this build really makes it special.

A neural network takes a whole bunch of inputs and represents them as a node in a network. Each node in [Davids]’s input layer corresponds to a pixel retrieved from his phone’s camera. All the inputs of the input layer are connected to 64 nodes in the ‘hidden layer’. The nodes in the hidden layer are connected to the four output nodes, namely left, right, forward and reverse.

After training the network and weighting all the connections, [David] got a toy car to drive around a track. Weird, but it works. All the code is up on github, so feel free to take a look behind the inner machinations of a neural net. Of course, you could check out the video of [David]’s car in action after the break.

EDIT: We originally credited [icebrain] as the author. Our bad, and we hope [David] doesn’t hate us now.



27 thoughts on “Neural Networks Control A Toy Car

    1. Most likely B and C.

      You could make it not look as obvious by having the car move forward a little while it thinks about the next move. You’d increase the chance of an accident, but it’s what people do while driving and talking on the cellphone.

    2. From the link:

      “You may also notice in the source that I chose to pulse the forward direction for 250 ms followed by a 500 ms pause – this was done simply because the car I used was very fast and difficult to drive round a small circuit – you might like to experiment with different values or remove this altogether if you try with a slower car.”

    1. The difference between a NN and a DAG, as I see it, is that there is no feedback (cycles) allowed in a NN. Data is only allowed to flow from inputs to the hidden layer (and possibly additional hidden layers) to the outputs.

      The reasons for this, I suspect, is primarily for stability. And also to ease the calculation of the neuron input weights and optional trigger levels, during the offline backpropagation training. e.g. These are the outputs I WANT this is the inputs I HAVE, calculate the weights and trigger levels. Having feedback would make this much much much much harder and take considerably longer to do.

      1. Actually, I am thinking of what are called “Feedforward neural network” (the only ones that I have ever used). There are <href= which are similar in some ways to DAG.

      2. the Directed and Acyclic in DAG means no feed back. Assuming you’re strictly adhering to the direction indicated on each edge, there should be no way when walking a DAG to get back to any of the nodes you visited previously.

  1. So, you like neural-network controlled RC cars, do you?

    take a look at a paper we had recently published; however to make it more like “human cognition”, we made it a FUZZY-Neural-network. Built using two Arduino Megas (and not a laptop, because that defeats the purpose of our paper) over 3 summers ago; all arduino code was created and compiled using the method described in my WikiHow on the subject:

    I’ll arrange to have everything put online(source code, etc.) if there is enough interest.

  2. “Once I got it working, I chose to remove the PCB from the original controller housing altogether and instead of powering it with 2 x AA batteries, I fed it 3.3V from the Arduino board (so all power for this unit comes over USB from the computer).”

    Careful because the maximum current limit of the FTDI chip (3.3V source) is only 50mA!

  3. He said the car was very fast so he pulsed the mechanical operation, why not just limit the power to the motor so it travels slower but continuously (by reducing motor voltage, inserting power resistor in series with the motor, you get the idea!)

      1. Also keeping the motor pulsing would allow the possibility for additional future outputs from the neural network “faster” (long straight) and “slower” (bends). But then there would be extra neurons required in the hidden layer.

  4. getting problem while driving the car in manual mode ,bcz while pressing the arrow key data is receivd in command promt but it is not transfered to the car and its not moving

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.