Self-Learning Helicopter Uses Neural Network

Though this project uses an RC helicopter, it’s merely a vessel to demonstrate a fascinating machine learning algorithm developed by two Cornell students – [Akshay] and [Sergio]. The learning environment is set up with the helicopter at its center, attached to a boom. The boom restricts the helicopter’s movement down to one degree of motion, so that it can only move up from the ground (not side to side or front to back).

The goal is for the helicopter to teach itself how to get to a specific height in the quickest amount of time. A handful of IR sensors are used to tell the Atmega644 how high the helicopter is. The genius of this though, is in the firmware. [Akshay] and [Sergio] are using an evolutionary algorithm adopted from Floreano et al, a noted author on biological inspired artificial intelligences. The idea is for the helicopter to create random “runs” and then check the data. The runs that are closer to the goal get refined while the others are eliminated, thus mimicking evolutions’ natural selection.

We’ve seen neural networks before, but nothing like this. Stay with us after the break, as we take this awesome project and narrow it down so that you too can implement this type of algorithm in your next project.

Consider the image above. The goal is for the helicopter to start at Point A, go to Point C and hover. Allotted time is 10 seconds per run. It has to teach itself how to do this and do it as quickly as possible. Remember, it knows where these points are via IR sensors.  [Akshay] and [Sergio] developed an equation using a piecewise function to determine which runs were closest to Point C for the longest amount of time.

Each of the points in the above equation is known via a voltage from the IR sensors, with Point A being 0.1 volts and Point D being 3.7 volts.  The equation is designed to give the greatest value for the longest time spent at Point C. This value is known as a Fitness Value.

A neural network is used to determine at what level the throttle should be at to achieve the highest Fitness Value. This network is apart the Evolutionary Algorithm that runs in the firmware. Basically, it starts off with random values that generate random levels of throttle. The values that achieve the highest Fitness Value get ‘mutated’, while the others are discarded.

The mutations in the values are done at random, and the process repeats. In the end, the firmware learns the best throttle levels to achieve the goal of being at Point C for the longest time in the allotted 10 seconds.

Be sure to check out this linked project for full details on these mutations are carried out in the source.

1. BohemianHacks says:

Seems like a pretty standard evolutionary algorithm, very cool application but I’m not seeing anything too groundbreaking here.

• Blue Footed Booby says:

Uhhhhh did I miss somewhere the creator claimed it to be groundbreaking? This isn’t a peer reviewed science journal; “has literally anyone done it before” isn’t the bar for postworthyness.

• BohemianHacks says:

“…fascinating machine learning algorithm developed by two Cornell students” The word groundbreaking wasn’t used directly and perhaps I misread the hype in that sentence. I also didn’t say it wasn’t post worthy. I even said it was a very cool application for an evolutionary algo, but I’m not seeing anything super new here as far as that algorithm goes. Maybe the cool bit is in the fitness function but thats not too suprising as you usually need to create something pretty specific anyways.

• cooperised says:

I’m with @BohemianHacks here. The text did kind of imply that it was novel, which it’s not (at least not in any significant way). It is still cool, but it’s not novel.

• Sam Archer says:

I’ve certainly never seen a machine learning algorithm developed for getting an RC helicopter attached to a stick to get to a specified altitude as quickly as possible before… I’d say that *does* make it novel.

Also: what a bunch of negative Nancies.

• airmansnuffy says:

No post here is complete without a bunch of comments grumbling about the project.

• efahrenholz says:

Many folks here aren’t happy unless the post contains scientific journal sources. This site isn’t aimed at scholars, yet they love to come here and alienate those who don’t ‘hack’ these things for a living.

• Brian Booth says:

Ng used reinforcement learning to algorithmicly operate a helicopter and perform tricks by watching an experienced pilot. This was over 4 years ago. http://heli.stanford.edu

2. krb686 says:

This is cool stuff, though I’m not really sure why it is labeled evolutionary when it’s really just adaptive.

3. cr0sh says:

It’s almost like the 1990s didn’t happen, the CAM-Brain machine wasn’t a thing, and Hugo de Garis didn’t exist…

Honestly, I am not sure what the neural network is for, other than maybe a learning opportunity (nothing wrong with that). In a system this simple, a simple GA implementation should suffice.

• The answer is on the first sentence: Though this project uses an RC helicopter, it’s merely a vessel to demonstrate a fascinating machine learning algorithm developed by two Cornell students

So, the whole point is actually implemmenting a neural network. It is NOT about how to fly a helicopter.

• cr0sh says:

Here’s the thing though – for such a simple exercise, you don’t need both a neural network and a GA; either one could do the job perfectly. One could argue that using both merely increases the complexity for little to no extra functionality.

Usually, one uses a GA to evolve a neural network for the purpose of avoiding local minima issues, but that is typically only a problem with more complex NN implementations.

I can definitely appreciate it from an educational standpoint; perhaps this simple study will allow them to go on to apply similar ideas in areas where it will be an optimal solution…

4. Evan says:

I like it! The \$50 price point is nice,and you got some usable data from it, although having to tweak the constraints so the boom doesn’t break kinda renders the data inaccurate.

Try posting an ad to craigslist for broken carbon fiber arrows. I bet you anything that Ithaca has a ton of people who’ll give you them for free.

If you can rig up a carbon fiber boom, it’ll blow away that balsa in terms of both yield strength and rigidity.

5. sfrg says:

this is not a neural network, but a evolutionary (genetic) algorithm that works by selection.

• somun says:

They actually seem to be using EA to optimize their neural network. But their writing is quite convoluted.

6. opless says:

Um… first link in the article.

7. squark says:

Wouldn’t a PID algorithm be a vastly superior method of achieving the correct height in the fastest time? Using the above work to calculate the gains seems like a better and more general idea in terms of application

8. Verti Pilot says:

Bummed that nobody mentioned the Verti Bird.

9. Something seriously wrong with people that think it’s OK to do this to C code:

```#define begin {
#define end }
```

Followed by…

```ISR (TIMER0_COMPA_vect)
begin
if (pulseFlag==1)
begin
if (time1>0)
begin
PORTB^=0x08; // toggle B.3 at each count
end
```

My eyes are bleeding.

• Le Samourai says:

Hah! That gave me a good chuckle. How painful it must be to code that way.

My first language was C(++) so i’m not sure where ‘begin’ and ‘end’ came from… did these guys learn like… BASIC as their first?

• Dat Le says:

Or PASCAL or pseudo code. I’ve seen many people do it that way.