Self-Learning Helicopter Uses Neural Network

Though this project uses an RC helicopter, it’s merely a vessel to demonstrate a fascinating machine learning algorithm developed by two Cornell students – [Akshay] and [Sergio]. The learning environment is set up with the helicopter at its center, attached to a boom. The boom restricts the helicopter’s movement down to one degree of motion, so that it can only move up from the ground (not side to side or front to back).

The goal is for the helicopter to teach itself how to get to a specific height in the quickest amount of time. A handful of IR sensors are used to tell the Atmega644 how high the helicopter is. The genius of this though, is in the firmware. [Akshay] and [Sergio] are using an evolutionary algorithm adopted from Floreano et al, a noted author on biological inspired artificial intelligences. The idea is for the helicopter to create random “runs” and then check the data. The runs that are closer to the goal get refined while the others are eliminated, thus mimicking evolutions’ natural selection.

We’ve seen neural networks before, but nothing like this. Stay with us after the break, as we take this awesome project and narrow it down so that you too can implement this type of algorithm in your next project.

Evolutionary algorithms computes the best blackjack strategy

Don’t want to learn about evolutionary algorithms the usual way, by generating sentences from random letters, or randomly placing pixels to generate the Mona Lisa? Then make your own evolutionary algorithm! With blackjack!

[Brian] has been playing around with evolutionary algorithms, and wanted a task that’s well suited for optimization. He chose blackjack, because of the limited number of hands that can be dealt to the player (32) and low number of hands the dealer can have (10).

Even with the low number of initial conditions for the player and the dealer, there are still 4.562 x 10^192 possible combinations of hands, so brute forcing a blackjack strategy would require the computational power of the entire planet. An easier way to compute a good strategy is an evolutionary algorithm, implemented by [Brian] with the Watchmaker Java library.

For each generation in [Brian]‘s program, a 32×10 grid was generated, one cell each for possible player’s hands against the dealer’s hand. In each cell, the computer put a ‘hit’, ‘stay’, or ‘double down’, and played thousands of hands with that strategy. The best strategies were bred and eventually [Brian] ended up with a good blackjack strategy.

The resulting best strategy is pretty good – using his strategy, he can walk out of an Atlantic City casino with 96% of the money he arrived with.

On not designing circuits with evolutionary algorithms

[Henrik] has been working on a program to design electronic circuits using evolutionary algorithms. It’s still very much a work in progress, but he’s gotten to the point of generating a decent BJT inverter after 78 generations (9 minutes of compute time), as shown in the .gif above.

To evolve these circuits, [Henrik] told a SPICE simulation to generate an inverter with a 5V power supply, 2N3904 and 2N3906 transistors, and whatever resistors were needed. The first dozen or so generations didn’t actually do anything, but after 2000 generations the algorithm produced a circuit nearly identical to the description of a CMOS inverter you’d find in a circuit textbook.

Using evolution to guide electronic design is nothing new; an evolutionary algorithm and a a few bits of Verilog can turn an FPGA into a chip that can tell the difference between a 1kHz and 10kHz tone with extremely minimal hardware requirements. There’s also some very, very strange stuff that happened in this experiment; the evolutionary algorithm utilized things that are impossible for a human to program and relies on magnetic flux and quantum weirdness inside the FPGA.

[Henrik] says his algorithm didn’t test for how much current goes through the transistors, so implementing this circuit outside of a simulation will destroy the transistors and emit a puff of blue smoke. If you’d like design your own circuits using evolution, [Henrik] put all the code in a git for your perusal. It’s damn cool as it stands now, and once [Henrik] includes checking current and voltage in each component his project may actually be useful.