Last time, I talked about a simple kind of neural net called a perceptron that you can cause to learn simple functions. For the purposes of experimenting, I coded a simple example using Excel. That’s handy for changing things on the fly, but not so handy for putting the code in a microcontroller. This time, I’ll show you how the code looks in C++ and also tell you more about what you can do when faced with a more complex problem.
When you want a person to do something, you train them. When you want a computer to do something, you program it. However, there are ways to make computers learn, at least in some situations. One technique that makes this possible is the perceptron learning algorithm. A perceptron is a computer simulation of a nerve, and there are various ways to change the perceptron’s behavior based on either example data or a method to determine how good (or bad) some outcome is.
I’m no biologist, but apparently a neuron has a bunch of inputs and if the level of those inputs gets to a certain level, the neuron “fires” which means it stimulates the input of another neuron further down the line. Not all inputs are created equally: in the mathematical model of them, they have different weighting. Input A might be on a hair trigger, while it might take inputs B and C on together to wake up the neuron in question.
Continue reading “Machine Learning: Foundations”
It’s overkill, but it’s really cool. [Bob Bond] took an NVIDIA Jetson TX1 single-board computer and a webcam and wirelessly combined them with his lawn sprinklers. Now, when his neighbors’ cats come to poop in his yard, a carefully trained neural network detects them and gets them wet.
It is absolutely the case that this could have been done with a simple motion sensor, but if the neural network discriminates sufficiently well between cats and (for instance) his wife, this is an improved solution for sure. Because the single-board computer he’s chosen for the project has a ridiculous amount of horsepower, he can afford to do a lot of image processing, so there’s a chance that everyone on two legs will stay dry. And the code is up on GitHub for you to see, if you’re interested.
[Bob] promises more detail about the neural network in the future. We can’t wait. (And we’d love to see a sentry-turret style build in the future. Think of the water savings!)
Via the NVIDIA blog, and thanks [Jaqen] for the tip!
We wake up this morning to the news that Google’s deep-search neural network project called AlphaGo has beaten the second ranked world Go master (who happens to be a human being). This is the first of five matches between the two adversaries that will play out this week.
On one hand, this is a sign of maturing technology. It has been almost twenty years since Deep Blue beat Gary Kasparov, the reigning chess world champion at the time. Although there are still four games to play against Lee Sedol, it was recently reported that AlphaGo beat European Go champion Fan Hui in five games straight. Go is generally considered a more difficult game for machine minds to play than chess. This is because Go has a much larger pool of possible moves at any given time.
Okay, the news part of this event has been covered: machine beats man. Does it matter? Will this affect your life and how? We want to hear what you think in the comments below. But I’m going to keep going with some of my thoughts on the topic.
![You're still better at Ms. Pacman [Source: DeepMind paper in Nature]](https://hackaday.com/wp-content/uploads/2016/03/deepmind-atari2600-machine-learning.jpg)
DeepMind, the company behind AlphaGo which was acquired by Google in 2014, has published a paper in Nature about their approach. They were even nice enough to let us read without dealing with a paywall. The secret sauce is the learning process which at its core tries to mimic how living entities learn: observe repetitively while assigning values to outcomes. This is key as it leads past “intellect”, to “intelligence” (the “I” in AI that everyone seems to be waiting for). But this is a bastardized version of “intelligence”. AlphaGo is able to recognize and predict behavior, then make choices that lead to a desired outcome. This is more than intellect as it does value the purpose of an opponent’s decisions. But it falls short of intelligence as AlphaGo doesn’t consciously understand the purpose it has detected. In my mind this is exactly what we need. Truly successful machine learning will be able to make sense out of sometimes irrational input.
The paper from Nature doesn’t go into details about Go, but it explains the approach of the learning system applied to Atari 2600. The algorithm was given 210×160 color video at 60Hz as an input and then told it could use a joystick with one button. From there it taught itself to play 49 games. It was not told the purpose or the rules of the games, but it was given examples of scores from human performance and rewarded for its own quality performances. The chart above shows that it learned to play 29 of them at or above human skill levels.
Continue reading “Ask Hackaday: Google Beat Go; Bellwether Or Hype?”
Marvin Minsky, one of the early pioneers of neural networks, died on Sunday at the age of 88.
The obituary in the Washington Post paints a fantastic picture of his life. Minsky was friends with Richard Feynman, Isaac Asimov, Arthur C. Clarke, and Stanley Kubrick. He studied under Claude Shannon, worked with Alan Turing, had frequent conversations with John Von Neumann, and had lunch with Albert Einstein.
Minsky’s big ideas were really big. He built one of the first artificial neural networks, but was aiming higher — toward machines that could actually think rather than simply classify data. This was one of the driving forces behind his book, Perceptrons, that showed some of the limitations in the type of neural networks (single-layer, feedforward) that were being used at the time. He wanted something more.
Minsky’s book The Society of Mind is interesting because it reframes the problem of human thought from being a single top-down process to being a collaboration between many different brain regions, the nervous system, and indeed the body as a whole. This “connectionist” theme would become influential both in cognitive science and in robotics.
In short, Minksy was convinced that complex problems often had necessarily complex solutions. In research projects, he was in for the long-term, and encouraged a bottom-up design procedure where many smaller elements combined into a complicated whole. “The secret of what something means lies in how it connects to other things we know. That’s why it’s almost always wrong to seek the “real meaning” of anything. A thing with just one meaning has scarcely any meaning at all.”
Minsky was a very deep thinker, but he kept grounded by also being a playful inventor. Minsky is credited with inventing the “ultimate machine” which would pop up in modern geek culture and shared numerous times on Hackaday as the “most useless machine”. He inspired Claude Shannon to build one. Arthur C. Clarke said, “There is something unspeakably sinister about a machine that does nothing — absolutely nothing — except switch itself off.”
He also co-designed the Triadex Muse, which was an early synthesizer and sequencer and “automatic composer” that creates fairly complex and original patterns with minimal input. It’s an obvious offshoot of his explorations in artificial intelligence, and on our bucket list of must-play-with electronic instruments.
Minsky’s web site at MIT has a number of his essays, and the full text of “The Society of Mind”, all available for your reading pleasure. It’s worth a bit of your time, not just in memoriam of a great thinker and a wacky inventor, but also because we bet you’ll see the world a little bit differently afterwards. That’s a legacy that lasts.
Sir Winston Churchill often spoke of World War 2 as the “Wizard War”. Both the Allies and Axis powers were in a race to gain the electronic advantage over each other on the battlefield. Many technologies were born during this time – one of them being the ability to decipher coded messages. The devices that were able to achieve this feat were the precursors to the modern computer. In 1946, the US Military developed the ENIAC, or Electronic Numerical Integrator And Computer. Using over 17,000 vacuum tubes, the ENIAC was a few orders of magnitude faster than all previous electro-mechanical computers. The part that excited many scientists, however, was that it was programmable. It was the notion of a programmable computer that would give rise to the 
idea of artificial intelligence (AI).
As time marched forward, computers became smaller and faster. The invention of the transistor semiconductor gave rise to the microprocessor, which accelerated the development of computer programming. AI began to pick up steam, and pundits began to make grand claims of how computer intelligence would soon surpass our own. Programs like ELIZA and Blocks World fascinated the public and certainly gave the perception that when computers became faster, as they surely would in the future, they would be able to think like humans do.
But it soon became clear that this would not be the case. While these and many other AI programs were good at what they did, neither they, or their algorithms were adaptable. They were ‘smart’ at their particular task, and could even be considered intelligent judging from their behavior, but they had no understanding of the task, and didn’t hold a candle to the intellectual capabilities of even a typical lab rat, let alone a human.
Continue reading “A Short History Of AI, And Why It’s Heading In The Wrong Direction”
A lot of computers can play chess. [Matthew Lui’s] Giraffe is a chess playing computer, but unlike other common chess programs, Giraffe taught itself to play. It apparently learned pretty well, too, since it is rated as an International Master on the FIDE scale (putting it in the top 2.2% of players. The top chess playing computers clock in at super grandmaster level but they are not self-taught).
