Catastrophic Forgetting: Learning’s Effect on Machine Minds

What if every time you learned something new, you forgot a little of what you knew before? That sort of overwriting doesn’t happen in the human brain, but it does in artificial neural networks. It’s appropriately called catastrophic forgetting. So why are neural networks so successful despite this? How does this affect the future of things like self-driving cars? Just what limit does this put on what neural networks will be able to do, and what’s being done about it?

The way a neural network stores knowledge is by setting the values of weights (the lines in between the neurons in the diagram). That’s what those lines literally are, just numbers assigned to pairs of neurons. They’re analogous to the axons in our brain, the long tendrils that reach out from one neuron to the dendrites of another neuron, where they meet at microscopic gaps called synapses. The value of the weight between two artificial neurons is roughly like the number of axons between biological neurons in the brain.

To understand the problem, and the solutions below, you need to know a little more detail.

Continue reading “Catastrophic Forgetting: Learning’s Effect on Machine Minds”

Go Small, Get Big: The Hack that Revolutionized Bioscience

Few people outside the field know just how big bioscience can get. The public tends to think of fields like physics and astronomy, with their huge particle accelerators and massive telescopes, as the natural expressions of big science. But for decades, biology has been getting bigger, especially in the pharmaceutical industry. Specialized labs built around the automation equipment that enables modern pharmaceutical research would dazzle even the most jaded CERN physicist, with fleets of robot arms moving labware around in an attempt to find the Next Big Drug.

I’ve written before on big biology and how to get more visibility for the field into STEM programs. But how exactly did biology get big? What enabled biology to grow beyond a rack of test tubes to the point where experiments with millions of test occasions are not only possible but practically required? Was it advances in robots, or better detection methodologies? Perhaps it was a breakthrough in genetic engineering?

Nope. Believe it or not, it was a small block of plastic with some holes drilled in it. This is the story of how the microtiter plate allowed bioscience experiments to be miniaturized to the point where hundreds or thousands of tests can be done at a time.

Continue reading “Go Small, Get Big: The Hack that Revolutionized Bioscience”

You’re the Only One not Playing with Unity

It wasn’t too long ago that one could conjecture that most hackers are not avid video game players. We spend most of our free time taking things apart, tinkering with microcontrollers and reading the latest [Jenny List] article on Hackaday.com. When we do think of video games, our neurons generally fire in the direction of emulating a console on a single board computer, such as a Raspberry Pi or a Beaglebone. Or even emulating the actual console processor on an FPGA. Rarely do we venture off into 3D programs meant to make modern video games. If we can’t export an .STL with it, we’re not interested. It’s just not our bag.

Oculus Rift changed this. The VR headset was originally invented for 3D video games, but quickly became a darling to hackers the world over. Virtual Reality technology is far bigger than just video games, and brings opportunity to many fields such as real estate, construction, product visualization, education, social interaction… the list goes on and on.

The Oculus team got together with the folks over at Unity in the early days to make it easy for video game makers to make content for the Rift. Unity is a game engine designed with a shallow learning curve and is available for free for non-commercial use. The Oculus Rift can be integrated into a Unity environment with the check of a setting and importing a small package, available on the Oculus site. This makes it easy for anyone interested in VR technology to get a Rift and start pumping out content.

Hackers have taken things a step further and have written scripts that allow Unity to communicate with an Arduino. VR is fun. But VR plus physical reality is just down right exciting! In this article, we’re going to walk you through setting up your Oculus Rift and Unity game engine to communicate with the outside world via an Arduino.

Continue reading “You’re the Only One not Playing with Unity”

Practical IoT Cryptography on the Espressif ESP8266

The Espressif ESP8266 chipset makes three-dollar ‘Internet of Things’ development boards an economic reality. According to the popular automatic firmware-building site nodeMCU-builds, in the last 60 days there have been 13,341 custom firmware builds for that platform. Of those, only 19% have SSL support, and 10% include the cryptography module.

We’re often critical of the lack of security in the IoT sector, and frequently cover botnets and other attacks, but will we hold our projects to the same standards we demand? Will we stop at identifying the problem, or can we be part of the solution?

This article will focus on applying AES encryption and hash authorization functions to the MQTT protocol using the popular ESP8266 chip running NodeMCU firmware. Our purpose is not to provide a copy/paste panacea, but to go through the process step by step, identifying challenges and solutions along the way. The result is a system that’s end-to-end encrypted and authenticated, preventing eavesdropping along the way, and spoofing of valid data, without relying on SSL.

We’re aware that there are also more powerful platforms that can easily support SSL (e.g. Raspberry Pi, Orange Pi, FriendlyARM), but let’s start with the cheapest hardware most of us have lying around, and a protocol suitable for many of our projects. AES is something you could implement on an AVR if you needed to.

Continue reading “Practical IoT Cryptography on the Espressif ESP8266”

The Arduino Foundation: What’s Up?

The Arduino Wars officially ended last October, and the new Arduino-manufacturing company was registered in January 2017.  At the time, we were promised an Arduino Foundation that would care for the open-source IDE and code infrastructure in an open and community-serving manner, but we don’t have one yet. Is it conspiracy? Or foul play? Our advice: don’t fret. These things take time.

But on the other hand, the Arduino community wants to know what’s going on, and there’s apparently some real confusion out there about the state of play in Arduino-land, so we interviewed the principals, Massimo Banzi and Federico Musto, and asked them for a progress report.

The short version is that there are still two “Arduinos”: Arduino AG, a for-profit corporation, and the soon-to-be Arduino Foundation, a non-profit in charge of guiding and funding software and IDE development. The former was incorporated in January 2017, and the latter is still in progress but looks likely to incorporate before the summer is over.

Banzi, who is a shareholder of Arduino AG, is going to be the president of the Foundation, and Musto, AG’s CEO, is going to be on the executive board and both principals told us similar visions of incredible transparency and community-driven development. Banzi is, in fact, looking to get a draft version of the Foundation’s charter early, for comment by the community, before it gets chiseled in stone.

It’s far too early to tell just how independent the Foundation is going to be, or should be, of the company that sells the boards under the same name. Setting up the Foundation correctly is extremely important for the future of Arduino, and Banzi said to us in an interview that he wouldn’t take on the job of president unless it is done right. What the Arduino community doesn’t need right now is a Foundation fork.  Instead, they need our help, encouragement, and participation once the Foundation is established. Things look like they’re on track.

Continue reading “The Arduino Foundation: What’s Up?”

Megabots, Colliders, Rockets, Tunnels Underground, and Other Big Dumb Ideas Will Save Us

Humanity is a planetwide force. We have the power to change our weather. We have the power to change the shape of the land. We have the power to selectively wipe a species from this earth if we choose.  We’ve had this power for a while and we’re still coming to terms with it. Many of us even deny it.

With such power, what do we do? We have very few projects which are in line with our ability. Somewhere in the past few years, I feel like most of us have lost our audacity. We’ve culturally come to appreciate the safe bet too much. We pull the dreamers and doers down. We want to solve the small problems first, and see if we have time for the big problems later. We don’t dream big enough, and there is zero reason for this hesitation. We could leverage our planetwide power for planetwide improvements. Nothing is truly stopping us. No law, no government, nothing.

To put it simply, as far as technology goes, everything is still low-hanging fruit. We’ve barely done anything. Even some of our greatest accomplishments can happen randomly in nature. We’ve not left our planet in any numbers or for any length of time. Our cities are disorganized messes. In every single field today, the unexplored territory is orders larger than the explored. Yet despite this vast territory, there are very few explorers. People want to optimize the minutia of life. A slightly faster processor for a slightly smaller phone. It’s okay.

Yet that same small optimization applied to a larger effort could have vast positive impact. Those same microprocessors could catalog our planet or drive probes into space. The very same efforts we spend on micro upgrades could be leveraged if we just look at the bigger picture then get out of our own way. All that is lacking is ambition. Money, time, skill, industry, and people are all there, waiting. We have the need for and have the resources to support ten thousand Elon Musks, not just the one.

Big projects make us bigger than our cellphones and Facebook. When you see a rocket launch into the sky, suddenly, “the world” becomes, simply, “a world.” Order of magnitude improvements reduce the order of our perception of previously complex problems. They should be our highest goal. Whatever field you’re in, you should be trying to be ten times better than the top competitor.

However, there are some societal changes that have to occur before we can.

Continue reading “Megabots, Colliders, Rockets, Tunnels Underground, and Other Big Dumb Ideas Will Save Us”

Imaginary AC Circuits Aren’t Really Complex

If you have ever read advanced textbooks or papers about electronics, you may have been surprised to see the use of complex numbers used in the analysis of AC circuits. A complex number has two parts: a real part and an imaginary part. I’ve often thought that a lot of books and classes just kind of gloss over what this really means. What part of electricity is imaginary? Why do we do this?

The short answer is phase angle: the time delay between a voltage and a current in a circuit. How can an angle be a time? That’s part of what I’ll need to explain.

First, consider a resistor. If you apply a voltage to it, a certain current will flow that you can determine by Ohm’s law. If you know the instantaneous voltage across the resistor, you can derive the current and you can find the power–how much work that electricity will do. That’s fine for DC current through resistors. But components like capacitors and inductors with an AC current don’t obey Ohm’s law. Take a capacitor. Current only flows when  the capacitor is charging or discharging, so the current through it relates to the rate of change of the voltage, not the instantaneous voltage level.

That means that if you plot the sine wave voltage against the current, the peak of the voltage will be where the current is minimal, and the peak current will be where the voltage is at zero. You can see that in this image, where the yellow wave is voltage (V) and the green wave is current (I). See how the green peak is where the yellow curve crosses zero? And the yellow peak is where the green curve crosses zero?

These linked sine and cosine waves might remind you of something — the X and Y coordinates of a point being swept around a circle at a constant rate, and that’s our connection to complex numbers. By the end of the post, you’ll see it isn’t all that complicated and the “imaginary” quantity isn’t imaginary at all.

Continue reading “Imaginary AC Circuits Aren’t Really Complex”