What if every time you learned something new, you forgot a little of what you knew before? That sort of overwriting doesn’t happen in the human brain, but it does in artificial neural networks. It’s appropriately called catastrophic forgetting. So why are neural networks so successful despite this? How does this affect the future of things like self-driving cars? Just what limit does this put on what neural networks will be able to do, and what’s being done about it?
Numerical weights in an artificial neural network
Neurons in the brain
The way a neural network stores knowledge is by setting the values of weights (the lines in between the neurons in the diagram). That’s what those lines literally are, just numbers assigned to pairs of neurons. They’re analogous to the axons in our brain, the long tendrils that reach out from one neuron to the dendrites of another neuron, where they meet at microscopic gaps called synapses. The value of the weight between two artificial neurons is roughly like the number of axons between biological neurons in the brain.
To understand the problem, and the solutions below, you need to know a little more detail.
Two researchers of Responsive Environments, MIT Media Lab, have put to together a device that is an amazing array of musical instruments squeezed into one flexible package. Made using seven layers of fabrics with different electrical properties, the result can be played using touch, proximity, pressure, stretch, or with combinations of them. Using a fabric-based keyboard, ribbon-controller, and trackpad, it can be played as a one-octave keyboard, a theremin, and in ways that have no words, such as stretching while pressing keys. It can also be folded up and stuffed into a case along with your laptop, and care has even been taken to make it washable.
Layer one, the top layer, is a conductive fabric for detecting proximity and touch. The twelve keys can work independently with a MPR121 proximity touch controller or the controller can treat them all as one, extending the distance the hand can be and have it still work. Layer two is just a knit fabric but layers three to six detect pressure, consisting to two conductive layers with a mesh fabric and a piezo-resistive fabric in between. The piezo-resistive fabric is LTT-SPLA from eeonyx, a knit fabric coated with the conductive polymer, polypyrrole (PPy). Layer seven consists of two strips of knitted spandex fabric, also coated with PPy, and detects stretching. Two strips of this are sewn on the bottom, one horizontal and one vertical. You can see and hear the amazing sound this all produces in the video below.
As a fun project I thought I’d put Google’s Inception-v3 neural network on a Raspberry Pi to see how well it does at recognizing objects first hand. It turned out to be not only fun to implement, but also the way I’d implemented it ended up making for loads of fun for everyone I showed it to, mostly folks at hackerspaces and such gatherings. And yes, some of it bordering on pornographic — cheeky hackers.
An added bonus many pointed out is that, once installed, no internet access is required. This is state-of-the-art, standalone object recognition with no big brother knowing what you’ve been up to, unlike with that nosey Alexa.
[Cupcakus]’s mini Apple IIe must surely be a contender for the smallest computer running an Apple II emulator. We’d mentioned it a few months ago in a Links post when it had been posted to a forum along with a few videos of it in action, but now popular YouTube channel, [Tested], has released a video wherein they not only show what’s inside, and interview [Cupcakus] about his trials and tribulations in making it, but also go through the steps of making one of their own. Also, at the time of writing the Links post, [Cupcakus] hadn’t yet announced his detailed GitHub page about it.
This mini Apple IIe runs on a C.H.I.P., a small $9 single board computer, and has a speaker and a TFT LCD display. Input is via a full-sized wireless keyboard. He doesn’t have joysticks working but that was an oversight and having realized how many games require joysticks, he has plans to add support for them. The case is 3D printed from models available on Thingiverse and links are on the GitHub page, along with all other details for making one yourself.
He did have to do some hacking. The video signal from the C.H.I.P. wasn’t available from the pin headers so he had to solder a wire directly to the board itself. The C.H.I.P. requires from 3.3 V to 5 V whereas the display wants 6 V to 12 V. To accommodate both he gets power from a 12 V drone battery and uses a 5 V buck converter for the C.H.I.P. And he had to modify the emulator to be legible on the low resolution of the display. The code for that is also available through the GitHub page.
While he uses the display as the screen for the Apple II emulator, it actually has two video inputs. So just in case he wants to show something on the display from another source, perhaps to watch a video, he’s made the second video input available using a socket in the back.
Want to see all the details for yourself? Check out [Tested]’s video below.
There are so many nice hacks in [Joekutz]’s retro LED display project that it’s hard to know where to start. There’s his DIY LED display controlled by an Arduino UNO. To have some text or picture for the display, he’s wired the output of a Bluetooth speaker directly to the Arduino, and sends it speaker tones that encode the text to draw. And as if that wasn’t enough, he’s hacked a quartz driver board from an analog clock to use the display as a clock as well.
Let’s start with the LED matrix display, perhaps the best excuse for trying your hand at shift registers. This display uses two such 8-bit shift registers daisy chained together feeding two 8-bit Darlington arrays. The display has ten rows of sixteen columns, and you guessed it, the columns are controlled by the sixteen shift registers. Two Arduino pins tell the shift registers which column to turn on. The rows are turned on and off using ten transistors controlled by ten more Arduino pins. Scanning at 80 frames per second he gets a nice, flickerless display.
To make both the LED matrix circuit board and the control board, [Joekutz] carved out isolation paths in copper clad boards using his homemade CNC mill. Be sure to check out the first video below to see his misadventures with it that ultimately led to his gorgeous boards.
Not only has [Joop Brokking] built an easy to make balancing robot but he’s produced an excellent set of plans and software for anyone else who wants to make one too. Self-balancers are a milestone in your robot building life. They stand on two-wheels, using a PID control loop to actuate the two motors using data from some type of Inertial Measurement Unit (IMU). It sounds simple, but when starting from scratch there’s a lot of choices to be made and a lot of traps to fall into. [Joop’s] video explains the basic principles and covers the reasons he’s done things the way he has — all the advice you’d be looking for when building one of your own.
He chose steppers over cheaper DC motors because this delivers precision and avoids issues when the battery voltage drops. His software includes a program for getting a calibration value for the IMU. He also shows how to set the drive current for the stepper controllers. And he does all this clearly, and at a pace that’s neither too fast, nor too slow. His video is definitely worth checking out below.