Your job is to make a circuit that will illuminate a light bulb when it hears the song “Mary Had a Little Lamb”. So you breadboard a mic, op amp, your favorite microcontroller (and an ADC if needed) and get to work. You will sample the incoming data and compare it to a known template. When you get a match, you light the light. The first step is to make the template. But what to make the template of?
“Hey boss, what style of the song do you want to trigger the light? Is it children singing, piano, what?”
Your boss responds:
“I want the light to shine whenever any version of the song occurs. It could be singing, keyboard, guitar, any musical instrument or voice in any key. And I want it to work even if there’s a lot of ambient noise in the background.”
Uh oh. Your job just got a lot harder. Is it even possible? How do you make templates of every possible version of the song? Stumped, you talk to your friend about your dilemma over lunch, who just so happens to be [Jeff Hawkins] – a guy whose already put a great deal of thought into this very problem.
“Well, the brain solves your puzzle easily.” [Hawkins] says coolly. “Your brain can recall the memory of that song no matter if it’s vocal, instrumental in any key or pitch. And it can pick it out from a lot of noise.”
“Yea, but how does it do that though!” you ask. “The pattern’s of electrical signals entering the brain have to be completely different for different versions of the song, just like the patterns from my ADC. How does the brain store the countless number of templates required to ID the song?”
“Well…” [Hawkins] chuckles. “The brain does not store templates like that”. The brain only remembers the parts of the song that doesn’t change, or are invariant. The brain forms what we call invariant representations of real world data.”
Eureka! Your riddle has been solved. You need to construct an algorithm that stores only the parts of the song that doesn’t change. These parts will be the same in all versions – vocal or instrumental in any key. It will be these invariant, unchanging parts of the song that you will look for to trigger the light. But how do you implement this in silicon?
Some organizations have taken Hawkins’ ideas and stealthily run with them, with schemes already underway at companies like IBM and federal organizations like DARPA to implement his ideas in silicon…
Indeed, companies are already working to implement [Jeff Hawkin’s] theory of intelligence into their own systems. It’s a complicated theory, which is laid out in his book – On Intelligence. Forming invariant representations (IR) is only the beginning, and we will discuss other parts of the theory in later articles. But for now, we will concentrate on how one would go about forming IR’s of real world data in silicon. We simply cannot move forward with the theory until this core component is understood. The problem is nobody seems to know how to do this. Or if they do, they’re not talking This is where you come in!
Consider this image. Let us pretend these are serial signals coming off multiple ADCs. On the other end of the circuit would be different versions of our song, with A – E representing those different versions. Because the data is constantly changing, we sample 4 signals at the same time for each version, which are numbered 1 – 4.
Immediately, we see a common pattern in all versions at times T4, T5 and T6. If we can somehow set our microcontroller to listen to the these times, we can detect all versions of the song. Further, we can see another pattern between the versions at times T1, T2 and T3. This type of analysis can be used to distinguish between the different versions. Both patterns are invariant representations of the song – a common, unchanging pattern hidden in the mist of a constantly changing environment.
This is a hypothetical example of course. In the real world, the signals would vary wildly. The key is to find the part that does not. Can you do it? How would you create an invariant representation of a real world event?