Imagine you were walking down a beach, and you came across some driftwood resting against a pile of stones. You see it in the distance, and your brain has no trouble figuring out what you’re looking at. You see driftwood and rocks – you can clearly distinguish between the two objects without a second thought.
Think about the raw data entering the brain. The textures of the rocks and the driftwood are similar. The colors are similar. The irregular shapes are similar. Thus the raw data entering the brain’s V1 area for both objects must be similar as well. Now think about the borders that separate the pieces of driftwood from the edges of the rocks. From a raw data perspective, there is no border, and likewise no separation because the two objects are so similar. But yet your brain can clearly see a rock and a piece of driftwood – two distinctly different objects. So how does the brain do this? How does it so easily differentiate between the two? If the raw data on either side of the border separating the wood and the rocks is the same, then there must be an outside influence determining where that border is. Jeff Hawkins believes this outside influence is a very special and most interesting type of feedback. Read on as we explain and attempt to implement this form of feedback in our hierarchical structure of invariant representations.
Now, this is not your mother’s feedback we’re talking about here. We’re not simply injecting part of the output back into the input like a neural network would do. Because we are working with a hierarchy of patterns, the feedback must take place between the hierarchical levels. Basically, each level is a child to the parent levels above it. A parent level can “adjust” what the child level below it is actually seeing to accommodate what it predicts it (the parent level) should be seeing.
Each level of the hierarchy is constantly trying to predict what the next pattern will be. On the very low level, these are just lines and edges. But near the top, the constant prediction of high level patterns can act as a base for intelligence. While this is only one of many theories on how the brain does this, replicating this type of hierarchical feedback in silicon should be possible. But it’s not going to be easy. Let’s walk through a simplified example that continues off our previous work of forming an invariant representation of a basic shape. But this time, we shall apply hierarchical feedback.
Finding Structure with Prediction
In our previous example of forming an invariant representation of a square, raw binary data was taken into the model through Tier 4. Tier 4’s job was to find repeating binary sequences, give that sequence a name, and pass that name up to Tier 3. Tier 3’s job was to find repeating patterns of names, give the group of repeating patterns another name, and pass that other name up to Tier 2. This process repeats until the invariant representation of a square is formed at the top most tier.
This works in a fixed, unchanging environment with no other shapes or lines to confuse our model. But what happens if we attempt to replicate the rock/driftwood problem by placing our square alongside many other similar shapes? This is where feedback becomes necessary. If we are unable to distinguish between a single side of our square and the side of an adjoining triangle, then we must use stored invariant patterns of the square higher up in the hierarchy to determine what we are seeing. So how do we do this.
The Power of Variability
Each name that a Tier outputs will need an “adjustment” variable, with that variable controlled by the Tiers above it. For instance, imagine Tier 2 sees the pattern 115 & 125 repeat often, and it gives it a name of 240, and passes that name to Tier 1. Fast forward in time, and imagine the name of “115” once again comes across the registers of Tier 2. It can predict with a certain percentage that the next name it should see is a “125”. But say the name coming in is closer to a “123”. Tier 2 will adjust what Tier 3 is seeing by changing the “w” variable of the Tier 4 output, so that it now passes to Tier 3 what is needed for Tier 2 to meet its prediction.
This type of feedback occurs between all stages of the hierarchy. It’s why you can see both driftwood and rock laying together as clearly as you could see an orange basketball laying in their midst. While little to no feedback would be needed to identify the basketball, the majority of what you see of the rock and wood is actually just feedback based upon individual invariant representations of a rock and a tree branch. Likewise, we cannot see a single line of the square next to a line of a triangle – for the data coming off the ADC for these two lines is identical. The two shapes can only be separated by feedback from higher up in the hierarchy.
Keep in mind that this simple example of a working hierarchy is just that – simplified. To operate in the real world, a working hierarchy would consist of hundreds, perhaps thousands of Tiers – each with individual inputs that could reach into the hundreds of thousands. Luckily for you, the entire concept can be emulated in software. With FPGAs, gigabytes of RAM and CPUs that can run billions of cycles in a single second, all of which are available to you, what are you waiting for? Can you write a program that takes the torrent of data off the ADC and configure it into a hierarchy with feedback? How would you even start? Show us your rough draft pseudo code.