When you reach out to pet a dog, you expect it to feel soft. If it doesn’t feel like how you expect, your brain uses that feedback to inform your next action — maybe you pull your hand away. Previous models of how the brain works have typically separated perception and action. For Allen School professor Rajesh Rao, those two processes are closely intertwined, and their relationship can be mapped using a computational algorithm.
“This flips the traditional paradigm of perception occurring before action,” said Rao, the Cherng Jia and Elizabeth Yun Hwang Professor in the Allen School and University of Washington Department of Electrical & Computer Engineering and co-director of the Center for Neurotechnology.
In a recent paper titled “A sensory-motor theory of the neocortex” published in the journal Nature Neuroscience, Rao posited that the brain uses active predictive coding (APC) to understand the world and break down complicated problems into simpler tasks using a hierarchy. This architecture, which is inspired by previous work in artificial intelligence (AI), can in turn be potentially used to train AI algorithms on increasingly complex problems with less data and better predict different outcomes.
“Data from neuroscience suggests the brain uses a hierarchical generative model to constantly predict the consequences of actions,” Rao said. “The brain is creating its hypotheses saying, ‘Here’s what I predict will happen in the world. Now, let’s check this hypothesis with what’s really coming in through the sensors in my body.’ Errors in prediction can then be used to correct the hypothesis.”
For Rao, the anatomy of the neocortex indicates a “tight coupling” between sensory and motor processes, or perception and action, similar to the mathematical model used in reinforcement learning in AI. Reinforcement learning utilizes a generative model to capture the relationship between an agent’s motor output and the sensory input it receives from the environment. You reach out to pet a dog, for example, and you feel the texture of its fur.
This generative model is also called a world model, represented mathematically as a state transition function specifying how the agent’s actions change their world. Alongside the world model is a policy function that selects a sequence of actions. These functions work together to help you learn, perform new tasks and predict the consequences of actions, such as what you expect this dog’s fur to feel like compared to other dogs you have touched.
Anatomically, each section of the neocortex is “like a six-layered layer cake,” Rao explained, “with the middle and top layers processing sensory information and the bottom layers sending information to action centers of the brain.” The model suggests that areas higher up in the hierarchy can modulate the dynamics of the lower level neural networks and change the function being computed there, similar to hypernetworks in AI.
“The ability to change the functions being computed at different levels of a computational hierarchy endows the APC network with remarkable versatility,” Rao said.
Break it down: How AI can learn from the brain
The APC model’s hierarchical approach can break down a complicated, abstract problem into smaller parts in a process known as compositionality. If the higher level goal, for example, is to go to the grocery store, that task can be decomposed into a sequence of simpler steps, Rao explained. First, you might walk to the garage where your car is parked, open the door and then unlock the car door. Eventually, the tasks can be broken down to specific muscles in the hand and to the lowest level in the spinal cord controlling your hand.
Compositionality may help address one of the problems holding back traditional AI models. For every new problem that the AI model faces, it needs to be trained, potentially using reinforcement learning, on lots of new data, whereas the human brain is very good at quick generalization with little data, Rao noted.
Instead of using trial-and-error learning or planning each discrete step, if the agent has already learned to solve simpler tasks such as navigating smaller rooms or moving from corner to corner, it can use that knowledge to break down the task into a sequence of simpler tasks it already knows how to solve.
“The compositionality inherent in the APC model allows it to compose solutions to new problems in the world really quickly,” Rao said. “Suppose I already learned how to get into the car. I can keep using that policy function for all kinds of other tasks like driving to school or going to meet a friend.”
The same APC model architecture can also be used for visual perception and learning. An APC model learns to read and write the number eight, for example, by breaking it down into different strokes. It can then use those parts to compose other new characters.
“The APC model builds on past ideas of hierarchical reinforcement learning but goes beyond the usual hierarchy of policy functions,” Rao said. “The architecture of the neocortex suggests that there is a great benefit to modeling the world itself as a hierarchy. Coupling such a hierarchical world model with a hierarchy of policy functions may be how our brain tackles the complexity of the world we live in.”
The next step for Rao and his students in the Allen School’s Neural Systems Laboratory is to look at how to apply this architecture to large-scale AI problems such as language and robotics and test the model’s predictions in collaboration with neuroscientists.