Dendrites and gatekeeper cells unlock flexible learning in mice, study finds

The ability to adapt behavior flexibly to changing environments is a hallmark of intelligence. However, the biological mechanisms in the brain that are responsible for this cognitive flexibility remain poorly understood.

Eduardo Maristany de las Casas, a Ph.D. student in Professor Matthew Larkum’s research group, Neural Plasticity, at the Department of Biology at Humboldt-Universität zu Berlin, in collaboration with Professor Dieter Jaeger from Emory University in Atlanta, investigated the extent to which the biological basis of this intelligent behavior lies in the long, branched extensions of nerve cells, the dendrites.

In a technically sophisticated experimental setup, the researchers combined a behavioral test with mice—in which the animals had to switch back and forth between easy and difficult tasks—with microscopic imaging (two-photon fluorescence microscopy) of the relevant brain regions, as well as targeted manipulation of individual neurons.

“In this study, we show that dendrites and their interaction with interneurons lie at the core of this ability, and that different forms of learning are implemented differently in the brain,” says Eduardo Maristany. The study results were recently published in Science.

Interaction with interneurons and cluster formation enables behavioral adaptation

Maristany and his team discovered that the tree-like extensions of a specific type of neuron—pyramidal neurons—in the cerebral cortex of mice play a crucial role in flexible learning. These pyramidal neurons are located in the part of the cerebral cortex responsible for controlling voluntary movements (frontal motor cortex).

Flexible learning takes place through the interaction of these dendrites with a specific class of nerve cells (NDNF interneurons), which are located in the same outermost layer of the cerebral cortex and act as a kind of gatekeeper: When these cells are active, they suppress calcium signaling in the dendrites, thereby preventing the brain from updating learned rules. During a learning process, however, they reduce their activity. This opens a window of opportunity for the animal to adapt its behavior.

At the cellular level, this is accompanied by a change in the shape of the dendrites: Their spines group together according to their function, thereby forming functional clusters. It is only through this dendritic plasticity that the dendrites can transmit information effectively and thus enable the learning process.

In his experimental setup, Maristany combined a behavioral test with mice and microscopic images of the relevant brain regions (two-photon fluorescence microscopy) with targeted manipulation of individual neurons in the mice’s brains. For the test, the mice had to learn two different behavioral patterns of varying difficulty and were trained to switch back and forth between them (rule-switching paradigm).

The complex task involved learning that, whenever their whiskers were stimulated on the left or right side, they should lick sugar water (as a reward) on the corresponding side. In the simple task, they were required to always lick the sugar water on the left side, regardless of which side their whiskers had been stimulated. Switching back and forth between these tasks revealed whether and how learned behaviors are retained, overwritten or updated.

Dendritic plasticity is only required in complex learning situations

During the behavioral tests, the researchers observed that the dendritic plasticity described above was not required for the execution of the simple behavior once the mice mastered it, nor for the switch from the more difficult to the simpler rule, but only when the mice had to perform the complex behavioral rule again. Clusters formed by the spines on the dendrites disintegrated as soon as the animals performed the simple task.

This study provides some of the first direct evidence that the elaborate dendritic trees of cortical neurons act as active computational units essential for adaptive behavior. The extent to which these findings can be generalized to other brain regions or to more real-life learning conditions remains to be investigated.

Insights for the treatment of cognitive disorders and a source of inspiration for AI

The biological mechanisms of learning uncovered in the study could, in the future, play a role in understanding disorders of cognitive flexibility—such as those seen in autism spectrum disorders, schizophrenia and age-related cognitive decline, where the ability to adapt to changing rules and environments is impaired. Additionally, the principles uncovered here—particularly the gating of plasticity by inhibitory circuits—may inspire new architectures in artificial intelligence and machine learning.

“One of the aspects I find most fascinating is the brain’s ability to remain cognitively flexible and to keep learning with relatively limited resources,” says Maristany.

Who’s behind this story?

Lisa Lock

BA art history, MA material culture. Former museum editor, paramedic, and transplant coordinator. Editing for Science X since 2021.

Full profile →

Advertisements

Robert Egan

Bachelor’s in mathematical biology, Master’s in creative writing. Well-traveled with unique perspectives on science and language.

Full profile →