A new study has shown that the concept of the mammalian 'lizard brain' can be well and truly put to bed.

Based on a study that examined brains of bearded dragons (Pogona vitticeps), large lizards from the Australian desert, scientists have shown that mammal and reptile brains evolved separately from a common ancestor. It's another nail in the coffin of the notion of the so-called triune brain.

The idea of the lizard brain first emerged and rose to popularity in the 1960s and 1970s, based on comparative anatomical studies. Parts of the mammalian brain, neuroscientist Paul MacLean noticed, were very similar to parts of the reptilian brain. This led him to the conclusion that the brain had evolved in stages, after life moved to land.

First, according to MacLean's model, came the reptilian brain, defined as the basal ganglia. Then came the limbic system – the hippocampus, the amygdala, and the hypothalamus. Finally, the neocortex arose in primates.

Under the triune brain model, each of these sections is responsible for different functions; the more basal parts of the brain, for example, were supposedly more concerned with primal responses – like basic instincts for survival.

However, neuroscientists have been decrying the model for decades. The brain just doesn't work like that, in discrete sections that each play a separate part. Brain regions, anatomically distinct as they are, are highly interconnected, a web of humming neural networks. And with the advent of new techniques, we can start to better understand how brains evolved.

In a new study, a team of researchers from the Max Planck Institute for Brain Research turned to actual lizard brains to investigate, publishing their findings in a paper led by neuroscience graduate students David Hain and Tatiana Gallego-Flores.

By comparing molecular features of neurons in modern lizards and mice, the researchers hoped to unpick the evolutionary histories written into reptilian and mammalian brains.

"Neurons are the most diverse cell types in the body. Their evolutionary diversification reflects alterations in the developmental processes that produce them and may drive changes in the neural circuits they belong to," says neuroscientist Gilles Laurent of the Max Planck Institute for Brain Research.

Around 320 million years ago was a very important time for the evolution of vertebrates and their brains. That was when the first four-limbed animals (tetrapods) emerged from the water onto land, and started diversifying into the parent families that would ultimately produce birds and reptiles, on the one hand, and mammals, on the other.

There are structures in the brain established during the embryonic development of all tetrapods: a shared ancestral architecture in the subcortical regions.

But, because traditional anatomical comparisons of developmental regions might not be sufficient to fully detail all the differences and similarities between reptile and mammal brains, the researchers took a different approach.

They sequenced the RNA – a messenger molecule used as a template to form proteins – in individual cells from the brains of bearded dragons to determine the transcriptomes – the full range of RNA molecules in the cell – present, and thus generate a cell-type atlas of the lizard's brain. This atlas was then compared to existing datasets on mouse brains.

"We profiled over 280,000 cells from the brain of Pogona and identified 233 distinct types of neurons," Hain says.

"Computational integration of our data with mouse data revealed that these neurons can be grouped transcriptomically in common families, that probably represent ancestral neuron types."

In other words, there was a core set of neuron types with similar transcriptomes that both mammals and reptiles have in common, even though they have been evolving separately for over 320 million years.

But these neurons aren't restricted to a specific 'reptilian' region of the brain. Most regions of the brain, the analysis revealed, have a mix of ancestral and newer types of neurons within them, challenging the notion that some brain regions are more ancient than others.

In fact, the researchers found that neurons in the thalamus can be separated into two groups based on their connectivity to other regions of the brain. And these connected regions are quite different in mammals and reptiles.

The team found that the transcriptomes diverged in such a way that matched the connecting regions, suggesting that the transcriptomic identity of a neuron – the full genetic readout of what proteins it might need – is borne out of, or reflects, its connectivity.

"Since we do not have the brains of ancient vertebrates, reconstructing the evolution of the brain over the past half billion years will require connecting together very complex molecular, developmental, anatomical and functional data," Laurent says.

"We live in very exciting times, because this is becoming possible."

The research has been published in Science.