Large language models used in artificial intelligence can predict how the human brain responds to visual stimuli, according to a study conducted by researchers from Freie Universität Berlin last August. Does that mean AI can now read minds? It sometimes seems like it can, for example, when you were just thinking about a new pair of running shoes and suddenly find yourself flooded with online ads for them. But what is this study really about?

Its author, Adrien Doerig, guest professor for cognitive computational neuroscience, explains how images (such as a happy dog on a sailboat), “semantic fingerprints,” and thousands of MRI scans are yielding new insights for neuroscience, AI, and philosophy. 

As a child, the Swiss researcher was fascinated by the fact that the human mind exists thanks to neurons – those tiny cells that each send only a few electrical impulses, but that are the material basis for the complex phenomenon that is the mind. Originally a student of the life sciences, Doerig went on to study neuroscience and theoretical physics, and dabbled in philosophy.

Perception processing in the brain is complicated, he explains. For example, damage to a specific region of the brain can make it impossible to recognize faces. People with damage to this region, called the fusiform face area (FFA), in the lower temporal and occipital lobes can see a nose, eyes, and mouth, but cannot combine the parts into a whole. When it is not damaged, the FFA is particularly active when we look at faces.

Brains and LLMs Share These Traits

Working together with colleagues from Osnabrück, Minnesota, and Montréal, Doerig wanted to find out how the brain processes the perception of scenes – not just “a face,” but something a bit more elaborate like “a person sitting at a table writing” or “a cheerful dog standing on a sailboat.” “We had the idea that large language models (LLMs) – the kind behind AI bots like ChatGPT – could be of help to us,” he says. 

Large language models grasp meaning by learning statistical and semantic patterns from vast amounts of data in the form of texts. Words, sentences, and concepts correspond to high-dimensional vectors in this context, which allows language to be translated into numbers. These mathematical representations allow the LLMs to predict probable continuations of a text and produce coherent, context-appropriate responses. In this way, they reproduce certain aspects of human language understanding – although the models do not have conscious awareness.

For example, given the sentence fragment “I went to the market to…,” a model might complete it with “…buy tomatoes” because it has learned from millions of examples that such continuations are common. This “knowledge” is encoded in semantic fingerprints or embeddings (as experts call them). These numerical representations capture how words and ideas relate to one another in meaning and context.

Is it possible that our brains process information in a similar way when we see pictures of scenes like “a cheerful dog on a sailboat?” Does the body’s visual system use a coding scheme comparable to the semantic fingerprints of large language models? According to their study, the surprising answer is: Yes! Our visual system and large language models function in strikingly similar ways.

Predicting Brain Activity

To prove this similarity, the researchers used a large dataset from the University of Minnesota. Eight participants spent many hours inside an fMRI scanner that precisely recorded which regions of their brains were active while they looked at images of everyday scenes for three-second increments.

When the brain sees a picture of “a cheerful dog on a sailboat,” it produces a pattern of activity that can be quantified in numbers using neuroimaging. For each image that the team tested they derived one unique numerical pattern from the brain scan and one from the embedding of the scene’s textual description in the language model. 

“Using statistical methods – specifically representational similarity analysis and linear regression – we were able to show that these values correlate and predict each other,” Doerig explains. Just as you can predict ice cream sales from the weather (i.e., on warmer sunny days sales tend to increase), you can also predict patterns of brain activity from the embeddings of scene descriptions in language models.

The computational model was also able to predict brain activity patterns in higher visual cortex for new scenes, for example, “children playing Frisbee in the park.” Conversely, the researchers could infer the semantic fingerprints of the viewed image – and ultimately even generate captions – from the brain scans. “We managed to decode information directly from the brain,” says Doerig. The method is not perfect – factors like movement or lapses in concentration make analysis harder. Nevertheless, the team still produced short descriptions of what participants were seeing, based purely on their brain activity.

Will We Soon Be Reading Minds?

Decoding information directly from the brain can have medical applications. This kind of technology can allow individuals with paralysis to communicate their thoughts. But such decoding requires enormous amounts of data – “And that’s rather impractical,” says Doerig. That is why many researchers are currently working to reduce data requirements and make these methods usable in clinical contexts.

In prosthetics, comparable techniques are already more advanced. Today’s artificial limbs can be connected to brain signals, allowing patients to move them by will – and sometimes even feel feedback from them. Doerig sees parallels to his own work here. In prosthetics, researchers learn which brain signals correspond to specific movements and design prostheses to respond accordingly. “We are trying to do something similar, but with language instead of movement. We want to know which brain signals correspond to specific thoughts. That’s much harder to do because language is a more complex process and the corresponding brain activity is distributed more widely. That makes it all the more difficult to interpret.

Could this bring us one step closer to being able to read minds? Doerig understands why his research might raise concerns like this, but he refers to neuroscientist Nancy Kanwisher from the Massachusetts Institute of Technology, who famously wrote about why MRIs cannot (currently) serve as lie detectors. “It is too easy to scramble the data in an MRI. All you have to do is move your tongue or start doing sums in your head and you’ll create waves in your brain strong enough to disrupt the measurements,” Doerig says. MRIs only work with the subject’s full cooperation.

When Doerig first presented his project at conferences three years ago, many colleagues were surprised by his findings. Today, his approach has been taken up by researchers from a variety of disciplines. Researchers from neuroscience, AI, and philosophy are increasingly coming together to explore how different ways of describing the world might be fundamentally connected.

---

This article originally appeared in German in the Tagesspiegel newspaper supplement published by Freie Universität Berlin on November 29, 2025.