Recent research conducted by a team from Columbia University and the Feinstein Institutes for Medical Research Northwell Health has revealed significant insights into the similarities between artificial intelligence (AI) language models and human brain functions. Published in the journal Nature Machine Intelligence, the study sought to identify how emerging large language models (LLMs) are evolving to exhibit behaviours that mimic human cognitive processes, particularly in language comprehension.
The study involved a comprehensive analysis of twelve pre-trained LLMs, each equipped with varying numbers of parameters, including prominent models such as LLaMA, Falcon, and Mistral. The researchers deduced that as these language models develop, they begin to reflect intricate aspects of biological neural networking, with their design involving complex, deep processing layers akin to those found in human brains. First author Gavin Mischler commented on the research, stating, “Although previous research has demonstrated similarities between LLM representations and neural responses, the computational principles driving this convergence—especially as LLMs evolve—remain elusive.”
To draw parallels with human brain activity, the researchers employed intracranial electroencephalography (iEEG) on eight consenting patients undergoing neurosurgery for drug-resistant epilepsy. This technique allowed the team to record brain activity as participants listened to various verbal narratives. Speaking about the methodology, the scientists stated, “Here we used intracranial electroencephalography recordings from neurosurgical patients listening to speech to investigate the alignment between high-performance LLMs and the language-processing mechanisms of the brain.”
The analysis involved providing identical content to both the AI models and the human participants, followed by assessments of reading comprehension and commonsense reasoning. This structured approach allowed the researchers to develop a scoring benchmark for the LLMs based on their performance relative to the human cognitive tasks.
In their findings, the researchers discovered that the LLMs exhibiting the highest performance demonstrated a more pronounced ‘brain-like hierarchy’ in their processing layers. Mistral was identified as the top performer, followed by XwinLM and LLaMA2, among others. The overarching analysis illustrated that these LLMs exhibit hierarchical structures that align with the neurobiological frameworks responsible for auditory and language processing within the human cortex.
An intriguing aspect of the research lies in its focused comparison of different LLM models within a consistent architectural framework. This allowed the scientists to draw more precise conclusions about the degree to which language models are beginning to echo human cognitive processing patterns.
The researchers noted that the similarities between LLMs and human cognitive processes stem from the hierarchical organisation of language. The construction of language in both AI and human processing originates from smaller components—such as phonemes and syllables—that amalgamate into more complex constructs like words and sentences.
In conclusion, the findings from this study hint at potential future developments in AI technologies that may better align with human cognitive functions, thus advancing the field of AI language processing and opening paths for innovative applications in various industries.
Source: Noah Wire Services