Stanford Study Highlights AI Language Gaps, Sparks Debate on Technological Equity
Researchers at Stanford University recently probed the linguistic capabilities of popular artificial intelligence (AI) chatbots, unveiling a significant gap in the technological equity of AI systems when it comes to non-English languages. The investigation, part of a broader effort to understand AI's limitations, underscores concerns about the exclusion of low-resource languages—a term used to describe languages with limited available digital data.
The study, conducted by the Stanford Artificial Intelligence Laboratory, put Claude 3.5, a chatbot developed by AI company Anthropic, through various tests. One test involved asking the bot, in Vietnamese, to compose a poem in the traditional "song thất lục bát" format, which requires a pattern of lines consisting of seven, seven, six, and eight words. The bot produced a poem but failed to adhere to the specified structure. In another test, the bot was asked for the Vietnamese word for a mother’s younger brother, but it mistakenly provided terms for a father’s younger and older siblings.
These shortcomings reflect a broader issue within AI, as most AI systems are predominantly trained using English-language data. This training bias results in inaccuracies when these models are applied to other languages, posing a risk of exacerbating technological inequities and marginalising non-English speaking communities.
Sang Truong, a doctoral candidate at Stanford and part of the team evaluating Vietnamese language models, highlighted the economic stakes involved. He noted that a delay in accessing advanced technology could lead to significant economic setbacks, spanning potentially several decades.
The so-called low-resource languages, while spoken by millions worldwide, suffer from a dearth of digital content. This paucity stems from a concentrated focus on AI development in countries like the United States and China. Languages such as Hindi, Bengali, and Swahili, along with numerous lesser-known dialects, fall into this category.
A study by W3Techs, a tech survey company, reveals that over 60% of the internet’s content is in English, even though native English speakers comprise a mere 5% of the global population. Other languages with substantial online presence include Mandarin and Spanish, which benefit from robust digital data sets.
To address these disparities, academic institutions, grassroots organisations, and volunteer groups are striving to enhance resources for underrepresented languages. Johannesburg-based Lelapa AI is one such enterprise, working to develop multilingual AI products tailored to African communities. Pelonomi Moiloa, CEO and co-founder of Lelapa AI, emphasised the importance of creating technology that respects and incorporates local languages and cultures, thereby avoiding the necessity for cultural assimilation.
While tech giants like Google, Apple, and OpenAI have endeavoured to support a range of languages, experts argue that these efforts often fall short. Chinasa T. Okolo, a fellow at the Brookings Institution, pointed out that global low-resource language markets do not always offer sufficient commercial incentive for large companies to prioritise their inclusion.
The lack of linguistic inclusivity in AI also raises security concerns. Sara Hooker, head of Cohere for AI’s research arm, warned that users could potentially elude safety filters by switching languages, citing dangerous instructions on bomb-making as a possible example.
Despite these challenges, there are initiatives aimed at bridging the language gap. In February, Hooker's team launched Aya, a multilingual AI model encompassing 101 languages, created through the efforts of over 3,000 independent researchers. Nevertheless, Hooker acknowledged that such projects alone cannot completely resolve the issue.
Creating large-language models (LLMs) demands extensive high-quality data, which is currently more accessible for dominant languages. Truong likened constructing an LLM to educating a newborn, noting the stark contrast between the abundance of English-language resources and the scant availability in languages like Vietnamese.
In some regions, government intervention is nurturing language-specific AI development. Notable examples include Nigeria, which has committed to supporting tech startup Awarri in creating a local language model. Similarly, the governments of Iceland and Wales are collaborating with OpenAI to enhance ChatGPT’s understanding of their native tongues.
Sanmi Koyejo, head of Stanford Trustworthy AI Research, underscored the importance of incorporating more languages into AI products to preserve cultural nuances and diverse worldviews. He referenced a Stanford study that compared AI chatbot responses to Pew Research survey questions, revealing that the bots' answers aligned closely with the perspectives of Californians, a reflection of the geographical and cultural concentration of AI development.
Amid the complexities, industry insiders agree that a multicultural approach to AI is crucial for achieving inclusive technological progress.
Source: Noah Wire Services