New Study Reveals AI Safety Shortcomings in Leading Tech Labs
A recent study conducted by the French nonprofit SaferAI has unveiled significant gaps in risk management practices among some of the leading artificial intelligence labs worldwide. Released on Wednesday, the study evaluates and grades the risk management measures of top AI companies, highlighting areas where improvement is urgently needed.
The findings were most severe for Elon Musk's AI firm, xAI, which scored lowest with a 0 out of 5 in the study’s safety ratings. The assessment, which aimed to establish a standard for evaluating how AI companies are managing risks, comes as these technologies become increasingly powerful and widely used. The inadequacies pose potential threats, as AI systems have demonstrated capabilities to anonymously hack systems and aid in the development of bioweapons.
Siméon Campos, founder of SaferAI, pointed out the discrepancy between the swift development of AI technology and the lagging pace of effective risk management. "AI is extremely fast-moving technology, but AI risk management isn’t moving at the same pace," Campos remarked, indicating the critical need for the ratings as a temporary measure until governments implement formal assessments.
To conduct the study, SaferAI researchers employed "red teaming" methodologies to identify vulnerabilities within AI models and scrutinised the companies’ approaches to assess and mitigate potential threats. The six companies evaluated include Meta, Mistral AI, OpenAI, Google Deepmind, and Anthropic, alongside xAI. While xAI received the lowest score, Meta and Mistral AI were categorised under "very weak" risk management. OpenAI and Google Deepmind received "weak" ratings, and Anthropic was rated highest with a "moderate" score of 2.2 out of 5.
According to Campos, xAI's low score can be attributed to its lack of published information regarding risk management strategies. He expressed optimism that as xAI's model, Grok 2, competes with established systems like Chat-GPT, the organisation might shift its focus toward enhancing safety measures. "My hope is that it’s transitory: that they will publish something in the next six months and then we can update their grade accordingly,” Campos added.
The report has sparked discussions about the necessity for AI companies to bolster their internal safety frameworks. Campos suggested that adopting risk management strategies from high-stakes industries, such as nuclear power or aviation, could mitigate potential biases and prevent the misuse of AI technologies by malicious actors. "Despite these industries dealing with very different objects, they have very similar principles and risk management framework," he emphasised.
SaferAI’s rating system is aligned with significant AI standards globally, including those from the EU AI Act and the G7 Hiroshima Process. SaferAI is a constituent of the US AI Safety Consortium, established by the White House earlier this year. Its funding is primarily sourced from the tech nonprofit Founders Pledge and investor Jaan Tallinn.
Prominent AI figure Yoshua Bengio voiced support for the new ratings system, underscoring its importance in ensuring the safe development and deployment of AI models. Bengio stated he hopes the system will prevent companies from self-assessing their safety protocols, thus maintaining transparency and accountability in the industry.
Source: Noah Wire Services