Controversial Avatars on Character.ai Spark Outrage: Calls for Stricter AI Regulation Intensify
An AI-based website, Character.ai, has come under intense scrutiny following revelations that chatbots mimicking the deceased teenagers Molly Russell and Brianna Ghey were active on the platform. These digital avatars were crafted using artificial intelligence, a feature that Character.ai offers to its users to create digital imitations of various personalities.
Molly Russell tragically passed away at the young age of 14 in 2017 after exposure to damaging online content. Brianna Ghey, a transgender teenager aged 16, was brutally murdered in February 2023. The presence of these virtual representations on Character.ai has attracted considerable backlash for their insensitivity and potential to cause distress to the victims' families and friends.
The initial discovery of these controversial chatbots was made by The Telegraph. The publication reported that the avatars could be interacted with merely by establishing an account, even when the age provided was as young as 14. This revelation raises concerns about the platform's safety measures designed to protect young users from potentially harmful content.
In response to the situation, Character.ai stated that it takes the safety of its platform seriously. The tech company asserted that it proactively moderates characters both on its own initiative and in response to user reports. The firm indicated that the relevant chatbots were referred to its moderation team and pledged their removal within 24 hours following the revelation. Despite this promise, the incident has intensified calls for tighter regulations of AI and online platforms which allow user-generated content.
Andy Burrows, CEO of the Molly Rose Foundation—a charity centred around online safety established in Molly Russell's memory—expressed his dismay over the creation of such avatars. He described the action as a "sickening" oversight in moderation that exacerbates the pain of those connected to Molly. Burrows emphasised the need for urgent regulatory intervention to prevent similar incidents from recurring, stressing that safety should not be a secondary concern for AI companies.
Esther Ghey, Brianna's mother, also voiced her concerns. She highlighted the manipulative and hazardous nature of the online environment and underscored the necessity for immediate protective measures for children navigating the digital world.
Character.ai, a start-up launched in 2021, hosts a platform where users engage with avatars that simulate various personalities, including figures from popular culture and custom-created personas. The platform boasts approximately 20 million users, predominantly comprising younger people, with about 60 per cent aged between 18 and 24, and a significant portion under the age of 18.
This incident follows a similar legal challenge in the United States, where Character.ai faces a lawsuit involving a 14-year-old boy whose mother alleges that her son committed suicide due to his obsession with an avatar on the platform.
As digital and AI technologies continue to evolve and permeate everyday life, the incident at Character.ai spotlights the ongoing debate over the balance between innovation and safeguarding vulnerable users in the digital age.
Source: Noah Wire Services