In a recent incident that has brought ethical considerations of technology and privacy to the fore, an AI-generated chatbot version of Jennifer Crecente, a teen who was murdered nearly two decades ago, appeared on the AI companion platform Character.AI without the consent of her family. This revelation has sparked outrage, particularly from the late teenager's father, Drew Crecente, who has dedicated his efforts to honouring his daughter’s memory through advocacy against teen dating violence.
Jennifer Crecente was tragically killed by her ex-boyfriend in 2006 in Austin, Texas, at the age of 18. Following her death, her father founded a nonprofit organisation aimed at combating teen dating violence, and her mother, Elizabeth Crecente, also established a similar nonprofit with the same mission.
The presence of the Jennifer chatbot was discovered when Drew Crecente received a Google alert, which directed him to the Character.AI platform where his daughter's image and name were used to market the chatbot. To his shock, the chatbot featured Jennifer's yearbook photograph and was introduced as a "knowledgeable and friendly" AI persona, described erroneously as a "video game journalist." Drew noted that these details were inaccurate and seemed to have stemmed from a confusion with Jennifer's uncle, Brian Crecente, a well-known figure in the video game journalism industry.
Upon discovering the profile, Drew experienced an emotional turmoil. Describing the moment to The Washington Post, he recounted his immediate reaction: "My pulse was racing. I was just looking for a big flashing red stop button that I could slap and just make this stop." He expressed his disappointment and distress, remarking on what he perceived as a significant ethical violation.
Character.AI, recognising the breach of its user policies, has since taken down the chatbot. The situation raises important questions regarding the platform's responsibility and the effectiveness of its measures to prevent such incidents. Notably, Character.AI recently entered a significant agreement to provide its generative AI technology to Google, emphasising its growing influence in the tech domain.
Jen Caltrider, a privacy researcher with the Mozilla Foundation, criticised the company’s approach, which seems to rely on those affected by policy violations to report issues. She highlighted the ethical dilemma of the platform profiting from interactions with unauthorised content until it is flagged.
This situation underscores the complexities and challenges of regulating AI technologies, especially in cases involving sensitive and personal content. The use of generative AI for creating profiles based on real individuals without consent introduces serious ethical concerns, particularly when such profiles are based on deceased persons with compelling personal stories. As technology advances, ensuring responsible and respectful usage remains a pressing challenge for platforms like Character.AI.
Source: Noah Wire Services