Proliferation of AI-Generated Profiles Widespread on X, Research Finds
Recent research has unveiled a significant uptick in the use of AI-generated profile pictures on the digital platform X, notably used in efforts to amplify political messages and crypto-related schemes. A collaborative study conducted by researchers at Ruhr University Bochum, the GESIS Leibniz Institute, and the CISPA Helmholtz Center, identified nearly 8,000 accounts harnessing these synthetic images.
The advent of generative artificial intelligence has blurred the lines marking genuine from artificial media content, creating challenges in distinguishing between the two. This phenomenon was highlighted in the study, which underscored the increasing difficulty faced by humans in differentiating machine-generated outputs from authentic content.
The research further revealed that more than half of these synthetic-faced accounts were created in 2023, often through suspicious bulk creation events. This is characteristic of accounts created for purposes such as message amplification, disinformation campaigns, or other disruptive activities. Researchers discovered that many of these accounts tend to operate briefly, with more than half suspended within a year.
Adding depth to these findings, a report by the Center for Countering Digital Hate noted that posts by X owner Elon Musk, particularly those favouring former President Donald Trump, amassed a staggering 17.1 billion views. This number surpasses the total views of all US political campaign advertisements combined during the same timeframe. Additionally, Musk's posts containing unverified claims about the US elections accrued over 2 billion views without any of the posts receiving a Community Note, a user-generated fact-check service on the platform.
AI-generated accounts display discernible patterns that separate them from authentic users. The study found these fake-image accounts generally have fewer followers compared to accounts with real images. They also show less interaction with their follower ecosystem, often posting without engaging directly with other accounts.
A detailed content analysis revealed orchestrated posting patterns, pinpointing specific clusters of accounts participating in large-scale coordinated campaigns. These accounts often debated contentious issues spanning various languages, including topics like the war in Ukraine, US elections, and COVID-19 vaccination policies. Beyond political discourse, many accounts were also involved in promoting crypto scams and adult content.
The research team employed sophisticated detection methodologies, reporting almost complete accuracy in identifying AI-generated profiles. The team aims to expand their detection capabilities to identify AI-generated images created using different technological models such as Diffusion models, further refining their methodology to uncover coordinated inauthentic behaviour across various social media platforms.
The study sheds light on the increasingly sophisticated tactics employed in digital manipulation, underscoring the evolving landscape of social media disinformation.
Source: Noah Wire Services