A recent study by the Ludwig Maximilian University of Munich (LMU) has shed light on the disparities between traditionally-crafted news articles and those produced with the aid of automation. The study, featured in the journal Journalism: Theory, Practice, and Criticism, reveals that readers find manually written articles significantly more comprehensible than their AI-generated counterparts.
The research team, based in the Department of Media and Communication (IfKW) at LMU, conducted a survey involving over 3,000 online news consumers across the United Kingdom. Participants in the survey were asked to rate one of 24 articles, with half of these articles being produced via automation, while the remainder were manually crafted by journalists. The findings indicated a clear preference for the manually written pieces in terms of clarity and comprehensibility.
The lead author of the study, Sina Thäsler-Kordonouri, highlighted that readers found the 12 automated articles less comprehensible, despite the fact that these articles had undergone sub-editing by journalists before reaching publication. The study further delved into specific reasons contributing to this discrepancy in comprehensibility.
A key issue pinpointed by the survey was the choice of words in AI-generated articles. Readers noted that these articles often included inappropriate, difficult, or unusual words and phrases, which detracted from their ease of understanding. Moreover, another area of dissatisfaction was the handling of numbers and data within these automated reports. Readers expressed that the way numerical and data-driven content was presented in AI-generated articles was markedly less satisfactory, contributing to the overall challenge in comprehending the articles.
Despite these issues, readers reported no significant differences between automated and manually-written articles concerning the character of the writing, its narrative structure, and flow. This suggests that while AI-generated articles may capture the overarching style of traditional journalism, they fall short in specific aspects of communication, particularly in language and data presentation.
Professor Neil Thurman, who spearheaded this research project, advocated for increased human oversight in the creation and editing of automated news articles. Thurman suggested that greater attention should be paid to reducing the number of complicated numbers presented, clarifying difficult words, and enhancing descriptive language to help readers better grasp the content.
This study marks the first instance where researchers have explored the relative comprehensibility between manual and automated news articles, and it provides insights into why these differences exist. Thäsler-Kordonouri concluded that the findings underscore the necessity of maintaining human involvement in the automatic production of news content and the importance of further refining these automated processes to enhance the comprehensibility of AI-generated news.
Source: Noah Wire Services