A collaborative research initiative involving over 350 professionals from 58 countries has produced a comprehensive set of recommendations aimed at promoting inclusivity in datasets employed for training medical artificial intelligence (AI) systems. This initiative, named the STANDING Together project, appears in a report published in The Lancet Digital Health, highlighting the potential for these systems to deliver safe and effective healthcare solutions accessible to all members of society.
The STANDING Together project utilised a Delphi approach, enhanced by a public consultation and an international interview study, to gather input from global experts and establish 29 consensus recommendations. The project has explicitly divided its findings into two sets of recommendations: the first comprising 18 guidelines aimed at improving the documentation of health datasets, and the second consisting of 11 recommendations to enhance the utilisation of these datasets.
The initial recommendations focused on assisting dataset curators in fostering transparency regarding the composition and limitations of health datasets. Notable suggestions include providing a summary of the dataset in plain language, elucidating the dataset’s creation background and purpose, revealing the origin of the data, and detailing the demographics represented, including any known missing groups and the reasons for their absence.
The second set of recommendations seeks to address algorithmic bias within health datasets. This involves ensuring sufficient dataset information to support traceability and auditability, identifying contextualised groups who may face risks from AI technologies, and justifying the appropriate application of datasets to their intended populations. Furthermore, reporting on the performance of AI health technologies for specific groups of interest is suggested for enabling comparisons with aggregate performance metrics.
Dr Xiao Liu, an associate professor of AI and Digital Health Technologies at the University of Birmingham, served as the chief investigator for the study. He stressed the potential for data to “magnify societal biases” and emphasised the importance of addressing root causes to foster health equity, stating, “To create lasting change in health equity, we must focus on fixing the source, not just the reflection.”
The trends surrounding AI implementation in healthcare also point to challenges pertaining to regulation and the management of bias. A discussion panel held last year under the banner of HTN Now featured industry specialists exploring these themes, including Puja Myles from the MHRA, Shanker Vijayadeva from NHS England, and Ricardo Baptista Leite from HealthAI. They addressed inquiries surrounding responsible AI practices and the necessity for inclusive and equitable AI technologies, particularly pertaining to their deployment in the NHS and their potential role in transitioning from reactive to proactive health care.
A recent poll conducted on LinkedIn revealed that among over 100 participants, a majority of 52 percent identified regulation as their primary concern regarding AI in healthcare. Additional questions surrounding the barriers to responsible AI yielded a notable focus on inadequate regulation, which garnered 38 percent of the votes, whilst a lack of standardised data followed closely at 33 percent.
In parallel, Health Level Seven International (HL7) has circulated new guidelines concerning the AI and machine learning data lifecycle to aid developers in enhancing the quality and trustworthiness of interoperable data for AI applications. This guidance includes case studies illustrating AI’s influence on healthcare decision-making amid various patient scenarios.
Moreover, NHS England has released advisory content on evaluating AI projects, drawing lessons from the four-year ‘Artificial Intelligence in Health and Care Award’ programme, designed to facilitate the development and deployment of promising AI technologies. Concurrently, NICE has introduced a new reporting standard intended to bolster transparency in cost-effectiveness analyses of AI treatments, aiming to expedite patient access to promising AI-enabled healthcare solutions.
Source: Noah Wire Services