North Carolina's health care systems are increasingly integrating artificial intelligence (AI) technologies, prompting state leaders to consider the need for regulatory measures to guide the development and use of these rapidly evolving systems. Amidst this rising trend, officials aim to establish policies that might serve as a precedent for AI governance in the health sector.
Artificial intelligence, which utilises vast amounts of data to perform tasks traditionally requiring human intelligence, has seen major advancements, particularly with the emergence of generative AI. This innovative technology allows robots to not only analyse data for predictions or recommendations but also to create new content autonomously.
Within North Carolina, a number of health care providers are pioneering the application of generative AI tools. Notable implementations include ambient documentation systems capable of recording patient interactions and automatically generating clinical notes. As noted by Christina Silcox, research director for digital health at the Duke-Margolis Institute for Health Policy, while the potential for AI to enhance healthcare efficiency is substantial, the challenges it presents are equally significant. Silcox highlighted key concerns surrounding AI, including the risk of algorithms based on biased data, privacy issues stemming from the utilisation of extensive patient information, and the possibility of healthcare professionals over-relying on AI insights instead of their clinical judgement.
Amid these developments, Senator Jim Burgin (R-Angier) is preparing to propose legislation aimed at addressing some of the obstacles associated with the use of AI in health care. He expressed intentions to create a policy framework that could help set a standard for similar efforts in other states. "AI is making all these decisions for us, but if it makes the wrong decision, where’s the liability?" Burgin questioned, underscoring the urgency for accountability in clinical decision-making facilitated by AI technology.
The legislative landscape regarding AI regulation remains sparse at the federal level. As Congress has yet to advance comprehensive federal legislation, the responsibility for regulatory frameworks is beginning to shift to individual states. For instance, Utah recently enacted a law necessitating that state-licensed health professionals inform patients when engaging with generative AI applications. Meanwhile, other states are contemplating regulations designed to mitigate discrimination within AI algorithms.
In preparation for the potential implementation of AI-related policies, North Carolina's health care providers are already constructing their own oversight systems. Eric Poon, Chief Health Information Officer at Duke Health, noted that AI clinical tools undergo a rigorous vetting process involving a committee made up of health system and university experts before being approved for broader use. "We ask project teams to try out the technology, deploy them with the right guardrails and see whether it works before pushing it out to the masses," Poon stated.
Despite these strides toward self-regulation, concerns remain regarding the sufficiency of institutional oversight. The North Carolina Medical Board has articulated that while physicians are encouraged to use AI tools, they hold ultimate responsibility for any clinical decisions derived from AI recommendations. Currently, however, the board does not have plans to establish specific guidance on the use of AI nor has it initiated any disciplinary actions related to it.
As the landscape evolves, the challenge of creating coherent and effective state legislation continues to unfold. David McSwain, UNC Health's Chief Medical Informatics Officer, remarked on the complexities involved in achieving coherent regulations, advocating for a "gold-standard state legislation" that can address the specific needs of health systems while aiming to reduce burdens on providers. The developing situation underscores the critical balance between fostering technological innovation in health care and safeguarding patient rights and equitable treatment.
Source: Noah Wire Services