AI in Healthcare: High Stakes and Potential Risks
Recent Developments and Key Concerns
A significant development in medical technology has come under scrutiny, as artificial intelligence (AI) continues to integrate into the healthcare sector. OpenAI's latest AI model, despite substantial investment reaching billions of dollars, still exhibits fundamental shortcomings. For instance, the model fails to consistently determine the number of 'r' letters in the word "strawberry." This highlights persistent issues of accuracy, colloquially referred to as "hallucinations," where AI systems generate incorrect or misleading information.
While such inaccuracies might be trivial in academic settings, their implications in healthcare are potentially perilous. One of the latest platforms to incorporate AI is MyChart, a communication tool facilitating doctor-patient interactions. Developed to optimise time management for medical professionals, MyChart introduces an AI-powered feature designed to draft responses to patient inquiries. This novel function is already utilised by approximately 15,000 doctors globally.
Case Study: Real-World Impacts
One notable incident reported by the New York Times involved Dr. Vinay Reddy, a family medicine physician at UNC Health. He encountered a significant AI-generated error when the system wrongly assured a patient about her hepatitis B vaccination status, despite the unavailability of vaccination records. Such mistakes raise alarms about the reliability of AI in handling sensitive medical information.
The Underlying Technology
This AI capability leverages GPT-4, OpenAI's advanced large language model, which is known for its role in powering the chatbot, ChatGPT. The AI draws upon extensive resources like medical records and drug prescriptions to draft responses and even mimics the doctor's communication style, potentially blurring the line between human and machine-generated advice.
Ethical and Practical Concerns
Athmeya Jayaram, a bioethics researcher at the Hastings Center, critiques this approach, arguing that it ostensibly aims to free up doctors' time but at the risk of compromising patient interaction quality. Medical professionals are expected to review these AI drafts, yet the possibility of errors slipping through remains a matter of concern.
Evidence of AI Errors
Research corroborates these worries. A July study revealed "hallucinations" in seven out of 116 AI-generated responses via MyChart. Another study highlighted repeated errors by GPT-4 when engaged with patient queries. These findings underscore the critical need for scrutiny in AI applications in healthcare, where even minor inaccuracies could lead to severe consequences.
Transparency and Regulation
A striking factor in this integration is the absence of a mandate requiring AI-generated messages to be explicitly labelled. Patients are often unaware that their medical advice might originate from an algorithm. Jayaram emphasises the ethical dilemma where patients might feel betrayed upon discovering that their perceived personal communication with a doctor was in fact automated.
In conclusion, while AI's role in streamlining healthcare processes shows promising potential, the instances of errors and ethical concerns highlight the necessity for stringent oversight and clear labelling protocols. The blending of AI in doctor-patient communication must carefully balance efficiency with unwavering accuracy and transparency to maintain trust and ensure patient safety.
Source: Noah Wire Services