AI model outperforms human endoscopists in diagnostic tests

AI Model Outperforms Human Endoscopists in Diagnostic Tests

Recently, a team of researchers detailed the development and performance evaluation of an artificial intelligence (AI) model designed to classify small intestinal lesions using capsule endoscopy (CE) images. By utilising a dataset of 33,290 CE images, the researchers trained their neural network to identify 12 types of intestinal lesions with high accuracy. The initial model training showed promising results with a rapid decline in loss function, indicating the AI's swift grasp of the data characteristics. Over time, the learning process stabilised, warding off risks of overfitting.

Performance Evaluation

Five AI models, including DenseNet121, EfficientNetB2, ResNet50, HRNet-w18, and MobileNetv3-large, were evaluated on a validation set of 6,652 images. MobileNetv3-large emerged as the top performer with an impressive accuracy rate of 92.44%, along with high precision, recall, and F1-score figures. This placed it ahead of the efficient EfficientNetB2 model, which also scored well but fell slightly behind across all metrics.

For the final test, MobileNetv3-large was assessed against a test set of 1,013 CE images. The model demonstrated excellent diagnostic capabilities, particularly in categories such as “Lymphangiectasia” and “Foreign Body.” However, it showed challenges in more complex classifications like “Polyp” and “Erosion.” Such discrepancies were attributed to overlapping features between categories and external image quality factors like blurriness and reflections.

Comparison with Human Expertise

Remarkably, when put to the test against human endoscopists of varying experience levels, MobileNetv3-large not only displayed superior diagnostic accuracy but also excelled in speed. The AI model achieved an accuracy of 87.17%, outperforming both junior (75.88%) and senior (84.81%) endoscopists, as verified by a significant statistical analysis (χ² = 48.98, P < 0.05). Moreover, the AI completed the diagnostic task notably faster, analyzing 1,013 images in just 13.5 seconds, in stark contrast to the considerably longer durations taken by human experts.

Interpretable AI Decisions

Despite its impressive performance, understanding the AI model’s decision-making process is crucial for clinical applications. With the help of Grad-CAM and SHAP analysis techniques, researchers visualised how certain features were weighted during the model's classifications. For instance, the AI accurately identified "Bleeding" and "Stenosis" lesions by focusing on key image areas highlighted in red on activation maps, demonstrating precise predictive logic.

Potential Clinical Implications

For broader clinical application, the MobileNetv3-large model was converted into an ONNX format and integrated with a user-friendly graphical user interface (GUI). This allows medical professionals, regardless of their programming skills, to harness the power of AI in real-time diagnostic scenarios. The GUI, created using PyQt5 technology, facilitates efficient batch processing of CE images, promising enhanced speed and accuracy in medical diagnostics.

Conclusion

As demonstrated, artificial intelligence presents a potent tool in medical diagnostics, with models like MobileNetv3-large showing potential to significantly improve diagnostic accuracy and speed over traditional human methods. By addressing the nuances of image classification and providing meaningful interpretations of AI decisions, this advancement holds substantial promise for clinical enhancements and patient care improvements. The future of AI in endoscopy may offer an invaluable resource, especially as technological developments continue in medical imaging.

Source: Noah Wire Services