An Interpretable AI System for Cardiac MRI Analysis with Natural Language Report Generation.

Abstract

Problem: Cardiac Magnetic Resonance Imaging analysis is important for diagnosing cardiovascular diseases. However, it is prone to variability and time-consuming. Some of the existing artificial intelligence systems support interpretability, but not both interpretability and natural language reports. This reduces the probability of adopting these solutions in clinical settings. This research project addresses limitations by developing an interpretable artificial intelligence system that can analyze cardiac Magnetic Resonance Images accurately and generate a natural language report of findings. Methodology: I used a Shifted Window Transformer with a classification head for cardiac magnetic resonance image segmentation and classification, trained on a small dataset of left ventricle images from the Cardiac Atlas Project. To generate natural language reports, I integrated it with Generative Pre-trained Transformer Omni. For interpretability, I applied Gradient-weighted Class Activation Mapping to visualize influential regions in the images. Findings: The deep learning model achieved a segmentation loss of about 0.58 on the validation set, with the classification module reaching a prediction confidence of approximately 53%. Although limited and imbalanced data affected its accuracy, the model functioned as a core element of the system. The pipeline, featuring Gradient-weighted Class Activation Mapping for interpretability and Generative Pre-trained Transformer for reporting, showed clinical relevance in case studies. These results support integrating interpretable deep learning with automated reporting to enhance the transparency and usability of Cardiac Magnetic Resonance Imaging analysis tools.