Abstract:
Problem: Cardiac Magnetic Resonance Imaging analysis is important for diagnosing
cardiovascular diseases. However, it is prone to variability and time-consuming. Some of the
existing artificial intelligence systems support interpretability, but not both interpretability and
natural language reports. This reduces the probability of adopting these solutions in clinical
settings. This research project addresses limitations by developing an interpretable artificial
intelligence system that can analyze cardiac Magnetic Resonance Images accurately and
generate a natural language report of findings.
Methodology: I used a Shifted Window Transformer with a classification head for cardiac
magnetic resonance image segmentation and classification, trained on a small dataset of left ventricle images from the Cardiac Atlas Project. To generate natural language reports, I
integrated it with Generative Pre-trained Transformer Omni. For interpretability, I applied
Gradient-weighted Class Activation Mapping to visualize influential regions in the images.
Findings: The deep learning model achieved a segmentation loss of about 0.58 on the validation
set, with the classification module reaching a prediction confidence of approximately 53%.
Although limited and imbalanced data affected its accuracy, the model functioned as a core
element of the system. The pipeline, featuring Gradient-weighted Class Activation Mapping
for interpretability and Generative Pre-trained Transformer for reporting, showed clinical
relevance in case studies. These results support integrating interpretable deep learning with
automated reporting to enhance the transparency and usability of Cardiac Magnetic Resonance
Imaging analysis tools.
