A Novel Multimodal Deep Learning Approach for Polycystic Ovary Syndrome (PCOS) Diagnosis with Explainable AI

Abstract

Problem: PCOS is a very common endocrine disorder in reproductive-age females characterized by complex symptoms of hormonal imbalance, irregular menstrual cycle, and infertility. The traditional diagnosing methods include clinical examinations and ultrasound imaging, which are often inconsistent and lead to misdiagnosis and delayed treatment for the disease. Therefore, this study presents a novel multimodal system for PCOS diagnosis that integrates clinical data and ultrasound scan images to enhance its diagnosing accuracy and reliability. Additionally, Explainable AI (XAI) techniques are integrated to provide transparent decision-making, enabling users to understand and trust the model’s predictions.

Methodology: The proposed methodology involves a novel, multimodal diagnostic model based on deep learning techniques that use both ultrasound images and clinical data for the detection of PCOS. This approach uses CNNs to capture intricate features from ultrasound images and deploy advanced feature engineering on clinical data to optimize predictive performance. A fusion model integrates insights from both data types to diagnose PCOS accurately.

Initial Results: The proposed multimodal system achieved exceptional diagnostic performance, with the ultrasound image model reaching 99.8% accuracy and the clinical data model reaching 97% accuracy. By integrating both modalities, the combined approach delivered 98.4% accuracy, significantly outperforming multimodal approaches. The effectiveness of the multimodal approach in utilizing both clinical and ultrasound scan image data for reliable diagnosis was confirmed by its outstanding performance.