A Gated Transformer-Based U-Net Model for Cardiac MRI Segmentation

Abstract

"The study focuses on enhancing cardiac MRI segmentation using attention-based medical image segmentation models, particularly gated transformer architectures. It addresses the limitations of traditional convolutional neural networks (CNNs) in capturing vital long-range spatial dependencies and global contextual information necessary for accurate segmentation of cardiac MRI images.

A gated transformer-based U-Net architecture, integrating attention mechanisms, was developed to overcome these limitations. This novel approach facilitates better feature extraction and handling of intricate structures, resulting in improved segmentation performance.

The researchers evaluated their gated transformer-based U-Net model using the MnMs-2 dataset and compared its performance with conventional CNN-based models. They observed that employing a cosine annealing rate scheduler proved more effective than ReduceLROnPlateau in facilitating the convergence of transformer-based architectures. Moreover, introducing the gating mechanism into attention layers enhanced the transformer-based U-Net architecture's performance by a minimum of 2%.

These findings underline the potential of attention mechanisms and gated transformer architectures in advancing the accuracy and consistency of cardiac MRI segmentation. The study highlights the efficacy of this approach in addressing labor-intensive, inconsistent, and error-prone manual or semi-automatic segmentation methods. Overall, the research emphasizes the promising role of these advanced techniques in improving the evaluation of cardiac function and morphology through CMRI."