8 April 2024 Cascaded cross-attention transformers and convolutional neural networks for multi-organ segmentation in male pelvic computed tomography
Rahul Pemmaraju, Gayoung Kim, Lina Mekki, Daniel Y. Song, Junghoon Lee
Author Affiliations +
Abstract

Purpose

Segmentation of the prostate and surrounding organs at risk from computed tomography is required for radiation therapy treatment planning. We propose an automatic two-step deep learning-based segmentation pipeline that consists of an initial multi-organ segmentation network for organ localization followed by organ-specific fine segmentation.

Approach

Initial segmentation of all target organs is performed using a hybrid convolutional-transformer model, axial cross-attention UNet. The output from this model allows for region of interest computation and is used to crop tightly around individual organs for organ-specific fine segmentation. Information from this network is also propagated to the fine segmentation stage through an image enhancement module, highlighting regions of interest in the original image that might be difficult to segment. Organ-specific fine segmentation is performed on these cropped and enhanced images to produce the final output segmentation.

Results

We apply the proposed approach to segment the prostate, bladder, rectum, seminal vesicles, and femoral heads from male pelvic computed tomography (CT). When tested on a held-out test set of 30 images, our two-step pipeline outperformed other deep learning-based multi-organ segmentation algorithms, achieving average dice similarity coefficient (DSC) of 0.836±0.071 (prostate), 0.947±0.038 (bladder), 0.828±0.057 (rectum), 0.724±0.101 (seminal vesicles), and 0.933±0.020 (femoral heads).

Conclusions

Our results demonstrate that a two-step segmentation pipeline with initial multi-organ segmentation and additional fine segmentation can delineate male pelvic CT organs well. The utility of this additional layer of fine segmentation is most noticeable in challenging cases, as our two-step pipeline produces noticeably more accurate and less erroneous results compared to other state-of-the-art methods on such images.

© 2024 Society of Photo-Optical Instrumentation Engineers (SPIE)
Rahul Pemmaraju, Gayoung Kim, Lina Mekki, Daniel Y. Song, and Junghoon Lee "Cascaded cross-attention transformers and convolutional neural networks for multi-organ segmentation in male pelvic computed tomography," Journal of Medical Imaging 11(2), 024009 (8 April 2024). https://doi.org/10.1117/1.JMI.11.2.024009
Received: 25 April 2023; Accepted: 20 March 2024; Published: 8 April 2024
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Image segmentation

Image enhancement

Transformers

Education and training

Head

Computed tomography

Prostate

Back to Top