Hybrid transformer for lesion segmentation on adaptive optics retinal images

Jianfei Liu; Joanne Li; Amday Wolde; Catherine Cukras; Johnny Tam

doi:10.1117/12.2612379

4 April 2022 Hybrid transformer for lesion segmentation on adaptive optics retinal images

Jianfei Liu, Joanne Li, Amday Wolde, Catherine Cukras, Johnny Tam

Proceedings Volume 12033, Medical Imaging 2022: Computer-Aided Diagnosis; 120331W (2022) https://doi.org/10.1117/12.2612379
Event: SPIE Medical Imaging, 2022, San Diego, California, United States

Conference Poster

Abstract

Adaptive optics (AO) retinal imaging has enabled the visualization of cellular-level changes in the living human eye. However, imaging tissue-level lesions with such high resolution introduces unique challenges. At a fine spatial scale, intralesion features can resemble cells, effectively serving as camouflage and making it difficult to delineate the boundary of lesions. The size discrepancy between the tissue-level lesions and retinal cells is also highly variable, ranging from a difference of several-fold to greater than an order-of-magnitude. Here, we introduce a hybrid-transformer based on the combination of a convolutional LinkNet and a fully axial attention transformer network to consider both local and global image features, which excels at identifying tissue-level lesions within a cellular landscape. After training the hybrid transformer on 489 manually-annotated AO images, accurate lesion segmentation was achieved on a separate test dataset consisting of 75 AO images for validation. The segmentation accuracy achieved using the hybrid transformer was superior to the use of convolutional neural networks alone (U-Net and LinkNet) or transformer-based networks alone (AxialDeepLab and Medical Transformer) (p<0.05). These experimental results demonstrate that the combination of convolution and transformer networks are an efficient way to utilize both local and global image features for the purpose of lesion segmentation in medical imaging and may be important for computer-aided diagnosis that relies on accurate lesion segmentation.

Citation Download Citation

Jianfei Liu, Joanne Li, Amday Wolde, Catherine Cukras, and Johnny Tam "Hybrid transformer for lesion segmentation on adaptive optics retinal images", Proc. SPIE 12033, Medical Imaging 2022: Computer-Aided Diagnosis, 120331W (4 April 2022); https://doi.org/10.1117/12.2612379

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available