Enabling rapid and high-quality 3D scene reconstruction in cystoscopy through neural radiance fields

Pengcheng Chen; Nicole M. Gunderson; Andrew Lewis; Jason R. Speich; Michael P. Porter; Eric J. Seibel

doi:10.1117/12.3000828

29 March 2024 Enabling rapid and high-quality 3D scene reconstruction in cystoscopy through neural radiance fields

Pengcheng Chen, Nicole M. Gunderson, Andrew Lewis, Jason R. Speich, Michael P. Porter, Eric J. Seibel

Proceedings Volume 12928, Medical Imaging 2024: Image-Guided Procedures, Robotic Interventions, and Modeling; 129281M (2024) https://doi.org/10.1117/12.3000828
Event: SPIE Medical Imaging, 2024, San Diego, California, United States

Abstract

3D reconstruction of cystoscopy holds substantial value in the observation and guided treatment of urological conditions. 3D models of the bladder, obtained through the reconstruction, can aid physicians in performing a rapid and comprehensive assessment of various conditions, such as detection of bladder cancer and follow-up surveillance throughout the patients’ life. In recent years, significant advancements have been made in 3D reconstruction based on cystoscopy. However, the broader application of these advancements has been hampered by issues such as texture loss, slow computational speed, and susceptibility to interference. In this study, we have achieved, for the first time, a reconstruction of a dynamic cystoscopy scene using Neural Radiance Fields (NeRF). Unlike mesh-based 3D reconstruction, NeRF can restore scenes under conditions where the number of views and features are limited, thereby addressing the problem of texture loss that might arise from traditional 3D reconstruction algorithms. Additionally, to expedite the computation speed of NeRF, we employed Instant-NGP, an open-source software that accelerates NeRF computations using hash encoding. This potentially reduces computation time by a hundredfold, making it several tens of times faster than SfM when camera pose values can be streamed into the NeRF reconstruction. By comparing NeRF and SfM methods, we found that NeRF exhibits stronger resistance to interference than SfM, underscoring the formidable potential of NeRF as an innovative method in the field of endoscopy. In the future of robotic-assisted flexible cystoscopy, NeRF has the potential to provide rapid, robust, and comprehensive recording for remote diagnosis of bladder abnormalities.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Pengcheng Chen, Nicole M. Gunderson, Andrew Lewis, Jason R. Speich, Michael P. Porter, and Eric J. Seibel "Enabling rapid and high-quality 3D scene reconstruction in cystoscopy through neural radiance fields", Proc. SPIE 12928, Medical Imaging 2024: Image-Guided Procedures, Robotic Interventions, and Modeling, 129281M (29 March 2024); https://doi.org/10.1117/12.3000828

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
10 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

3D modeling

Atomic force microscopy

Bladder

Endoscopy

Cystoscopy

Video

Cameras

Show All Keywords

Keywords/Phrases

Search In:

Publication Years