Percutaneous liver ablation is a minimally invasive procedure to treat liver tumors. Postablation images are highly significant as they distinguish normal post-procedure changes from abnormalities, preventing unnecessary retreatment and confirming procedural quality. However, the cancer surveillance imaging reports after the procedure can be numerous and challenging to read. Moreover, annotated data is limited in this setting. In this study we used the cutting-edge large language model Llama 2 to automatically extract critical findings from real-world diagnostic imaging reports without the need of training a new information extraction model. This could potentially automate part of the outcome research and registry construction process, as well as decrease the number of studies needed to review for research purposes. A dataset of 87 full-text reports from 13 patients who underwent percutaneous thermal ablation for pancreatic liver metastases were used to benchmark the capability of Llama 2 for cancer progression finding extraction and classification. We asked Llama 2 to determine whether there is cancer progression within the given report and then classify progression findings into Local Tumor Progression (LTP), Intrahepatic Progression (IHP) and Extrahepatic Progression (EHP). Llama 2 achieved decent performance for detecting progression at study level. The precision is 0.91 and recall is 0.96, with specificity 0.84. However, the classification of progression into LTP, IHP and EHP still needs to be improved.
Due to the high intricacy and inter-patient variability of liver vascular anatomy, planning, and execution of liver resection is challenging. Currently, intraoperative ultrasound (IOUS) is an indispensable imaging modality in the surgical workflow; however, 2D-US imaging modality can be difficult to interpret due to noise and speckle. Determining the exact location of tumors and identifying critical structures to preserve during hepatectomy demands expertise and advanced skills. An AI-based model that can help identify vessels (inferior vena cava (IVC), right hepatic vein (RHV), left hepatic vein (LHV), and middle hepatic vein (MHV)) for real-time IOUS navigation can be of immense value. In this research work, we describe our visual saliency approach that integrates attention blocks into a U-Net model for real-time liver vessel segmentation. The IOUS dataset contains video recordings derived from 12 patients, procured during liver surgery. Experiments involve analyzing video frames using a leave-one-out crossvalidation (LOOCV) approach. To maintain objectivity, strict separation is ensured between training and testing subsets to prevent the concurrent inclusion of the same patients. Additionally, to assess model robustness, we kept video data from two distinct patients in the withheld test dataset. Our proposed DL model achieved a mean dice score of 0.88, 0.72, 0.53, and 0.78 for IVC, RHV, MHV, and LHV respectively using the LOOCV approach. In the future, this research will be extended for real-time segmentation of all vasculature in the liver to include portal vein anatomy, followed by the translation of our model in the operation room during surgery.
Microwave ablation (MWA) is an effective minimally invasive therapy for treating liver cancers, among various local cancer treatments. Computational studies are crucial in simulating MWA, offering insights that may be unreachable from experimental methods. This study investigated the complex relationships between blood perfusion rate and metabolic heat concerning MWA outcomes. 3D patient-specific finite element models are employed, shedding light on the interplay of these parameters and their impact on the efficacy of MWA procedures. Image data from five patients treated with MWA are chosen, creating detailed 3D models of the liver, tumor, and vasculature. Simulations are performed using a triaxial antenna operating at 2.45 GHz, with a standard ablation time of 10 minutes and an input power of 65 Watts. In addition, the microwave antenna mimics the clinical insertion path in each case. The simulation model encompasses the coupled electromagnetic field and bioheat transfer, comprehensively understanding the underlying dynamics. The simulations contain seven distinct blood perfusion rates, both with and without considering metabolic heat. This variation allows for a thorough exploration of their combined impact on tissue damage and tumor destruction throughout MWA therapy. These findings underscore the intricate interplay of factors influencing the outcomes of MWA procedures, emphasizing the importance of comprehensive modeling that incorporates various parameters for accurate predictions.
Image guided percutaneous thermal ablation is widely used for patients with primary or secondary liver tumors who do not qualify for surgical resection. The COVER-ALL study is a randomized Phase II clinical trial that evaluates the impact of using software aid in confirming probe position and ablation coverage. Current practice in the trial involves acquisition of a pre-procedure contrast enhanced computed tomography (CECT) scan for gross tumor volume (GTV) definition and non-contrast CT after probe placement, followed by a biomechanical model-based deformable image registration (Morfeus) between the two scans to map the GTV onto the non-contrast CT for position confirmation. CT scan length that covers the entire liver is needed for Morfeus. In this work, we investigated an alternative workflow with a reduced length non-contrast CT using image padding on the first 50 COVER-ALL trial patients. The full-length non-contrast CT was first cropped to a fixed thickness, ranging from 2.5-7.5 cm, along the GTV. The remaining volume was padded with the CECT based on intensity-based deformable image registration (DIR). Morfeus DIR was performed between the CECT and resultant padded non-contrast CT to map the GTV segmentation from CECT to padded non-contrast CT. The GTV mapping results were compared to the original GTV mapping results performed on the full-length CT. The median mapping differences using cropping thickness of 7.5 cm was 1.2 (0.5-2.3) mm, with only 3 cases having larger than 5 mm discrepancies. The comparable DIR performance suggests the feasibility of acquiring a reduced-length non-contrast CT to maintain image registration accuracy.
Liver tumor involvement by either primary or secondary cancers is responsible for over 1 million deaths per year worldwide. Image-guided percutaneous thermal ablation (PTA) has become a widely utilized option for patients not eligible for surgery, demonstrating similar 5-year overall survival rates between surgery and PTA. Achieving a 5 mm ablation margin has been shown to correlate with improved survival, however, achieving accurate needle placement and confirming sufficient ablation is challenging in the presence of liver deformation, needle artifacts, and inability to distinguish between the tumor boundary and ablation region post-PTA. This presentation will describe data demonstrating the need for accurate ablation measurement for improved outcomes, the emerging role of deep learning to provide segmentation of the liver, tumor, and ablation region, and the advances in precision of targeting the tumor and assessing the outcomes of the PTA through the use of biomechanical modeling of the liver.
Purpose Investigate and evaluate the accuracy of deep learning (DL)-based segmentation and deformable image registration (DIR) for the automatization of recurrence risk map atlas definition. Materials and methods Twelve patients with visible recurrence on 18F-DCFPyL PET/CT after prostatectomy were retrospectively analyzed. The bladder, rectum, iliac arteries and veins, and recurrence sites were manually delineated. A previously trained DL model for female pelvic anatomy was re-optimized for male to automatically segment the anatomical regions of interest (ROI). Inter-patient registration was investigated using 4 registration methods: rigid, B-Spline Plastimatch, intensity DIR, and a hybrid intensity-based DIR with varying number of controlling ROI. Performance of the methods were reported using contour-based metrics, determinant of the Jacobian, contour variability in term of volume and position, and probability of overlap with the template organs. Results Transfer learning of the DL model provided greater accuracy for the bladder and rectum than for new structures such as iliac arteries and veins with average Dice similarity coefficient ranges of 0.82-0.96 and 0.63-0.77, respectively. Compared to intensity only DIR, hybrid intensity-based DIR with controlling ROI provided better contour-based metrics, determinant of Jacobian, and less incidence of overlap between recurrence sites and template organs. Centroid position variability between the registration approaches were reported with average range of 1.6-11.3 mm and up to 5.7-30 mm. Conclusion DL and hybrid DIR models can be used to automatize inter-patient registration in the definition of population-based recurrence risk map. DIR uncertainties in the propagation of the recurrence between patients need to be carefully verified before being used in population-based model.
KEYWORDS: Computed tomography, Image registration, Magnetic resonance imaging, 3D image processing, Image segmentation, Tissues, In vivo imaging, Optical coherence tomography, Positron emission tomography, Medical imaging
Histopathology is the accepted gold standard for identifying cancerous tissues. Validation of in vivo imaging signals with precisely correlated histopathology can potentially improve the delineation of tumors in medical images for focal therapy planning, guidance, and assessment. Registration of histopathology with other imaging modalities is challenging due to soft tissue deformations that occur between imaging and histological processing of tissue. In this paper, a framework for precise registration of medical images and pathology using white-light images (photographs) is presented. A euthanized normal mouse was imaged using four imaging modalities: CBCT, PET-CT, MRI and micro CT. The mouse was then fixed in an embedding medium, optical cutting temperature (OCT) compound, with co-registration markers and sliced at 50 m intervals in a cryostatmicrotome. The device automatically photographed each slice with a mounted camera and reconstructed the 3D white-light image of the mouse through co-registering of consecutive slices. The white-light image was registered to the four imaging modalities based on the external contours of the mouse. Six organs (brain, liver, stomach, pancreas, kidneys and bladder) were contoured on the MR image while the skeletal structure and lungs were segmented on the CBCT image. The contours of these structures were propagated to the additional imaging modalities based on the registrations to the white-light image and were analyzed qualitatively by developing an anatomical atlas of normal mouse defined using three imaging modalities. This work will serve as the foundation to include histopathology through the transfer of the imaged slice onto tape for histological processing.
The purpose of this research is to improve treatment of colorectal liver metastases (CLM) in the clinic. It has been previously shown that an ablation margin of 5 mm or more for CLM greatly increases 5 year local tumor progression free survival, however it is often difficult to ensure proper ablation using intraprocedural imaging. CT images of 30 patients with CLM treated with ablation were retrospectively obtained from the MD Anderson Cancer Center. Contours defining the liver, ablation probes, CLM margins, and ablation margin were created from the pre-treatment contrast enhanced CTs and intra-interventional CT images. Using a biomechanical model-based deformable image registration these contours were deformed onto the contrast enhanced CT images obtained just after treatment. The propagated ablation region was then compared with the GTV, as defined before the procedure, to determine the ablation margin delivered. There was a statistically significant difference (p<0.01) in the achieved ablation margin between patients who did and did not have local recurrence. Results showed that patients without local recurrence received on average 3.19 mm of minimum ablation margin around the gross tumor volume(GTV), while those with local recurrence received an average of 1.14 mm. The model presented can assist in the treatment of CLM by identifying the minimum distance to agreement between the GTV and the ablation region directly after treatment. This metric can help determine if sufficient ablation has been delivered to the treat the disease.
Deformable Image Registration (DIR) has been extensively studied over the past two decades due to its essential role in many image-guided interventions. Morfeus is a DIR algorithm that works based on finite element biomechanical modeling. However, Morfeus does not utilize the entire image contrast and features which could potentially lead to a more accurate registration result. A hybrid biomechanical intensity-based method is proposed to investigate this potential benefit. Inhale and exhale 4DCT lung images of 26 patients were initially registered using Morfeus by modeling contact surface between the lungs and the chest cavity. The resulting deformations using Morfeus were refined using a B-spline intensity-based algorithm (Drop, Munich, Germany). Important parameters in Drop including grid spacing, number of pyramids, and regularization coefficient were optimized on 10 randomly-chosen patients (out of 26). The remaining parameters were selected empirically. Target Registration Error (TRE) was calculated by measuring the Euclidean distance of common anatomical points on both images before and after registration. For each patient a minimum of 30 points/lung were used. The Hybrid method resulted in mean±SD (90th%) TRE of 1.5±1.4 (2.8) mm compared to 3.1±2.0 (5.6) using Morfeus and 2.6±2.6 (6.2) using Drop alone.
High-quality intraoperative 3D imaging systems such as cone-beam CT (CBCT) hold considerable promise for imageguided
surgical procedures in the head and neck. With a large amount of preoperative imaging and planning information
available in addition to the intraoperative images, it becomes desirable to be able to integrate all sources of imaging
information within the same anatomical frame of reference using deformable image registration. Fast intensity-based
algorithms are available which can perform deformable image registration within a period of time short enough for
intraoperative use. However, CBCT images often contain voxel intensity inaccuracy which can hinder registration
accuracy - for example, due to x-ray scatter, truncation, and/or erroneous scaling normalization within the 3D
reconstruction algorithm. In this work, we present a method of integrating an iterative intensity matching step within the
operation of a multi-scale Demons registration algorithm. Registration accuracy was evaluated in a cadaver model and
showed that a conventional Demons implementation (with either no intensity match or a single histogram match)
introduced anatomical distortion and degradation in target registration error (TRE). The iterative intensity matching
procedure, on the other hand, provided robust registration across a broad range of intensity inaccuracies.
Deformable image registration of four head and neck cancer patients was conducted using biomechanical based model.
Patient specific 3D finite element models have been developed using CT and cone beam CT image data of the planning
and a radiation treatment session. The model consists of seven vertebrae (C1 to C7), mandible, larynx, left and right
parotid glands, tumor and body. Different combinations of boundary conditions are applied in the model in order to find
the configuration with a minimum registration error. Each vertebra in the planning session is individually aligned with
its correspondence in the treatment session. Rigid alignment is used for each individual vertebra and to the mandible
since deformation is not expected in the bones. In addition, the effect of morphological differences in external body
between the two image sessions is investigated. The accuracy of the registration is evaluated using the tumor, and left
and right parotid glands by comparing the calculated Dice similarity index of these structures following deformation in
relation to their true surface defined in the image of the second session. The registration improves when the vertebrae
and mandible are aligned in the two sessions with the highest Dice index of 0.86±0.08, 0.84±0.11, and 0.89±0.04 for the
tumor, left and right parotid glands, respectively. The accuracy of the center of mass location of tumor and parotid
glands is also improved by deformable image registration where the error in the tumor and parotid glands decreases from
4.0±1.1, 3.4±1.5, and 3.8±0.9 mm using rigid registration to 2.3±1.0, 2.5±0.8 and 2.0±0.9 mm in the deformable image
registration when alignment of vertebrae and mandible is conducted in addition to the surface projection of the body.
A system for intraoperative cone-beam CT (CBCT) surgical guidance is under development and translation to trials in
head and neck surgery. The system provides 3D image updates on demand with sub-millimeter spatial resolution and
soft-tissue visibility at low radiation dose, thus overcoming conventional limitations associated with preoperative
imaging alone. A prototype mobile C-arm provides the imaging platform, which has been integrated with several novel
subsystems for streamlined implementation in the OR, including: real-time tracking of surgical instruments and
endoscopy (with automatic registration of image and world reference frames); fast 3D deformable image registration (a
newly developed multi-scale Demons algorithm); 3D planning and definition of target and normal structures; and
registration / visualization of intraoperative CBCT with the surgical plan, preoperative images, and endoscopic video.
Quantitative evaluation of surgical performance demonstrates a significant advantage in achieving complete tumor
excision in challenging sinus and skull base ablation tasks. The ability to visualize the surgical plan in the context of
intraoperative image data delineating residual tumor and neighboring critical structures presents a significant advantage
to surgical performance and evaluation of the surgical product. The system has been translated to a prospective trial
involving 12 patients undergoing head and neck surgery - the first implementation of the research prototype in the
clinical setting. The trial demonstrates the value of high-performance intraoperative 3D imaging and provides a valuable
basis for human factors analysis and workflow studies that will greatly augment streamlined implementation of such
systems in complex OR environments.
Patient specific 3D finite element models have been developed to investigate the effect of heterogeneous material
properties on modeling of the deformation of the lungs by including the bronchial trees of each lung. Each model
consists of both lungs, body, tumor, and bronchial trees. Triangular shell elements with 0.1 cm wall thickness are used to
model the bronchial trees. Body, lungs and tumor are modeled using 4-node tetrahedral elements. Experimental test data
are used for the nonlinear material properties of the lungs. Three elastic modulii of 0.5, 10 and 18 MPa are used for the
bronchial tree. Frictionless contact surfaces are applied to lung surfaces and cavities. The accuracy of the results is
examined using an average of 40 bifurcation points. Preliminary results have shown an insignificant effect of modeling
the bronchial trees explicitly on the overall accuracy of the model. However, local changes in the predicted motion of
the bronchial tree of up to 5.2 mm were observed, indicating that modeling the bronchial tree explicitly, with unique
material properties, may ensure a more accurately detailed model of the lung as well as reduced maximum residual
errors.
High-performance intraoperative imaging is essential to an ever-expanding scope of therapeutic procedures ranging from
tumor surgery to interventional radiology. The need for precise visualization of bony and soft-tissue structures with
minimal obstruction to the therapy setup presents challenges and opportunities in the development of novel imaging
technologies specifically for image-guided procedures. Over the past ~5 years, a mobile C-arm has been modified in
collaboration with Siemens Medical Solutions for 3D imaging. Based upon a Siemens PowerMobil, the device includes:
a flat-panel detector (Varian PaxScan 4030CB); a motorized orbit; a system for geometric calibration; integration with
real-time tracking and navigation (NDI Polaris); and a computer control system for multi-mode fluoroscopy,
tomosynthesis, and cone-beam CT. Investigation of 3D imaging performance (noise-equivalent quanta), image quality
(human observer studies), and image artifacts (scatter, truncation, and cone-beam artifacts) has driven the development
of imaging techniques appropriate to a host of image-guided interventions. Multi-mode functionality presents a valuable
spectrum of acquisition techniques: i.) fluoroscopy for real-time 2D guidance; ii.) limited-angle tomosynthesis for fast
3D imaging (e.g., ~10 sec acquisition of coronal slices containing the surgical target); and iii.) fully 3D cone-beam CT
(e.g., ~30-60 sec acquisition providing bony and soft-tissue visualization across the field of view). Phantom and cadaver
studies clearly indicate the potential for improved surgical performance - up to a factor of 2 increase in challenging
surgical target excisions. The C-arm system is currently being deployed in patient protocols ranging from brachytherapy
to chest, breast, spine, and head and neck surgery.
Magnetic resonance imaging (MRI) with an endorectal receiver coil (ERC) provides superior visualization of the prostate gland and its surrounding anatomy at the expense of large anatomical deformation. The ability to correct for this deformation is critical to integrate the MR images into the CT-based treatment planning for radiotherapy. The ability to quantify and understand the physiological motion due to large changes in rectal filling can also improve the precision of image-guided procedures. The purpose of this study was to understand the biomechanical relationship between the prostate, rectum, and bladder using a finite element-based multi-organ deformable image registration method, 'Morfeus' developed at our institution. Patients diagnosed with prostate cancer were enrolled in the study. Gold seed markers were implanted in the prostate and MR scans performed with the ERC in place and its surrounding balloon inflated to varying volumes (0-100cc). The prostate, bladder, and rectum were then delineated, converted into finite element models, and assigned appropriate material properties. Morfeus was used to assign surface interfaces between the adjacent organs and deform the bladder and rectum from one position to another, obtaining the position of the prostate through finite element analysis. This approach achieves sub-voxel accuracy of image co-registration in the context of a large ERC deformation, while providing a biomechanical understanding of the multi-organ physiological relationship between the prostate, bladder, and rectum. The development of a deformable registration strategy is essential to integrate the superior information offered in MR images into the treatment planning process.
KEYWORDS: Computed tomography, Kidney, Magnetic resonance imaging, In vivo imaging, Iodine, Gadolinium, Liver, Image registration, Medical imaging, Blood
Multimodality imaging has gained momentum in radiation therapy planning and image-guided treatment delivery. Specifically, computed tomography (CT) and magnetic resonance (MR) imaging are two complementary imaging modalities often utilized in radiation therapy for visualization of anatomical structures for tumour delineation and accurate registration of image data sets for volumetric dose calculation. The development of a multimodal contrast agent for CT and MR with prolonged in vivo residence time would provide long-lasting spatial and temporal correspondence of the anatomical features of interest, and therefore facilitate multimodal image registration, treatment planning and delivery. The multimodal contrast agent investigated consists of nano-sized stealth liposomes encapsulating conventional iodine and gadolinium-based contrast agents. The average loading achieved was 33.5 ± 7.1 mg/mL of iodine for iohexol and 9.8 ± 2.0 mg/mL of gadolinium for gadoteridol. The average liposome diameter was 46.2 ± 13.5 nm. The system was found to be stable in physiological buffer over a 15-day period, releasing 11.9 ± 1.1% and 11.2 ± 0.9% of the total amounts of iohexol and gadoteridol loaded, respectively. 200 minutes following in vivo administration, the contrast agent maintained a relative contrast enhancement of 81.4 ± 13.05 differential Hounsfield units (ΔHU) in CT (40% decrease from the peak signal value achieved 3 minutes post-injection) and 731.9 ± 144.2 differential signal intensity (ΔSI) in MR (46% decrease from the peak signal value achieved 3 minutes post-injection) in the blood (aorta), a relative contrast enhancement of 38.0 ± 5.1 ΔHU (42% decrease from the peak signal value achieved 3 minutes post-injection) and 178.6 ± 41.4 ΔSI (62% decrease from the peak signal value achieved 3 minutes post-injection) in the liver (parenchyma), a relative contrast enhancement of 9.1 ± 1.7 ΔHU (94% decrease from the peak signal value achieved 3 minutes post-injection) and 461.7 ± 78.1 ΔSI (60% decrease from the peak signal value achieved 5 minutes post-injection) in the kidney (cortex) of a New Zealand white rabbit. This multimodal contrast agent, with prolonged in vivo residence time and imaging efficacy, has the potential to bring about improvements in the fields of medical imaging and radiation therapy, particularly for image registration and guidance.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.