Brain metastases are the most common malignant form of tumors and occur in 10%-30% of adult patients with systematic cancer. With recent advances in treatment options, there is an increasing evidence that automated detection and segmentation from MRI can assist clinicians for diagnosis and therapy planning. In this study, we investigate the impact of data domain on self-supervised learning (SSL) for pretraining a deep learning network to detect and segment brain metastases on 3D post-contrast T1-weighted images. We performed pretraining a 3D patch-based U-Net using the Model Genesis framework on three subject cohorts that have different data domain. The pretrained networks were then finetuned on brain MR scans from patients with metastases as a downstream task dataset. We analyzed the impact of data domain on SSL by examining validation metric evolution, FROC analyses and testing performance of early-trained models and best-validated models. Our results suggested that, in the early stage of finetuning for the target task, SSL is crucial for faster training convergence and similar data domain on SSL could be helpful to attain improved detection and segmentation performance earlier. However, we observed that the importance of data domain similarity for SSL progressively diminished as training continued with sufficient amount of iterations in our relatively large data regime. After training convergence, the best-validated models pretrained with SSL provided enhanced detection performance over the model without pretraining regardless of data domain.
Recent technological advances in deep learning (DL) have led to more accurate brain metastasis (BM) detection. As a data driven approach, DL’s performance highly relies on the size and quality of the training data. However, collecting large amount of medical data is costly, and it’s difficult to include BMs with various locations, sizes, and structures etc. Thus, we propose a 3D-2D GAN for fully 3D BM synthesis with configurable parameters. First, two 3D networks are used to synthesize the mask and quantized intensity map of a lesion from 3 concentric spheres, which are used to control the lesion’s location, size and structure. Then, a 2D network is used to synthesize the final lesion with proper appearance from the quantized intensity map and the background MR image. With this 3D-2D design, the 3D networks enable the synthetic metastasis to be spatially continuous in all 3 dimensions through the guidance of the 3D intermediate presentation of the lesion, while the 2D network enables the use of 2D perceptual loss to make the final synthesized lesion look realistic. In addition, different network up-sampling strategies and postprocessing are used to control the heterogeneity and contrast of the synthetic lesion. All the synthesized images were reviewed by a radiologist. The indistinguishability rate of the synthesized lesion is above 70%. The configurable parameters for the lesion’s location, size, and structure, heterogeneity and contrast were reviewed to be effective. Our work demonstrates the feasibility of synthesizing configurable 3D BM lesions for fully 3D data augmentation.
KEYWORDS: Positron emission tomography, Magnetic resonance imaging, Point spread functions, Tissues, Deconvolution, Convolution, Brain, 3D modeling, Monte Carlo methods, Neuroimaging
Accurate quantification of positron emission tomography (PET) is important for diagnosis and assessment of cancer
treatment. The low spatial resolution of PET imaging induces partial volume effect to PET images that biases
quantification. A PET partial volume correction method is proposed using high-resolution, anatomical information from
magnetic resonance images (MRI). The corrected PET is pursued by removing the convolution of PET point spread
function (PSF) and by preserving edges present in PET and the aligned MR images. The correction is implemented in a
Bayesian's deconvolution framework that is minimized by a conjugate gradient method. The method is evaluated on
simulated phantom and brain PET images. The results show that the method effectively restores 102 ± 7% of the true
PET activity with a size of greater than the full-width at half maximum of the point spread function. We also applied the
method to synthesized brain PET data. The method does not require prior information about tracer activity within tissue
regions. It can offer a partial volume correction method for various PET applications and can be particularly useful for
combined PET/MRI studies.
We are developing and evaluating choline molecular imaging with positron emission tomography
(PET) for monitoring tumor response to photodynamic therapy (PDT) in animal models. Human
prostate cancer (PC-3) was studied in athymic nude mice. A second-generation photosensitizer
Pc 4 was used for PDT in tumor-bearing mice. MicroPET images with 11C-choline were acquired
before PDT and 48 h after PDT. Time-activity curves of 11C-choline uptake were analyzed before
and after PDT. For treated tumors, normalized choline uptake decreased significantly 48 h after
PDT, compared to the same tumors pre-PDT (p ⪅ 0.001). However, for the control tumors,
normalized choline uptake increased significantly (p ⪅ 0.001). PET imaging with 11C-choline is
sensitive to detect early tumor response to PDT in the animal model of human prostate cancer.
We are developing MRI-based attenuation correction methods for PET images. PET has high sensitivity but
relatively low resolution and little anatomic details. MRI can provide excellent anatomical structures with high
resolution and high soft tissue contrast. MRI can be used to delineate tumor boundaries and to provide an anatomic
reference for PET, thereby improving quantitation of PET data. Combined PET/MRI can offer metabolic, functional
and anatomic information and thus can provide a powerful tool to study the mechanism of a variety of diseases.
Accurate attenuation correction represents an essential component for the reconstruction of artifact-free, quantitative
PET images. Unfortunately, the present design of hybrid PET/MRI does not offer measured attenuation correction
using a transmission scan. This problem may be solved by deriving attenuation maps from corresponding anatomic
MR images. Our approach combines image registration, classification, and attenuation correction in a single scheme.
MR images and the preliminary reconstruction of PET data are first registered using our automatic registration
method. MRI images are then classified into different tissue types using our multiscale fuzzy C-mean classification
method. The voxels of classified tissue types are assigned theoretical tissue-dependent attenuation coefficients to
generate attenuation correction factors. Corrected PET emission data are then reconstructed using a threedimensional
filtered back projection method and an order subset expectation maximization method. Results from
simulated images and phantom data demonstrated that our attenuation correction method can improve PET data
quantitation and it can be particularly useful for combined PET/MRI applications.
We are developing in vivo small animal imaging techniques that can measure early effects of photodynamic therapy
(PDT) for prostate cancer. PDT is an emerging therapeutic modality that continues to show promise in the treatment
of cancer. At our institution, a new second-generation photosensitizing drug, the silicon phthalocyanine Pc 4, has been
developed and evaluated at the Case Comprehensive Cancer Center. In this study, we are developing magnetic
resonance imaging (MRI) techniques that provide therapy monitoring and early assessment of tumor response to PDT.
We generated human prostate cancer xenografts in athymic nude mice. For the imaging experiments, we used a highfield
9.4-T small animal MR scanner (Bruker Biospec). High-resolution MR images were acquired from the treated
and control tumors pre- and post-PDT and 24 hr after PDT. We utilized multi-slice multi-echo (MSME) MR
sequences. During imaging acquisitions, the animals were anesthetized with a continuous supply of 2% isoflurane in
oxygen and were continuously monitored for respiration and temperature. After imaging experiments, we manually
segmented the tumors on each image slice for quantitative image analyses. We computed three-dimensional T2 maps
for the tumor regions from the MSME images. We plotted the histograms of the T2 maps for each tumor pre- and
post-PDT and 24 hr after PDT. After the imaging and PDT experiments, we dissected the tumor tissues and used the
histologic slides to validate the MR images. In this study, six mice with human prostate cancer tumors were imaged
and treated at the Case Center for Imaging Research. The T2 values of treated tumors increased by 24 ± 14% 24 hr
after the therapy. The control tumors did not demonstrate significant changes of the T2 values. Inflammation and
necrosis were observed within the treated tumors 24 hour after the treatment. Preliminary results show that Pc 4-PDT
is effective for the treatment of human prostate cancer in mice. The small animal MR imaging provides a useful tool
to evaluate early tumor response to photodynamic therapy in mice.
We are investigating in vivo small animal imaging and analysis methods for the assessment of photodynamic therapy
(PDT), an emerging therapeutic modality for cancer treatment. Multiple weighted MR images were acquired from
tumor-bearing mice pre- and post-PDT and 24-hour after PDT. We developed an automatic image classification method
to differentiate live, necrotic and intermediate tissues within the treated tumor on the MR images. We used a multiscale
diffusion filter to process the MR images before classification. A multiscale fuzzy C-means (FCM) classification method
was applied along the scales. The object function of the standard FCM was modified to allow multiscale classification
processing where the result from a coarse scale is used to supervise the classification in the next scale. The multiscale
fuzzy C-means (MFCM) method takes noise levels and partial volume effects into the classification processing. The
method was validated by simulated MR images with various noise levels. For simulated data, the classification method
achieved 96.0 ± 1.1% overlap ratio. For real mouse MR images, the classification results of the treated tumors were
validated by histologic images. The overlap ratios were 85.6 ± 5.1%, 82.4 ± 7.8% and 80.5 ± 10.2% for the live, necrotic,
and intermediate tissues, respectively. The MR imaging and the MFCM classification methods may provide a useful tool
for the assessment of the tumor response to photodynamic therapy in vivo.
We are investigating imaging techniques to study the tumor response to photodynamic therapy (PDT). PET can provide physiological and functional information. High-resolution MRI can provide anatomical and morphological changes. Image registration can combine MRI and PET images for improved tumor monitoring. In this study, we acquired high-resolution MRI and microPET [18F]fluorodeoxyglucose (FDG) images from C3H mice with RIF-1 tumors that were treated with Pc 4-based PDT. For tumor registration, we developed a finite element model (FEM)-based deformable registration scheme. To assess the registration quality, we performed slice by slice review of both image volumes, computed the volume overlap ratios, and visualized both volumes in color overlay. The mean volume overlap ratios for tumors were 94.7% after registration. Registration of high-resolution MRI and microPET images combines anatomical and functional information of the tumors and provides a useful tool for evaluating photodynamic therapy.
High intensity focused ultrasound (HIFU) is a promising method for ablation therapy in the heart. Little is understood about early lesion development with HIFU because the lesions cannot be imaged reliably with sufficient resolution, and no other real time monitoring techniques are available to date. We investigated Optical coherence tomography (OCT) for monitoring early lesion formation. We created a series of lesions in fresh canine cardiac tissue using 5W (frequency=4.23Mhz, F#=1.2) of acoustic power with 10sec., 7sec., and 5sec. exposures. The lesions were then imaged using an OCT imaging system with an axial resolution of 12μm and a lateral resolution of 15μm. The maximum width of the lesions were measured using custom software. In separate experiments, lesion formation was investigated under varying acoustic power levels ranging from 5W to 20W at 0.1sec. and 0.2 sec. exposures. The average maximum widths of the lesions were 1.06mm for 10sec. lesions, .65mm for 7sec. lesions, and .59mm for 5sec. lesions. We observed both subsurface lesions and superficial blister-like formations, which may be a precursor of cavitation inception or tissue vaporization. The subsurface lesion forms over time as expected from thermal energy deposition. The surface blister forms prior to the subsurface lesion at high power, and after subsurface lesion formation at lower powers. OCT provides a method for monitoring HIFU lesion formation at high resolution, and can potentially be used to optimize HIFU dose for clinical applications.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.