KEYWORDS: Education and training, Signal detection, Imaging systems, Signal attenuation, Breast, Binary data, Information operations, Image restoration, Image processing, Signal processing
PurposeThe objective assessment of image quality (IQ) has been advocated for the analysis and optimization of medical imaging systems. One method of computing such IQ metrics is through a numerical observer. The Hotelling observer (HO) is the optimal linear observer, but conventional methods for obtaining the HO can become intractable due to large image sizes or insufficient data. Channelized methods are sometimes employed in such circumstances to approximate the HO. The performance of such channelized methods varies, with different methods obtaining superior performance to others depending on the imaging conditions and detection task. A channelized HO method using an AE is presented and implemented across several tasks to characterize its performance.ApproachThe process for training an AE is demonstrated to be equivalent to developing a set of channels for approximating the HO. The efficiency of the learned AE-channels is increased by modifying the conventional AE loss function to incorporate task-relevant information. Multiple binary detection tasks involving lumpy and breast phantom backgrounds across varying dataset sizes are considered to evaluate the performance of the proposed method and compare to current state-of-the-art channelized methods. Additionally, the ability of the channelized methods to generalize to images outside of the training dataset is investigated.ResultsAE-learned channels are demonstrated to have comparable performance with other state-of-the-art channel methods in the detection studies and superior performance in the generalization studies. Incorporating a cleaner estimate of the signal for the detection task is also demonstrated to significantly improve the performance of the proposed method, particularly in datasets with fewer images.ConclusionsAEs are demonstrated to be capable of learning efficient channels for the HO. The resulting significant increase in detection performance for small dataset sizes when incorporating a signal prior holds promising implications for future assessments of imaging technologies.
In the medical imaging field, task-based metrics of image quality have been advocated as a mean to evaluate the performance of imaging systems and/or reconstruction algorithms. One such way of obtaining these metrics is through a numerical observer. Although the Bayesian ideal observer is optimal by definition, it is frequently intractable and nonlinear. Therefore, linear approximations to the IO are sometimes used to obtain task-based statistics. The optimal linear observer for maximizing the signal-to-noise ratio (SNR) of the test statistic is the Hotelling Observer (HO). However, the computational cost for obtaining the HO increases with image size and becomes intractable for large scale images. In multimodal data, this further becomes an issue because each additional modality dramatically increases the size of the composite image. An alternative to obtaining the HO is approximating the test statistic using a feed-forward neural network (FFNN). However, these methods of learning the HO have not been evaluated on multi-modal data. In this work, a tractable learned multi-modal observer is implemented. The considered task is a signal-known-statistically/background known statistically binary signal detection task. A stylized operator representing an ultrasound computed tomography imaging system and numerical breast phantoms with speed of sound and attenuation modalities are considered. The considered signal is a microcalcification cluster with a random amplitude. It is demonstrated that the learned HO can closely approximate the HO for the considered task.
KEYWORDS: Signal detection, Lawrencium, Super resolution, Binary data, Information operations, Network architectures, Medical imaging, Surgery, Signal attenuation, Image quality
Purpose: Deep learning-based image super-resolution (DL-SR) has shown great promise in medical imaging applications. To date, most of the proposed methods for DL-SR have only been assessed using traditional measures of image quality (IQ) that are commonly employed in the field of computer vision. However, the impact of these methods on objective measures of IQ that are relevant to medical imaging tasks remains largely unexplored. We investigate the impact of DL-SR methods on binary signal detection performance.
Approach: Two popular DL-SR methods, the super-resolution convolutional neural network and the super-resolution generative adversarial network, were trained using simulated medical image data. Binary signal-known-exactly with background-known-statistically and signal-known-statistically with background-known-statistically detection tasks were formulated. Numerical observers (NOs), which included a neural network-approximated ideal observer and common linear NOs, were employed to assess the impact of DL-SR on task performance. The impact of the complexity of the DL-SR network architectures on task performance was quantified. In addition, the utility of DL-SR for improving the task performance of suboptimal observers was investigated.
Results: Our numerical experiments confirmed that, as expected, DL-SR improved traditional measures of IQ. However, for many of the study designs considered, the DL-SR methods provided little or no improvement in task performance and even degraded it. It was observed that DL-SR improved the task performance of suboptimal observers under certain conditions.
Conclusions: Our study highlights the urgent need for the objective assessment of DL-SR methods and suggests avenues for improving their efficacy in medical imaging applications.
In medical imaging, it is sometimes desirable to acquire high resolution images that reveal anatomical and physiological information to support clinical practice such as diagnosis and image-guided therapies. However, for certain imaging modalities (i.e., magnetic resonance imaging (MRI)), acquiring high resolution images can be a very time-consuming and resource-intensive process. One popular solution recently developed is to create a high resolution version of the acquired low-resolution image by use of deep image super-resolution (DL-SR) methods. It has been demonstrated in literature that deep super-resolution networks can improve the image quality measured by traditional physical metrics such as mean square error (MSE), structural similarity index metric (SSIM) and peak signal-to-noise ratio (PSNR). However, it is not clear how well these metrics quantify the diagnostic value of the generated SR images. Here, a task-based super-resolution (SR) image quality assessment is conducted to quantitatively evaluate the efficiency and performance of DL-SR methods. A Rayleigh task is designed to investigate the impact of signal length and super-resolution network complexity on s binary detection performance. Numerical observers (NOs) including the regularized Hotelling Observer (RHO), the anthropomorphic Gabor channelized observers (Gabor CHO) and the ResNet-approximated ideal observer (ResNet-IO) are implemented to assess the Rayleigh task performance. For the datasets considered in this study, little to no improvement in task performance of the considered NOs due to the considered DL-SR SR networks, despite substantial improvement in traditional IQ metrics.
Image reconstruction algorithms seek to reconstruct a sought-after object from a collection of measurements. However, complete measurements such that an object can be uniquely reconstructed are seldom available. Analysis of the null components of the imaging system can guide both physical design of the imaging system and algorithmic design of reconstruction algorithms to more closely reconstruct the true object. Characterizing the null space of an imaging operator is a computationally demanding task. While computationally efficient methods have been proposed to iteratively estimate the null space components of a single or a small number of images, full characterization of the null space remains intractable for large images using existing methods. This work proposes a novel learning-based framework for constructing a null space projection operator of linear imaging operators utilizing an artificial neural network autoencoder. To illustrate the approach, a stylized 2D accelerated MRI reconstruction problem (for which an analytical representation of the null space is known) was considered. The proposed method was compared to state-of-the-art randomized linear algebra techniques in terms of accuracy, computational cost, and memory requirements. Numerical results show that the proposed framework achieves comparable or better accuracy than randomized singular value decomposition. It also has lower computational cost and memory requirements in many practical scenarios, such as when the dimension of the null space is small compared to the dimension of the object.
KEYWORDS: Systems modeling, Stochastic processes, Imaging systems, 3D image processing, Gallium nitride, 3D metrology, Stereoscopy, Network architectures, Medical imaging, Magnetic resonance imaging
Medical imaging systems are commonly assessed and optimized by use of objective-measures of image quality (IQ) that quantify the performance of an observer at specific tasks. Variation in the objects to-be-imaged is an important source of variability that can significantly limit observer performance. This object variability can be described by stochastic object models (SOMs). In order to establish SOMs that can accurately model realistic object variability, it is desirable to use experimental data. To achieve this, an augmented generative adversarial network (GAN) architecture called AmbientGAN has been developed and investigated. However, AmbientGANs cannot be immediately trained by use of advanced GAN training methods such as the progressive growing of GANs (ProGANs). Therefore, the ability of AmbientGANs to establish realistic object models is limited. To circumvent this, a progressively-growing AmbientGAN (ProAmGAN) has been proposed. However, ProAmGANs are designed for generating two-dimensional (2D) images while medical imaging modalities are commonly employed for imaging three-dimensional (3D) objects. Moreover, ProAmGANs that employ traditional generator architectures lack the ability to control specific image features such as fine-scale textures that are frequently considered when optimizing imaging systems. In this study, we address these limitations by proposing two advanced AmbientGAN architectures: 3D ProAmGANs and Style-AmbientGANs (StyAmGANs). Stylized numerical studies involving magnetic resonance (MR) imaging systems are conducted. The ability of 3D ProAmGANs to learn 3D SOMs from imaging measurements and the ability of StyAmGANs to control fine-scale textures of synthesized objects are demonstrated.
KEYWORDS: Gallium nitride, 3D image processing, Medical imaging, Magnetic resonance imaging, Super resolution, Stochastic processes, Neuroimaging, Medical image reconstruction, Imaging systems, Image restoration
Generative adversarial networks (GANs) have proven useful for several medical imaging tasks, including image reconstruction and stochastic object model generation. Thus far, most of the work with GANs has been constrained to twodimensional images. Considering that medical imaging data are often inherently three-dimensional (3D), a 3D GAN would be a more principled way to synthesize realistic volumes. Training a 3D GAN is both computationally and memory intensive. However, prior work has not considered the anisotropic nature of many medical imaging systems. In this paper, the SlabGAN is proposed to reduce the inefficiencies associated with training a 3D GAN. The SlabGAN uses the progressive GAN architecture extended to 3D, but removes the requirement of the three dimensions being equal sizes. This permits the generation of anisotropic 3D volumes with large x and y dimensions. The SlabGAN is trained on MRI brain images from the fastMRI dataset to generate images of dimension 256×256×16. The x and y dimensions of these images are comparable to previously published results while requiring significantly fewer computational resources to generate. The trained SlabGAN is applicable to tasks such as 3D medical image reconstruction and thin-slice MR super resolution.
In medical imaging systems, task-based metrics have been advocated as a means of evaluating image quality. Mathematical observers are one method of computing such metrics. Although the Bayesian Ideal Observer (IO) is optimal by definition, it is frequently intractable and non-linear. Linear approximations to the IO are sometimes employed to obtain task-based statistics when computing the IO is infeasible. The optimal linear observer for maximizing the SNR of the test statistic is the Hotelling Observer (HO). However, the computational cost for computing the HO increases with image size and becomes intractable for larger images. Channelized methods of reducing the dimensionality of the data before computing the HO have become popular, with efficient channels capable of approximating the HO’s performance at significantly reduced computational cost. State-of-the-art channels have been learned by using an autoencoder (AE) to encode data by employing a known signal template as the desired reconstruction, but the method is dependant on a high-quality estimate of the signal. An alternative to channels is approximating the test statistic directly using a feed-forward neural network (FFNN). However, this approach can overfit when the amount of training data is limited. In this work, a generalized method for learning channels utilizing an AE with dual losses (AEDL) is proposed. The AEDL framework jointly minimizes both task-specific and reconstruction losses to learn a set of efficient channels, even when the number of training images is relatively small. Preliminary results indicate that the proposed network outperforms state-of-the-art methods on the selected imaging task. Additionally, the AEDL framework suffers from less overfitting than the FFNN.
Task-based measures of image quality (IQ) quantify the ability of an observer to perform a specific task. Such measures are employed for assessing and optimizing medical imaging systems. Although the Bayesian ideal observer is optimal by definition, it is frequently both non-linear and intractable. In such cases, linear observers are commonly employed. However, the optimal linear observer, the Hotelling observer (HO), becomes intractable when considering large images. Channelized methods have become popular for reducing the dimensionality of image data. In this work, we propose a novel method for determining efficient channels by learning them with autoencoders (AEs). Autoencoders are neural networks that can be employed to learn concise representations of data, frequently for the purposes of reducing dimensionality. We trained several AEs to encode task-specific information by modifying the standard loss function and examined the effect of hidden layer size and the use of tied/untied weights on the resulting representation accuracy. Subsequently, HOs were applied to both the original images and the dimensionality-reduced versions of them produced by the AEs. It was demonstrated that, for a suitable specification of the AE, the performance of the HO was relatively unaffected by the encoding of the image. However, the computational cost of inverting the covariance matrix was greatly reduced when the HO was applied with the encoded data due to its reduced dimensionality. Our findings suggest that AEs may represent an attractive alternative to the use of heuristic channels for reducing the dimensionality of image data when seeking to accurately approximate the performance of the HO on signal detection tasks.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.