Editor-in-chief Maryellen Giger introduces the JMI Special Issue on COVID Medical Imaging Research

Purpose: The recent coronavirus disease 2019 (COVID-19) pandemic, which spread across the globe in a very short period of time, revealed that the transmission control of disease is a crucial step to prevent an outbreak and effective screening for viral infectious diseases is necessary. Since the severe acute respiratory syndrome (SARS) outbreak in 2003, infrared thermography (IRT) has been considered a gold standard method for screening febrile individuals at the time of pandemics. The objective of this review is to evaluate the efficacy of IRT for screening infectious diseases with specific applications to COVID-19.

Approach: A literature review was performed in Google Scholar, PubMed, and ScienceDirect to search for studies evaluating IRT screening from 2002 to present using relevant keywords. Additional literature searches were done to evaluate IRT in comparison to traditional core body temperature measurements and assess the benefits of measuring additional vital signs for infectious disease screening.

Results: Studies have reported on the unreliability of IRT due to poor sensitivity and specificity in detecting true core body temperature and its inability to identify asymptomatic carriers. Airport mass screening using IRT was conducted during occurrences of SARS, Dengue, Swine Flu, and Ebola with reported sensitivities as low as zero. Other studies reported that screening other vital signs such as heart and respiratory rates can lead to more robust methods for early infection detection.

Conclusions: Studies evaluating IRT showed varied results in its efficacy for screening infectious diseases. This suggests the need to assess additional physiological parameters to increase the sensitivity and specificity of non-invasive biosensors.

The coronavirus disease 2019 (COVID-19) pandemic has wreaked havoc across the world. It also created a need for the urgent development of efficacious predictive diagnostics, specifically, artificial intelligence (AI) methods applied to medical imaging. This has led to the convergence of experts from multiple disciplines to solve this global pandemic including clinicians, medical physicists, imaging scientists, computer scientists, and informatics experts to bring to bear the best of these fields for solving the challenges of the COVID-19 pandemic. However, such a convergence over a very brief period of time has had unintended consequences and created its own challenges. As part of Medical Imaging Data and Resource Center initiative, we discuss the lessons learned from career transitions across the three involved disciplines (radiology, medical imaging physics, and computer science) and draw recommendations based on these experiences by analyzing the challenges associated with each of the three associated transition types: (1) AI of non-imaging data to AI of medical imaging data, (2) medical imaging clinician to AI of medical imaging, and (3) AI of medical imaging to AI of COVID-19 imaging. The lessons learned from these career transitions and the diffusion of knowledge among them could be accomplished more effectively by recognizing their associated intricacies. These lessons learned in the transitioning to AI in the medical imaging of COVID-19 can inform and enhance future AI applications, making the whole of the transitions more than the sum of each discipline, for confronting an emergency like the COVID-19 pandemic or solving emerging problems in biomedicine.

Purpose: We describe the creation of computational models of lung pathologies indicative of COVID-19 disease. The models are intended for use in virtual clinical trials (VCT) for task-specific optimization of chest x-ray (CXR) imaging.

Approach: Images of COVID-19 patients confirmed by computed tomography were used to segment areas of increased attenuation in the lungs, all compatible with ground glass opacities and consolidations. Using a modeling methodology, the segmented pathologies were converted to polygonal meshes and adapted to fit the lungs of a previously developed polygonal mesh thorax phantom. The models were then voxelized with a resolution of 0.5 mm × 0.5 mm × 0.5 mm and used as input in a simulation framework to generate radiographic images. Primary projections were generated via ray tracing while the Monte Carlo transport code was used for the scattered radiation. Realistic sharpness and noise characteristics were also simulated, followed by clinical image processing. Example images generated at 120 kVp were used for the validation of themodels in a reader study. Additionally, images were uploaded to an Artificial Intelligence (AI) software for the detection of COVID-19.

Results: Nine models of COVID-19 associated pathologies were created, covering a range of disease severity. The realism of the models was confirmed by experienced radiologists and by dedicated AI software.

Conclusions: A methodology has been developed for the rapid generation of realistic 3D models of a large range of COVID-19 pathologies. The modeling framework can be used as the basis for VCTs for testing detection and triaging of COVID-19 suspected cases.

Purpose: The outbreak of COVID-19 or coronavirus was first reported in 2019. It has widely and rapidly spread around the world. The detection of COVID-19 cases is one of the important factors to stop the epidemic, because the infected individuals must be quarantined. One reliable way to detect COVID-19 cases is using chest x-ray images, where signals of the infection are located in lung areas. We propose a solution to automatically classify COVID-19 cases in chest x-ray images.

Approach: The ResNet-101 architecture is adopted as the main network with more than 44 millions parameters. The whole net is trained using the large size of 1500 × 1500 x-ray images. The heatmap under the region of interest of segmented lung is constructed to visualize and emphasize signals of COVID-19 in each input x-ray image. Lungs are segmented using the pretrained U-Net. The confidence score of being COVID-19 is also calculated for each classification result.

Results: The proposed solution is evaluated based on COVID-19 and normal cases. It is also tested on unseen classes to validate a regularization of the constructed model. They include other normal cases where chest x-ray images are normal without any disease but with some small remarks, and other abnormal cases where chest x-ray images are abnormal with some other diseases containing remarks similar to COVID-19. The proposed method can achieve the sensitivity, specificity, and accuracy of 97%, 98%, and 98%, respectively.

Conclusions: It can be concluded that the proposed solution can detect COVID-19 in a chest x-ray image. The heatmap and confidence score of the detection are also demonstrated, such that users or human experts can use them for a final diagnosis in practical usages.

Purpose: Given the recent COVID-19 pandemic and its stress on global medical resources, presented here is the development of a machine intelligent method for thoracic computed tomography (CT) to inform management of patients on steroid treatment.

Approach: Transfer learning has demonstrated strong performance when applied to medical imaging, particularly when only limited data are available. A cascaded transfer learning approach extracted quantitative features from thoracic CT sections using a fine-tuned VGG19 network. The extracted slice features were axially pooled to provide a CT-scan-level representation of thoracic characteristics and a support vector machine was trained to distinguish between patients who required steroid administration and those who did not, with performance evaluated through receiver operating characteristic (ROC) curve analysis. Least-squares fitting was used to assess temporal trends using the transfer learning approach, providing a preliminary method for monitoring disease progression.

Results: In the task of identifying patients who should receive steroid treatments, this approach yielded an area under the ROC curve of 0.85±0.10 and demonstrated significant separation between patients who received steroids and those who did not. Furthermore, temporal trend analysis of the prediction score matched expected progression during hospitalization for both groups, with separation at early timepoints prior to convergence near the end of the duration of hospitalization.

Conclusions: The proposed cascade deep learning method has strong clinical potential for informing clinical decision-making and monitoring patient treatment.

Purpose: Coronavirus disease 2019 (COVID-19) is a new infection that has spread worldwide and with no automatic model to reliably detect its presence from images. We aim to investigate the potential of deep transfer learning to predict COVID-19 infection using chest computed tomography (CT) and x-ray images.

Approach: Regions of interest (ROI) corresponding to ground-glass opacities (GGO), consolidations, and pleural effusions were labeled in 100 axial lung CT images from 60 COVID-19-infected subjects. These segmented regions were then employed as an additional input to six deep convolutional neural network (CNN) architectures (AlexNet, DenseNet, GoogleNet, NASNet-Mobile, ResNet18, and DarkNet), pretrained on natural images, to differentiate between COVID-19 and normal CT images. We also explored the model’s ability to classify x-ray images as COVID-19, non-COVID-19 pneumonia, or normal. Performance on test images was measured with global accuracy and area under the receiver operating characteristic curve (AUC).

Results: When using raw CT images as input to the tested models, the highest accuracy of 82% and AUC of 88.16% is achieved. Incorporating the three ROIs as an additional model inputs further boosts performance to an accuracy of 82.30% and an AUC of 90.10% (DarkNet). For x-ray images, we obtained an outstanding AUC of 97% for classifying COVID-19 versus normal versus other. Combing chest CT and x-ray images, DarkNet architecture achieves the highest accuracy of 99.09% and AUC of 99.89% in classifying COVID-19 from non-COVID-19. Our results confirm the ability of deep CNNs with transfer learning to predict COVID-19 in both chest CT and x-ray images.

Conclusions: The proposed method could help radiologists increase the accuracy of their diagnosis and increase efficiency in COVID-19 management.

Purpose: We propose a deep learning method for the automatic diagnosis of COVID-19 at patient presentation on chest radiography (CXR) images and investigate the role of standard and soft tissue CXR in this task.

Approach: The dataset consisted of the first CXR exams of 9860 patients acquired within 2 days after their initial reverse transcription polymerase chain reaction tests for the SARS-CoV-2 virus, 1523 (15.5%) of whom tested positive and 8337 (84.5%) of whom tested negative for COVID-19. A sequential transfer learning strategy was employed to fine-tune a convolutional neural network in phases on increasingly specific and complex tasks. The COVID-19 positive/negative classification was performed on standard images, soft tissue images, and both combined via feature fusion. A U-Net variant was used to segment and crop the lung region from each image prior to performing classification. Classification performances were evaluated and compared on a held-out test set of 1972 patients using the area under the receiver operating characteristic curve (AUC) and the DeLong test.

Results: Using full standard, cropped standard, cropped, soft tissue, and both types of cropped CXR yielded AUC values of 0.74 [0.70, 0.77], 0.76 [0.73, 0.79], 0.73 [0.70, 0.76], and 0.78 [0.74, 0.81], respectively. Using soft tissue images significantly underperformed standard images, and using both types of CXR failed to significantly outperform using standard images alone.

Conclusions: The proposed method was able to automatically diagnose COVID-19 at patient presentation with promising performance, and the inclusion of soft tissue images did not result in a significant performance improvement.

Purpose: The goal of this research is to develop innovative methods of acquiring simultaneous multidimensional molecular images of several different physiological random processes (PRPs) that might all be active in a particular disease such as COVID-19.

Approach: Our study is part of an ongoing effort at the University of Arizona to derive biologically accurate yet mathematically tractable models of the objects of interest in molecular imaging and of the images they produce. In both cases, the models are fully stochastic, in the sense that they provide ways to estimate any estimable property of the object or image. The mathematical tool we use for images is the characteristic function, which can be calculated if the multivariate probability density function for the image data is known. For objects, which are functions of continuous variables rather than discrete pixels or voxels, the characteristic function becomes infinite dimensional, and we refer to it as the characteristic functional.

Results: Several innovative mathematical results are derived, in particular for simultaneous imaging of multiple PRPs. Then the application of these methods to cancers that disrupt the mammalian target of rapamycin signaling pathway and to COVID-19 are discussed qualitatively. One reason for choosing these two problems is that they both involve lipid rafts.

Conclusions: We found that it was necessary to employ a new algorithm for energy estimation to do simultaneous single-photon emission computerized tomography imaging of a large number of different tracers. With this caveat, however, we expect to be able to acquire and analyze an unprecedented amount of molecular imaging data for an individual COVID patient.

Purpose: We used computerized image analysis and machine learning approaches to characterize spatial arrangement features of the immune response from digitized autopsied H&E tissue images of the lung in coronavirus disease 2019 (COVID-19) patients. Additionally, we applied our approach to tease out potential morphometric differences from autopsies of patients who succumbed to COVID-19 versus H1N1.

Approach: H&E lung whole slide images from autopsy specimens of nine COVID-19 and two H1N1 patients were computationally interrogated. 606 image patches (∼55 per patient) of 1024 × 882 pixels were extracted from the 11 autopsied patient studies. A watershed-based segmentation approach in conjunction with a machine learning classifier was employed to identify two types of nuclei families: lymphocytes and non-lymphocytes (i.e., other nucleated cells such as pneumocytes, macrophages, and neutrophils). Based off the proximity of the individual nuclei, clusters for each nuclei family were constructed. For each of the resulting clusters, a series of quantitative measurements relating to architecture and density of nuclei clusters were calculated. A receiver operating characteristics-based feature selection method, violin plots, and the t-distributed stochastic neighbor embedding algorithm were employed to study differences in immune patterns.

Results: In COVID-19, the immune response consistently showed multiple small-size lymphocyte clusters, suggesting that lymphocyte response is rather modest, possibly due to lymphocytopenia. In H1N1, we found larger lymphocyte clusters that were proximal to large clusters of non-lymphocytes, a possible reflection of increased prevalence of macrophages and other immune cells.

Conclusions: Our study shows the potential of computational pathology to uncover immune response features that may not be obvious by routine histopathology visual inspection.

Purpose: The coronavirus disease (COVID-19) has been spreading rapidly around the world. As of August 25, 2020, 23.719 million people have been infected in many countries. The cumulative death toll exceeds 812,000. Early detection of COVID-19 is essential to provide patients with appropriate medical care and protecting uninfected people.

Approach: Leveraging a large computed tomography (CT) database from 1112 patients provided by China Consortium of Chest CT Image Investigation (CC-CCII), we investigated multiple solutions in detecting COVID-19 and distinguished it from other common pneumonia (CP) and normal controls. We also compared the performance of different models for complete and segmented CT slices. In particular, we studied the effects of CT-superimposition depths into volumes on the performance of our models.

Results: The results show that the optimal model can identify the COVID-19 slices with 99.76% accuracy (99.96% recall, 99.35% precision, and 99.65% F1-score). The overall performance for three-way classification obtained 99.24% accuracy and a macroaverage area under the receiver operating characteristic curve (macro-AUROC) of 0.9998. To the best of our knowledge, our method achieves the highest accuracy and recall with the largest public available COVID-19 CT dataset.

Conclusions: Our model can help radiologists and physicians perform rapid diagnosis, especially when the healthcare system is overloaded.

Purpose: The objective of this study is to develop and evaluate a fully automated, deep learning-based method for detection of COVID-19 infection from chest x-ray images.

Approach: The proposed model was developed by replacing the final classifier layer in DenseNet201 with a new network consisting of global averaging layer, batch normalization layer, a dense layer with ReLU activation, and a final classification layer. Then, we performed an end-to-end training using the initial pretrained weights on all the layers. Our model was trained using a total of 8644 images with 4000 images each in normal and pneumonia cases and 644 in COVID-19 cases representing a large real dataset. The proposed method was evaluated based on accuracy, sensitivity, specificity, ROC curve, and F1-score using a test dataset comprising 1729 images (129 COVID-19, 800 normal, and 800 pneumonia). As a benchmark, we also compared the results of our method with those of seven state-of-the-art pretrained models and with a lightweight CNN architecture designed from scratch.

Results: The proposed model based on DenseNet201 was able to achieve an accuracy of 94% in detecting COVID-19 and an overall accuracy of 92.19%. The model was able to achieve an AUC of 0.99 for COVID-19, 0.97 for normal, and 0.97 for pneumonia. The model was able to outperform alternative models in terms of overall accuracy, sensitivity, and specificity.

Conclusions: Our proposed automated diagnostic model yielded an accuracy of 94% in the initial screening of COVID-19 patients and an overall accuracy of 92.19% using chest x-ray images.

Purpose: To detect and diagnose coronavirus disease 2019 (COVID-19) better and faster, separable VGG-ResNet (SVRNet) and separable VGG-DenseNet (SVDNet) models are proposed, and a detection system is designed, based on lung x-rays to diagnose whether patients are infected with COVID-19.

Approach: Combining deep learning and transfer learning, 1560 lung x-ray images in the COVID-19 x-ray image database (COVID-19 Radiography Database) were used as the experimental data set, and the most representative image classification models, VGG16, ResNet50, InceptionV3, and Xception, were fine-tuned and trained. Then, two new models for lung x-ray detection, SVRNet and SVDNet, were proposed on this basis. Finally, 312 test set images (including 44 COVID-19 and 268 normal images) were used as input to evaluate the classification accuracy, sensitivity, and specificity of SVRNet and SVDNet models.

Results: In the classification experiment of lung x-rays that tested positive and negative for COVID-19, the classification accuracy, sensitivity, and specificity of SVRNet and SVDNet are 99.13%, 99.14%, 99.12% and 99.37%, 99.43%, 99.31%, respectively. Compared with the VGG16 network, SVRNet and SVDNet increased by 3.07%, 2.84%, 3.31% and 3.31%, 3.13%, 3.50%, respectively. On the other hand, the parameters of SVRNet and SVDNet are 5.65 × 106 and 6.57 × 106, respectively. These are 61.56% and 55.31% less than VGG16, respectively.

Conclusions: The SVRNet and SVDNet models proposed greatly reduce the number of parameters, while improving the accuracy and increasing the operating speed, and can accurately and quickly detect lung x-rays containing COVID-19.

The note corrects an erroneous DOI registration.