KEYWORDS: Mammography, Breast cancer, Tumor growth modeling, Diagnostics, Data modeling, Breast, Performance modeling, Education and training, Deep learning, Cancer
In recent years, deep learning has showcased substantial promise in breast cancer diagnosis via mammograms. However, the integration of longitudinal changes between consecutive mammograms, which clinicians frequently consider for diagnosis, remains under-explored. In this study, we introduce an novel method that leverages crossattention mechanisms to capture longitudinal information between consecutive mammograms taken at various time intervals. Our method’s efficacy was assessed using a case-control internal dataset consisting of 590 cases. Preliminary results underscore its superiority over models relying solely on a single ”current” mammogram exam and those that merely combine features extracted from ”current” and ”prior” mammograms. By harnessing the power of longitudinal data, our model achieved enhanced diagnostic performance.
Recent research has shown that Generative Adversarial Networks (GANs) can generate highly realistic breast images through synthesis. Nevertheless, most of these studies assessed image quality solely through visual appraisal or reader studies, lacking quantitative analysis for specific clinical applications. This study aimed to quantitatively assess the quality of GAN-generated breast MRI images in predicting breast cancer recurrence risk. To achieve this, we developed a GAN model to synthesize the first post-contrast sequences from precontrast MRI sequences, utilizing an in-house dataset comprising 200 patients with confirmed breast cancer and available breast Dynamic Contrast-Enhanced MRI (DCE-MRI) staging data. In our study, we conducted a statistical analysis of radiomic features, revealing that among the 98 features assessed, 83 showed no significant differences (with p-values greater than 0.05) when comparing synthesized images with real images. Additionally, we employed a Lasso-Regression model to predict the Oncotype DX recurrence risk score. This analysis indicated that the predictive results for recurrence risk, derived from both real and synthesized images, did not exhibit significant differences, underscoring the comparability of synthesized images in this context.
Data shift, also known as dataset shift, is a prevalent concern in the field of machine learning. It occurs when the distribution of the data used for training a machine learning model is different from the distribution of the data the model will encounter in a real-world, operational environment (i.e., test set). This issue becomes even more significant in the field of medical imaging due to the multitude of factors that can contribute to data shifts. It is crucial for medical machine learning systems to identify and address these issues. In this paper, we present an automated pipeline designed to identify and alleviate certain types of data shift issues in medical imaging datasets. We intentionally introduce data shift into our dataset to assess and address it within our workflow. More specifically, we employ Principal Components Analysis (PCA) and Maximum Mean Discrepancy (MMD) algorithms to detect data shift between the training and test datasets. We utilize image processing techniques, including data augmentation and image registration methods, to individually and collectively mitigate data shift issues and assess their impacts. In the experiments we use a head CT image dataset of 537 patients with severe traumatic brain injury (sTBI) for patient outcome prediction. Results show that our proposed method is effective in detecting and significantly improving model performance.
KEYWORDS: Breast density, Image classification, Education and training, Mammography, Deep learning, Surgery, Medical imaging, Machine learning, Statistical modeling, Breast
Classification of Breast Imaging Reporting and Data System (BI-RADS) breast density categories generally reflects the amount of dense/fibroglandular tissue in the breast. Studies have consistently shown that breast with higher density has a higher risk of developing breast cancer compared to breast with lower density. In this paper, we propose a novel end-to-end method, namely, Medical Knowledge-guided Deep Learning (MKDL), for breast mammogram density classification. The principle behind MKDL lies in the fact that many breast image density classification tasks are partly or largely based on certain pre-known image features, such as image contrast and brightness. These pre-known features can be computationally represented and then leveraged as prior knowledge to facilitate more effective model learning and thus boost the classification performance. We designed specific knowledge-based transformations for breast density classification and showed that our model outperformed several state-of-the-art models.
KEYWORDS: Magnetic resonance imaging, Bone, Image segmentation, Deep learning, Data modeling, Injuries, Visual process modeling, Machine learning, Performance modeling
Anterior cruciate ligament (ACL) is one of the most common injuries associated with sports. Knee osseous morphology can play a role in increased knee instability. Our hypothesis is that the morphological features of the knee, as seen in knee osseous morphology, can contribute to increased knee instability and, thus, increase the likelihood of ACL tear. To test this relationship, it is necessary to segment the femur and tibia bones and extract relevant imaging features. However, manual annotation of 3D medical images, such as on magnetic resonance imaging (MRI) scans, can be a time-consuming and challenging task. In this work, we propose an automated pipeline for creating pseudo-masks of the femur and tibia bones in knee MRI. Our approach involves unsupervised segmentation and deep learning models to classify ACL integrity (intact or torn). Our results demonstrate a high agreement between the automated pseudo-masks and a radiologist’s manual segmentation, which also leads to comparable AUC values for the ACL integrity classification.
We design an intelligent tool that imitates radiologists' reading behavior and knowledge for lesion localization on radiographs using deep reinforcement learning (DRL). We formulate a human visual search behavior, i.e., 'first searching for, then focusing on' a region of interest (ROI), as a Markov Decision Process (MDP). The state of MDP is represented by a candidate ROI’s imaging features and historical actions of lesion searching. The imaging features are extracted by a convolutional neural network that is pre-trained for disease classification. The state transition in MDP is achieved by Qlearning. Specifically, we design a Q network to simulate a radiologist's succession of saccades and fixations - iteratively choosing the next ROI of radiograph to pay attention to while reading images. Guided by a novel rewarding scheme, our algorithm learns to iteratively zoom in for a close-up assessment of the potential abnormal sub-regions, until the termination condition is met. We train and test our model with 80% and 20% of the ChestX-ray8 dataset with pathologically confirmed bounding boxes (B-Boxes), respectively. The localization accuracy is measured at different thresholds of intersection over union (IoU) between the DRL-generated and the ground truth B-Box. The proposed method achieves accuracy of 0.996, 0.412, 0.667, 0.650 at threshold 0.1 respectively for cardiomegaly, mass, pneumonia, and pneumothorax. While our study is a preliminary work, it demonstrates a potential approach and the feasibility of incorporating human expert experience into a lesion detection-based machine learning task. Further investigation and evaluation of our method will be needed in future work.
Mammography is used as a standard screening procedure for the potential patients of breast cancer. Over the past decade, it has been shown that deep learning techniques have succeeded in reaching near-human performance in a number of tasks, and its application in mammography is one of the topics that medical researchers most concentrate on. In this work, we propose an end-to-end Curriculum Learning (CL) strategy in task space for classifying the three categories of Full-Field Digital Mammography (FFDM), namely Malignant, Negative, and False recall. Specifically, our method treats this three-class classification as a “harder” task in terms of CL, and creates an “easier” sub-task of classifying False recall against the combined group of Negative and Malignant. We introduce a loss scheduler to dynamically weight the contribution of the losses from the two tasks throughout the entire training process. We conduct experiments on an FFDM dataset of 1,709 images using 5-fold cross validation. The results show that our curriculum learning strategy can boost the performance for classifying the three categories of FFDM compared to the baseline strategies for model training.
Breast cancer risk prediction is becoming increasingly important especially after recent advances in deep learning models. In breast cancer screening, it is common that patients have multiple longitudinal mammogram examinations, where the longitudinal imaging data may provide additional information to boost the learning of a risk prediction model. In this study, we aim to leverage quantitative imaging features extracted from prior mammograms to augment the training of a risk prediction model, through two technical approaches: 1) prior data-enabled transfer learning, and 2) multi-task learning. We evaluated the two approaches on a study cohort of 306 patients in a case-control setting, where each patient has 3 longitudinal screening mammogram examinations. Our results show that both two approaches improved the 1-, 2-, and 3-year risk prediction, indicating that additional knowledge can be learned by our approaches from longitudinal imaging data to improve near-term risk prediction.
Deep learning models are traditionally trained purely in a data-driven approach; the information for the model training usually only comes from a single source of the training data. In this work, we investigate how to supply additional clinical knowledge that is associated with the training data. Our goal is to train deep learning models for breast cancer diagnosis using mammogram images. Along with the main classification task between clinically proven cancer vs negative/benign cases, we design two auxiliary tasks each capturing a form of additional knowledge to facilitate the main task. Specifically, one auxiliary task is to classify images according to the radiologist-made BI-RADS diagnosis scores and the other auxiliary task is to classify images in terms of the BI-RADS breast density categories. We customize a Multi-Task Learning model to jointly perform the three tasks (main task and two auxiliary tasks). We test four deep learning architectures: CBR–Tiny, ResNet18, GoogleNet, and DenseNet and we investigate the benefit of incorporating such knowledge over ImageNet pre-trained models and in the case of randomly initialized models. We run experiments on an internal dataset consisting of screening full field digital mammography images for a total of 1,380 images (341 cancer and 1,039 negative or benign). Our results show that, by adding clinical knowledge conveyed through the two auxiliary tasks to the training process, we can improve the performance of the target task of breast cancer diagnosis, thus highlighting the benefit of incorporating clinical knowledge into data-driven learning to enhance deep learning model training.
Identifying “suspicious” regions is an essential process for clinical assessment of digital mammograms in breast cancer screening. Traditional solutions attempt to model malignant lesions directly, necessitating segmentations/annotations for training machine learning models. In this paper, we present a novel approach to identify a suspicion map – a middleware preserving only the suspicious regions in digital mammograms to effectively narrow down the image search space. Our unsupervised method is implemented by modeling normal breast tissue and subsequently identifying tissue abnormal to the model as suspicious. Our method consists of three main components: superpixel-based breast tissue patch generation, deep learning-based feature extraction from normal tissue patches, and breast density-guided one-class modeling of normal patches using the extracted features. Our machine learning approach is able to safely eliminate normal regions of tissue in a digital mammogram. Our normal tissue models were learned from 2,602 normal mammogram images and tested on 180 images (including 90 normal screening mammogram images and an independent set of 90 mammogram images with breast cancer diagnoses). Initial experiments showed that our proposed method can eliminate 97% of normal regions in the normal testing mammograms and 96% of normal regions in the malignant testing mammograms. Our method, based on modeling normal breast tissue, provides a novel and unsupervised scheme to more effectively analyze digital mammogram images towards identifying suspicious regions, and has the potential to benefit a variety of downstream applications for computeraided detection, diagnosis, and triage of breast cancer in mammogram images.
Elbow fractures are one of the most common fracture types. Diagnoses on elbow fractures often need the help of radiographic imaging to be read and analyzed by a specialized radiologist with years of training. Thanks to the recent advances of deep learning, a model that can classify and detect different types of bone fractures needs only hours of training and has shown promising results. However, most existing deep learning models are purely data-driven, lacking incorporation of known domain knowledge from human experts. In this work, we propose a novel deep learning method to diagnose elbow fracture from elbow X-ray images by integrating domain-specific medical knowledge into a curriculum learning framework. In our method, the training data are permutated by sampling without replacement at the beginning of each training epoch. The sampling probability of each training sample is guided by a scoring criterion constructed based on clinically known knowledge from human experts, where the scoring indicates the diagnosis difficultness of different elbow fracture subtypes. We also propose an algorithm that updates the sampling probabilities at each epoch, which is applicable to other sampling-based curriculum learning frameworks. We design an experiment with 1865 elbow X-ray images for a fracture/normal binary classification task and compare our proposed method to a baseline method and a previous method using multiple metrics. Our results show that the proposed method achieves the highest classification performance. Also, our proposed probability update algorithm boosts the performance of the previous method.
Computer-aided diagnosis plays an important role in clinical image diagnosis. Current clinical image classification tasks usually focus on binary classification, which need to collect samples for both the positive and negative classes in order to train a binary classifier. However, in many clinical scenarios, there may have many more samples in one class than in the other class, which results in the problem of data imbalance. Data imbalance is a severe problem that can substantially influence the performance of binary-class machine learning models. To address this issue, one-class classification, which focuses on learning features from the samples of one given class, has been proposed. In this work, we assess the one-class support vector machine (OCSVM) to solve the classification tasks on two highly imbalanced datasets, namely, space-occupying kidney lesions (including renal cell carcinoma and benign) data and breast cancer distant metastasis/non-metastasis imaging data. Experimental results show that the OCSVM exhibits promising performance compared to binary-class and other one-class classification methods.
Breast magnetic resonance imaging (MRI) plays an important role in high-risk breast cancer screening, clinical problemsolving, and imaging-based outcome prediction. Breast tumor segmentation in MRI is an essential step for quantitative radiomics analysis, where automated and accurate tumor segmentation is needed but very challenging. Automated breast tumor segmentation methods have been proposed and can achieve promising results. However, these methods still need a pre-defined a region of interest (ROI) before performing segmentation, which makes them hard to run fully automatically. In this paper, we investigated automated localization and segmentation method for breast tumor in breast Dynamic Contrast-Enhanced MRI (DCE-MRI) scans. The proposed method takes advantage of kinetic prior and deep learning for automatic tumor localization and segmentation. We implemented our method and evaluated its performance on a dataset consisting of 74 breast MR images. We quantitatively evaluated the proposed method by comparing the segmentation with the manual annotation from an expert radiologist. Experimental results showed that the automated breast tumor segmentation method exhibits promising performance with an average Dice Coefficient of 0.89±0.06.
Breast magnetic resonance imaging (MRI) plays an important role in high-risk breast cancer screening, clinical problemsolving, and imaging-based outcome prediction. Breast tumor segmentation in MRI is an essential step for quantitative radiomics analysis, where automated and accurate tumor segmentation is needed but very challenging. Manual tumor annotation by radiologists requires medical knowledge and is time-consuming, subjective, prone to error, and inter-user inconsistency. Several recent studies have shown the ability of deep-learning models in image segmentation. In this work, we investigated a deep-learning based method to segment breast tumors in Dynamic Contrast-Enhanced MRI (DCE-MRI) scans in both 2D and 3D settings. We implemented our method and evaluated its performance on a dataset of 1,246 breast MR images by comparing the segmentation to the manual annotations from expert radiologists. Experimental results showed that the deep-learning-based methods exhibit promising performance with the best Dice Coefficient of 0.92 ± 0.02.
In this study, we proposed a multi-space-enabled deep learning modeling method for predicting Oncotype DX recurrence risk categories from digital mammogram images on breast cancer patients. Our study included 189 estrogen receptor-positive (ER+) and node-negative invasive breast cancer patients, who all have Oncotype DX recurrence risk score available. Breast tumors were segmented manually by an expert radiologist. We built a 3- channel convolutional neural network (CNN) model that accepts three-space tumor data: the spatial intensity information and the phase and amplitude components in the frequency domain. We compared this multi-space model to a baseline model that is based on sorely the intensity information. Classification accuracy is based on 5- fold cross-validation and average area-under the receiver operating characteristics curve (AUC). Our results showed that the 3-channel multi-space CNN model achieved a statistically significant improvement than the baseline model.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.