Computer vision (CV) algorithms have improved tremendously with the application of neural network-based approaches. For instance, Convolutional Neural Networks (CNNs) achieve state of the art performance on Infrared (IR) detection and identification (e.g., classification) problems. To train such algorithms, however, requires a tremendous quantity of labeled data, which are less available in the IR domain than for “natural imagery”, and are further less available for CV-related tasks. Recent work has demonstrated that synthetic data generation techniques provide a cheap and attractive alternative to collecting real data, despite a “realism gap” that exists between synthetic and real IR data.
In this work, we train deep models on a combination of real and synthetic IR data, and we evaluate model performance on real IR data. We focus on the tasks of vehicle and person detection, object identification, and vehicle parts segmentation. We find that for both detection and object identification, training on a combination of real and synthetic data performs better than training only on real data. This classification improvement demonstrates an advantage to using synthetic data for computer vision. Furthermore, we believe that the utility of synthetic data – when combined with real data – will only increase as the realism gap closes.
In order to achieve state of the art classification and detection performance with modern deep learning approaches, large amounts of labeled data are required. In the infrared (IR) domain, the required quantity of data can be prohibitively expensive and time-consuming to acquire. This makes the generation of synthetic data an attractive alternative. The well-known Unreal Engine (UE) software enables multispectral simulation addon packages to obtain a degree of physical realism, providing a possible avenue for generating such data. However, significant technical challenges remain to design a synthetic IR dataset—varying class, position, object size, and many other factors is critical to achieving a training dataset useful for object detection and classification. In this work we explore these critical axes of variation using standard CNN architectures, evaluating a large UE training set on a real IR validation set, and provide guidelines for variation in many of these critical dimensions for multiple machine learning problems.
Achieving state of the art performance with CNNs (Convolutional Neural Networks) on IR (infrared) detection and classification problems requires significant quantities of labeled training data. Real data in this domain can be both expensive and time-consuming to acquire. Synthetic data generation techniques have made significant gains in efficiency and realism in recent work, and provide an attractive and much cheaper alternative to collecting real data. However, the salient differences between synthetic and real IR data still constitute a “realism gap”, meaning that synthetic data is not as effective for training CNNs as real data. In this work we explore the use of image compositing techniques to combine real and synthetic IR data, improving realism while retaining many of the efficiency benefits of the synthetic data approach. In addition, we demonstrate the importance of controlling the object size distribution (in pixels) of synthetic IR training sets. By evaluating synthetically-trained models on real IR data, we show notable improvement over previous synthetic IR data approaches and suggest guidelines for enhanced performance with future training dataset generation.
Experimental results are presented from an investigation that evaluated the effects of introducing degraded imagery into the training and test sets of an algorithm. Degradation consisted of various applied MTFs (blur) and noise profiles. The hypothesis was that the introduction of degraded imagery into the training set would increase the algorithm's accuracy when degraded imagery was present in the test set. Preliminary experimentation confirmed this hypothesis, with some additional observations regarding robustness and feature selection for degraded imagery. Further investigations are suggested to advance this work, including increased variety of objects for classification, additional wave bands, and randomized degradations.
Simulation-based training for target acquisition algorithms is an important goal for reducing the cost and risk associated with live data collections. To this end, the US Army Night Vision and Electronic Sensors Directorate (NVESD) has developed high-fidelity virtual scenes of terrains and targets using the DIRSIG in pursuit of a virtual DRI (detect, recognize, identify) capability. In this study, the NVESD has developed a neural network (NN) algorithm that can be trained on simulated data to classify targets of interest when presented with real data. This paper discusses the classification performance of a NN algorithm and the potential impact training with simulated data has on algorithm performance.
In the pursuit of fully-automated display optimization, the US Army RDECOM CERDEC Night Vision and Electronic Sensors Directorate (NVESD) is evaluating a variety of approaches, including the effects of viewing distance and magnification on target acquisition performance. Two such approaches are the Targeting Task Performance (TTP) metric, which NVESD has developed to model target acquisition performance in a wide range of conditions, and a newer Detectivity metric, based on matched-filter analysis by the observer. While NVESD has previously evaluated the TTP metric for predicting the peak-performance viewing distance as a function of blur, no such study has been done for noise-limited conditions. In this paper, the authors present a study of human task performance for images with noise versus viewing distance using both metrics. Experimental results are compared to predictions using the Night Vision Integrated Performance Model (NV-IPM). The potential impact of the results on the development of automated display optimization are discussed, as well as assumptions that must be made about the targets being displayed.
Radiation testing results for a Geiger-mode avalanche photodiode (GM-APD) array-based imager are reviewed. Radiation testing is a crucial step in technology development that assesses the readiness of a specific device or instrument for space-based missions or other missions in high-radiation environments. Pre- and postradiation values for breakdown voltage, dark count rate (DCR), after pulsing probability, photon detection efficiency (PDE), crosstalk probability, and intrapixel sensitivity are presented. Details of the radiation testing setup and experiment are provided. The devices were exposed to a total dose of 50 krad(Si) at the Massachusetts General Hospital’s Francis H. Burr Proton Therapy Center, using monoenergetic 60 MeV protons as the radiation source. This radiation dose is equivalent to radiation absorbed over 10 solar cycles at an L2 orbit with 1-cm aluminum shielding. The DCR increased by 2.3 e−/s/pix/krad(Si) at 160 K, the afterpulsing probability increased at all temperatures and settings by a factor of ∼2, and the effective breakdown voltage shifted by +1.5 V. PDE, crosstalk probability, and intrapixel sensitivity were unchanged by radiation damage. The performance of the GM-APD imaging array is compared to the performance of the CCD on board the ASCA satellite with a similar radiation shield and radiation environment.
The human visual system (HVS) is a complicated network of filters and algorithms evolved to provide humans with an optimal set of inputs for the task at hand. Temporal and spatial averaging, matched filter analysis, variable gain settings, real time adjustments and feedback – all of these are seamlessly available to humans as they view the world around them via the HVS. In certain situations, however, these abilities may be limited by circumstances necessitated by the task, such as an intermediate display from an external sensor, constrained viewing distance or gain settings, etc. In order to improve the performance of individuals in these situations, a more thorough understanding of how the HVS compensates and performs is required. This paper investigates the denoising performance of the HVS in the presence of noise and various display settings to establish a baseline for optimal display adjustment quality under environmental or system constraints.
KEYWORDS: Performance modeling, Device simulation, Detection and tracking algorithms, Digital imaging, Cameras, Atmospheric modeling, Computer simulations, Data modeling, Roads, Signal to noise ratio
The US Army’s Communications Electronics Research, Development and Engineering Center (CERDEC) Night Vision and Electronic Sensors Directorate (referred to as NVESD) is developing a virtual detection, recognition, and identification (DRI) testing methodology using simulated imagery as a means of augmenting the field testing component of sensor performance evaluation, which is expensive, resource intensive, time consuming, and limited to the available target(s) and existing atmospheric visibility and environmental conditions at the time of testing. Existing simulation capabilities such as the Digital Imaging Remote Sensing Image Generator (DIRSIG) and NVESD’s Integrated Performance Model Image Generator (NVIPM-IG) can be combined with existing detection algorithms to reduce cost/time, minimize testing risk, and allow virtual/simulated testing using full spectral and thermal object signatures, as well as those collected in the field. NVESD has developed an end-to-end capability to demonstrate the feasibility of this approach. Simple detection algorithms have been used on the degraded images generated by NVIPM-IG to determine the relative performance of the algorithms on both DIRSIG-simulated and collected images. Evaluating the degree to which the algorithm performance agrees between simulated versus field collected imagery is the first step in validating the simulated imagery procedure.
The Center for Detectors at Rochester Institute of Technology and Raytheon Vision Systems (RVS) are leveraging RVS capabilities to produce large format, short-wave infrared HgCdTe focal plane arrays on silicon (Si) substrate wafers. Molecular beam epitaxial (MBE) grown HgCdTe on Si can reduce detector fabrication costs dramatically, while keeping performance competitive with HgCdTe grown on CdZnTe. Reduction in detector costs will alleviate a dominant expense for observational astrophysics telescopes. This paper presents the characterization of 2.5μm cutoff MBE HgCdTe/Si detectors including pre- and post-thinning performance. Detector characteristics presented include dark current, read noise, spectral response, persistence, linearity, crosstalk probability, and analysis of material defects.
The ability to count single photons is necessary to achieve many important science objectives in the near future. This paper presents the lab-tested performance of a photon-counting array-based Geiger-mode avalanche photodiode (GMAPD) device in the context of low-light-level imaging. Testing results include dark count rate, afterpulsing probability, intra-pixel sensitivity, and photon detection efficiency, and the effects of radiation damage on detector performance. The GM-APD detector is compared to the state-of-the-art performance of other established detectors using Signal-to-noise ratio as the overall evaluation metric.
KEYWORDS: Signal to noise ratio, Sensors, Avalanche photodiodes, Monte Carlo methods, Photodetectors, Photon transport, Interference (communication), Optical engineering, Signal detection, Avalanche photodetectors
Geiger-mode avalanche photodiodes (GM-APDs) use the avalanche mechanism of semiconductors to amplify signals in individual pixels. With proper thresholding, a pixel will be either “on” (avalanching) or “off.” This discrete detection scheme eliminates read noise, which makes these devices capable of counting single photons. Using these detectors for imaging applications requires a well-developed and comprehensive expression for the expected signal-to-noise ratio (SNR). This paper derives the expected SNR of a GM-APD detector in gated operation based on gate length, number of samples, signal flux, dark count rate, photon detection efficiency, and afterpulsing probability. To verify the theoretical results, carrier-level Monte Carlo simulation results are compared to the derived equations and found to be in good agreement.
KEYWORDS: Sensors, Avalanche photodetectors, Image sensors, Signal to noise ratio, Signal detection, Electrons, Silicon, Charge-coupled devices, Single photon, Photodetectors
Single-photon imaging detectors promise the ultimate in sensitivity by eliminating read noise. These devices could
provide extraordinary benefits for photon-starved applications, e.g., imaging exoplanets, fast wavefront sensing, and
probing the human body through transluminescence. Recent implementations are often in the form of sparse arrays that
have less-than-unity fill factor. For imaging, fill factor is typically enhanced by using microlenses, at the expense of
photometric and spatial information loss near the edges and corners of the pixels. Other challenges include afterpulsing
and the potential for photon self-retriggering. Both effects produce spurious signal that can degrade the signal-to-noise
ratio. This paper reviews development and potential application of single-photon-counting detectors, including highlights
of initiatives in the Center for Detectors at the Rochester Institute of Technology and MIT Lincoln Laboratory.
Current projects include single-photon-counting imaging detectors for the Thirty Meter Telescope, a future NASA
terrestrial exoplanet mission, and imaging LIDAR detectors for planetary and Earth science space missions.
The Rochester Imaging Detector Laboratory, University of Rochester, Infotonics Technology Center, and Jet Process
Corporation developed a hybrid silicon detector with an on-chip sigma-delta (ΣΔ) ADC. This paper describes the process
and reports the results of developing a fabrication process to robustly produce high-quality bump bonds to hybridize a
back-illuminated detector with its ΣΔ ADC. The design utilizes aluminum pads on both the readout circuit and the
photodiode array with interconnecting indium bumps between them. The development of the bump bonding process is
discussed, including specific material choices, interim process structures, and final functionality. Results include
measurements of bond integrity, cross-wafer uniformity of indium bumps, and effects of process parameters on the final
product. Future plans for improving the bump bonding process are summarized.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.