|
1.IntroductionThe frequency range of a millimeter wave (MMW) is 30 to 300 GHz.1 The frequency range of terahertz (THz) is normally 100 GHz to 10 THz.2 The wavelength of MMW is longer; it has higher atmospheric transmittance, and in the process of propagation, poor environmental conditions have less impact on it.1 The frequency of THz is higher; it has higher spatial resolution or longer depth of field when the spatial resolution is consistent.2 Therefore, the images of the two bands may contain different information. Since the human body radiates MMW and THz waves, the two bands are often used for human body security imaging. Compared with commonly used MMW or THz single-band imaging systems, MMW and THz dual-band imaging systems can acquire more information because the images obtained using image fusion algorithms contain more accurate descriptions of the scene than single-band images could.3–5 In our investigation, we employed an MMW and THz dual-band passive human-body security imaging system composed of 94 and 250 GHz single-element detectors to study MMW and THz dual-band image preprocessing and fusion algorithms.6,7 We developed an algorithm called the MMW and THz image preprocessing and fusion integrated algorithm (MMW-THz IPFIA), which is proposed herein. The MMW-THz IPFIA consists of denoising, enhancement, registration, and fusion algorithms and yields fused images that contain most of the information from the corresponding dual-band images, improves the probability of detection, and enhances the performance of the dual-band imaging system. The effectiveness of the algorithm is analyzed by using image signal-to-noise ratio (SNR) and information entropy.8–10 The performance improvement provided by the MMW-THz IPFIA to the system is also demonstrated in this report. Because the system responds to blackbody radiation, the commonly used thermal imaging system performance evaluation parameter—the minimum detectable temperature difference (MDTD)—was employed to analyze the system’s performance. Finally, we performed an experiment in which the MDTD was measured both before and after applying the MMW-THz IPFIA to obtain objective, quantitative evidence of the improvements it yields. 2.Millimeter Wave and Terahertz Dual-Band Passive Human-Body Security Imaging SystemFigure 1 depicts a schematic diagram of a MMW and THz dual-band passive human-body security imaging system, which is composed of a plane mirror, trihedral focusing scanning mirror, reflector, beam splitter, 94-GHz single-element detector, and 250-GHz single-element detector. The plane mirror collects the MMW and THz radiation emitted by the human body as it is swept along the direction perpendicular to the image plane to conduct a line scan. The trihedral focusing scanning mirror focuses the target radiation on the reflector and rotates along the direction parallel to the image plane to perform a column scan. The reflector reflects the radiation to the beam splitter. The beam splitter is an optical element with a semitransparent, semireflective characteristic, and it can divide the radiation into two beams: the transmitted beam and the reflected beam. The transmitted beam and the reflected beam can be captured by the 94- and 250-GHz detectors, respectively, and used to form image signals and create dual-band images. As shown in Fig. 2(a), the images obtained by the 94 and 250 GHz detectors depict a test subject hiding a steel ruler in front of the abdomen. Figures 2(b) and 2(c) present similarly generated images of the subject with a THz attenuator in front of the left side of the chest and with papers hidden within the subject’s clothes, respectively. In these original images, there is significant noise, the resolution and contrast are low, and affine problems are obvious. Thus, it is necessary to preprocess such images by performing denoising, enhancement, and registration before fusing them to obtain high SNRs, high contrast, and rich details in the resulting fused images. 3.Millimeter Wave and Terahertz Image Preprocessing and Fusion Integrated AlgorithmIn this report, we propose an MMW-THz IPFIA, which can be used to preprocess and fuse MMW and THz dual-band human-body images. 3.1.Millimeter Wave-Terahertz Image Preprocessing and Fusion Integrated AlgorithmThe MMW-THz IPFIA employs a block matching and three-dimensional (BM3D) filtering denoising algorithm, a contrast-limited adaptive histogram-equalization (CLAHE) enhancement algorithm, a mutual information based registration algorithm, and a wavelet transform-based image-fusion algorithm. 3.1.1.Block matching and three-dimensionThe BM3D algorithm, a three-dimensional (3-D) transform-domain filtering algorithm based on block matching, is one of the most effective image denoising algorithms currently in use. Figure 3 presents the structure of the BM3D algorithm, which consists of two steps: basic estimation and final denoising.11,12 Basic estimation: Obtaining a basic estimate requires three steps.
Because the step length of the sliding window may be less than its size, the blocks may overlap. Thus, an element may appear in different blocks. Estimating the value of for the ’th element requires reassessing the overlapping blocks by using the weighted average as follows: where represents any image block containing element in and is the estimator of after denoising. In addition, andThe final denoising process involves three steps.
3.1.2.Image enhancementImage enhancement technology can improve the visibility of barely observable targets and enhance the level of detail of the information acquired. Some small targets and detailed texture features occupy relatively few pixels in MMW and THz images. Thus, they occupy smaller portions of the images’ grayscales. In the proposed method, weak target loss during the fusion process is avoided since a CLAHE algorithm is used to enhance the denoised images.13,14 The CLAHE algorithm, a classic image enhancement algorithm, involves four steps.
3.1.3.Image registrationThe MMW and THz dual-band, passive human-body security imaging system employed in this investigation uses two different detectors to capture dual-band images. Variations in the detector parameters lead to geometric deviation problems, such as translation, scaling, and rotation between images. It is necessary to correct such geometric deviations to register images. In this study, we employed a mutual-information-based method to register the images, since otherwise the grayscale details in the dual-band images would be minimal and the characteristics of the images would not be obvious.15,16 The mutual-information-based registration method utilizes mutual information to conduct similarity measurements and image registration, and includes the following four basic components:
Each generation in (1 + 1), ES consists of only two individuals: a parent and an offspring. The offspring is generated through a mutation of the parent; the better of the parent and offspring is chosen as the next generation’s parent. This process can be summarized by the following four steps: 3.1.4.Image fusionImage fusion involves integrating information from two or more source images to obtain a new image that can present the scene exactly, entirely, and reliably. Compared with the prefusion images, a fused image contains more information, more details, and greater clarity. Thus, the potential target detection rate is greater. The wavelet-transform-based fusion method takes full advantage of the wavelet’s multiresolution analysis capabilities by dividing the original image into a series of subimages with different spatial resolutions and frequency domain features. The subimages reflect the local characteristics of the original image. In this method, the subimages are fused according to certain rules, the multiscale description of the fused image is formed, and, finally, the fused image is obtained by performing inverse-wavelet-transform reconstruction. Figure 5 presents a schematic diagram of the steps of the fusion algorithm.19,20 The wavelet base used in this research is the simplest Haar wavelet base: The original image is decomposed by performing a three-level discrete wavelet transform. The image characteristics are acquired using different resolutions, and after Eq. (13) is employed to complete the computation of the weighted average of the wavelet coefficient, the fused image is obtained: 3.2.Algorithm Evaluation and Verification3.2.1.Objective evaluation parametersWe use SNR (SNR is a measure of the strength relationship between the signal and the noise) and the information entropy (the larger the value, the richer the details contained in the image) to analyze the performance of the algorithm. Signal-to-noise ratioFor an image , it is first divided into several sub-blocks by sliding a window with dimensions of on it. Then the local mean and the local variance of the sub-bocks are calculated.9 Suppose that the central pixel of a block is , the local mean and the local variance are as follows: The maximum value of the local variance is selected as the signal variance of the image: A flat area without significant changes in the grayscale of the image is selected, and the mean of the local variance of the flat area is taken as the noise variance of the image:8,9 The SNR of the image is as follows: Information entropyFor an image I, the information entropy is as follows: where is the information entropy; is total number of the gray levels in the image; is the ratio of the number of pixels in grayscale to the total number of pixels in image.103.2.2.Algorithm verificationSubjective verificationIn this section, indicators of the effectiveness of applying the MMW-THz IPFIA to the images depicted in Fig. 2 are discussed. Figures 6Fig. 7–8 correspond to Figs. 2(a)–2(c), respectively. From left to right in Figs. 6–8, the profiles are presented in three groups. The first and second groups of four images, which were acquired using the 94 and 250 GHz detectors, respectively, each include the original, denoised, enhanced, and registered images, from left to right. The image on the far right in each figure is the final fused image. The processed images display reduced noise, improved detail contrast, and effective affine problem correction. The features of the two-band image pairs are synthesized in the fused images, which contain more detail. The probability of detection and capabilities of the dual-band passive THz human-body security imaging system are improved by applying the MMW-THz IPFIA. Objective verificationAccording to Eqs. (14)–(18), the SNR of the original image and the fused image are calculated first. The size of the sliding window is . The rows of the flat area are [10 25] and the columns are [30 50] (as shown in Figs. 6–8, the area in the red lines is the flat area). The grayscale distribution of the selected area in the fused images is flat. It does not contain edges or significant changes. Moreover, in this area, there is only noise in the original images, and the grayscale distribution is also flat. Table 1 shows that the values of SNR of the fused image are several times larger than that of the original image. This indicates that the noise in the image is obviously reduced. Table 1SNRs of the original image and the fused image.
Table 2 shows the values of information entropy of the original image and the fused image. The values are increased by 1 basically (for 8-bit digital images, the maximum of the information entropy is 8). This indicates that the detail information contained in the image is enhanced effectively. Table 2Information entropy of the original image and the fused image.
The two parameters fully illustrate that the algorithm can effectively reduce the noise and improve the level of detail in the images. The fused image contains more details than the original image, thus it can describe the scene more precisely. 4.Minimum Detectable Temperature Difference Measurement and VerificationMDTD is an important parameter for evaluating the performances of thermal imaging systems. It can reflect a system's thermal sensitivity characteristics and spatial resolutions. Furthermore, the MDTD does not restrict the duration of the observation. The MDTD can be measured as soon as the position of a square or circular blackbody target in the image displayed on the screen starts to become identifiable, as illustrated in Fig. 9, and equals the temperature difference between the blackbody and background. The imaging target of the MMW-THz dual band passive human-body security imaging system is the scene at room temperature, therefore, the evaluation of the system performance can be objectively carried out by using the measurement method similar to the thermal imaging system MDTD. The components of the experimental system designed to measure the MDTD are depicted in Fig. 10. Given that the environmental temperature was 20°C, both the 94 and 250 GHz subsystem unit detectors were 10 mm in size. The focal length of the system was 300 mm; the aperture diameter was 200 mm; the imaging object distance was 1.7 m; the imaging range of the object was ; and the sample spacing was 10 mm. By calculating the optical diffraction limit, the minimum resolvable sizes of the optical system of the 250 GHz subsystem and the 94-GHz subsystem are 2.196 and 5.84 mm, respectively. They are smaller than the size of the detector, therefore, the spatial resolution of the imaging system is mainly affected by the size of the detector. Since the imaging object distance was 1.7 m, and the MDTD measurement targets were designed to be circular and to range in diameter from 60 to 130 mm in 10-mm increments, the spatial angular frequency is presented in Table 3. Images of the 60- and 100-mm-diameter targets are illustrated in Fig. 11 as examples. The measured MDTD curve is shown in Fig. 12. Table 3Spatial angular frequencies of the targets f (cyc/mrad).
The curve shown in Fig. 12 can be used to compare the original MDTDs with those after preprocessing with the 94 GHz and 250 GHz subsystems. In principle, for the same spatial angular frequency, the smaller the MDTD, the higher the system detection rate. In each case, the MDTD is reduced by preprocessing, indicating that the detection performance of the system is improved. Under identical conditions, the MDTD of the 94-GHz subsystem is larger than that of the 250-GHz subsystem in both the original and processed systems. Thus, the MDTD of the system processed by using MMW-THz IPFIA lies between those of the processed 94- and 250-GHz subsystems, that is, . In addition, the MDTDs obtained using the proposed algorithm are smaller and are favorable compared with those obtained using the two original subsystems. This outcome, attributable to the difference between the performances of the two subsystems, demonstrates that the final MDTD is related to the proportions of the subsystem images in the fused images. These results quantitatively demonstrate that the detection performance of the system is improved by applying the MMW-THz IPFIA. 5.ConclusionIn this article, a comprehensive algorithm, MMW-THz IPFIA, was proposed for use in MMW and THz dual-band passive human-body security imaging systems. This algorithm, which consists of denoising, enhancement, registration, and fusion algorithms, can be employed to generate fused images while preserving most of the details present in the original dual-band images. The performance of the algorithm was analyzed by calculating the SNR and information entropy of the original images and fused images. The result indicated that the resulting fused images exhibited less noise, greater detail, and more information than the single-band images did. We used a test method similar to the MDTD of the thermal imaging system to objectively verify the performance improvement of the system provided by applying the MMW-THz IPFIA. The result indicated that the MDTD performance has been improved. Thus, the performance of the system can be improved by employing the MMW-THz IPFIA. Since the denoising, enhancement, registration, and fusion algorithms for MMW and THz images cannot yet be employed in real-time processing, the objectives of the next phase of research will be to integrate these processes with hardware systems and to achieve real-time fusion of dual-band images to support efficient security monitoring using MMW and THz dual-band, passive human-body security imaging systems. AcknowledgmentsThis work is supported by the project of National Science Foundation of China (Grant No. 61171051). ReferencesR. Appleby and R. N. Anderton,
“Millimeter-wave and submillimeter-wave imaging for security and surveillance,”
Proc. IEEE, 95 1683
–1690
(2007). http://dx.doi.org/10.1109/JPROC.2007.898832 Google Scholar
J. Z. Xu and X. C. Zhang, Terahertz Science Technology and Application, 1 Peking University Press, Beijing, China
(2007). Google Scholar
Z. Zhang and R. S. Blum,
“A categorization of multiscale-decomposition-based image fusion schemes with a performance study for a digital camera application,”
Proc. IEEE, 87 1315
–1326
(1999). http://dx.doi.org/10.1109/5.775414 Google Scholar
C. Poh et al.,
“Review article multisensor image fusion in remote sensing: concepts, methods and applications,”
Int. J. Remote Sens., 9 823
–854
(1998). http://dx.doi.org/10.1080/014311698215748 Google Scholar
G. Pajares and J. M. De La Cruz,
“A wavelet-based image fusion tutorial,”
Pattern Recognit., 37 1855
–1872
(2004). http://dx.doi.org/10.1016/j.patcog.2004.03.010 Google Scholar
A. Luukanen et al.,
“Real-time passive terahertz imaging system for standoff concealed weapons imaging,”
Proc. SPIE, 7670 767004
(2010). http://dx.doi.org/10.1117/12.850369 PSISDG 0277-786X Google Scholar
C. Mann et al.,
“First demonstration of a vehicle mounted 250 GHz real time passive imager,”
Proc. SPIE, 7311 73110Q
(2009). http://dx.doi.org/10.1117/12.821775 PSISDG 0277-786X Google Scholar
R. L. Lagendijk and J. Biemond, Iterative Identification and Restoration of Images, Kluwer Academic Publishers, The Netherlands
(1991). Google Scholar
M. Y. Zhou, Deconvolution and Signal Recovery, 186
–188 Nation Defense Industry Press, Beijing, China
(1900). Google Scholar
W. L. Lau et al.,
“Comparison of image data fusion techniques using entropy and INI,”
in 22nd Asian Conf. on Remote Sensing,
(2001). Google Scholar
K. Dabov et al.,
“Image denoising with block-matching and 3D filtering,”
Proc. SPIE, 6064 354
(2006). http://dx.doi.org/10.1117/12.643267 Google Scholar
K. Dabov et al.,
“Image denoising by sparse 3-D transform-domain collaborative filtering,”
IEEE Trans. Image Process., 16
(8), 2080
–2095
(2007). http://dx.doi.org/10.1109/tip.2007.901238 IIPRE4 1057-7149 Google Scholar
S. M. Pizer et al.,
“Contrast-limited adaptive histogram equalization: speed and effectiveness,”
in Proc. First Conf. on Visualization in Biomedical Computing,
337
–345
(1990). http://dx.doi.org/10.1109/VBC.1990.109340 Google Scholar
Y. Wang et al.,
“Image enhancement based on equal area dualistic sub-image histogram equalization method,”
IEEE Trans. Consum. Electron., 45 68
–75
(1999). http://dx.doi.org/10.1109/30.754419 Google Scholar
B. Zitová and J. Flusser,
“Image registration methods: a survey,”
Image Vis. Comput., 21
(11), 977
–1000
(2003). http://dx.doi.org/10.1016/s0262-8856(03)00137-9 Google Scholar
J. B. A. Maintz and M. A. Viergever,
“A survey of medical image registration,”
Med. Image Anal., 2
(1), 1
–36
(1998). http://dx.doi.org/10.1016/s1361-8415(01)80026-8 Google Scholar
H.-G. Beyer,
“Toward a theory of evolution strategies: some asymptotical results from the (1, )-theory,”
Evol. Comput., 1
(2), 165
–188
(1993). http://dx.doi.org/10.1162/evco.1993.1.2.165 EOCMEO Google Scholar
D. V. Arnold and H. G. Beyer,
“Local performance of the (1 + 1)-ES in a noisy environment,”
IEEE Trans. Evol. Comput., 6 30
–41
(2002). http://dx.doi.org/10.1109/4235.985690 Google Scholar
B. V. Dasarathy,
“Guest editorial: special section on sensor fusion 1996,”
Opt. Eng., 35 601
–602
(1996). http://dx.doi.org/10.1117/1.601002 Google Scholar
S. G. Mallat,
“Multifrequency channel decompositions of images and wavelet models,”
IEEE Trans. Acoust. Speech Signal Process., 37 2091
–2110
(1990). http://dx.doi.org/10.1109/29.45554 Google Scholar
BiographyLi Tian is a doctoral student at Beijing Institute of Technology in China. Her current research interests include THz imaging theory, THz imaging systems, and THz image processing. |