Research on fusion technology based on low-light visible image and infrared image

Shuo Liu; Yan Piao; Muhammad Tahir

doi:10.1117/1.OE.55.12.123104

8 December 2016 Research on fusion technology based on low-light visible image and infrared image

Shuo Liu, Yan Piao, Muhammad Tahir

Author Affiliations +

Optical Engineering, Vol. 55, Issue 12, 123104 (December 2016). https://doi.org/10.1117/1.OE.55.12.123104

Abstract

Image fusion technology usually combines information from multiple images of the same scene into a single image so that the fused image is often more informative than any source image. Considering the characteristics of low-light visible images, this study presents an image fusion technology to improve contrast of low-light images. This study proposes an adaptive threshold-based fusion rule. Threshold is related to the brightness distribution of original images. Then, the fusion of low-frequency coefficients is determined by threshold. Pulse-coupled neural networks (PCNN)-based fusion rule is proposed for fusion of high-frequency coefficients. Firing times of PCNN reflect the amount of detail information. Thus, a high-frequency coefficient corresponding to maximum firing times is chosen as the fused coefficient. Experimental results demonstrate that the proposed method obtains high-contrast images and outperforms traditional fusion approaches on image quality.

1. Introduction

Image fusion technology is aimed at obtaining one image with high quality from two more images of the same scene.¹ Image fusion technology has been applied in various fields, such as remote sensing,² medical diagnosis,³^,⁴ face recognition,⁵ target detection,⁶^,⁷ and art analysis.⁸ Fusion technology of infrared and visible images is an important branch in the image fusion field. An infrared image can display camouflaged objects under trees or smog. But its contrast is low and it is difficult to observe or recognize targets for the human vision system. On the contrary, a visible image usually contains much more texture and color information of the scene. But there are many dark regions in low-light visible images. Objects are fuzzy and difficult to be distinguished in these regions. However, fused images can effectively combine advantages of two images that can extend and enhance image information. Thus, it is necessary to study fusion technology of infrared and low-light visible images.

In the past few years, some techniques and software algorithms for image fusion have been developed.⁹^,¹⁰ Generally, image fusion methods mainly include pixel-level fusion, feature-level fusion, and decision-level fusion. Compared to others, pixel-level fusion can better preserve original image information. In most pulse-coupled neural networks (PCNN)-based fusion algorithms, only the single pixel value is used to motivate a PCNN neuron. It is not effective enough because human eyes are usually sensitive to features. Qu proposed an orientation information motivated PCNN algorithm (OI-PCNN). Orientation information is considered as a feature to motivate PCNN.¹¹ This algorithm can preserve spatial characteristics of source images well and only grayscale source images are considered in this method. Kong et al.¹² proposed an adaptive fusion technique based on nonsubsampled contourlet transform and intensity–hue–saturation (IHS) transform. The pseudocolor principle is used in the fusion step. Fused images can obtain color information of reference images based on the pseudocolor principle. Results obtained show a good performance. But the algorithm costs too much time and pseudocolor information of targets may result in erroneous determination. Li et al.¹³ proposed a guided filtering-based fusion method (GFF). First, source images are decomposed into a base layer containing large scale variations in intensity and a detail layer capturing small-scale details. Then a weighted average technique based on guided filtering is chosen for fusion of base and detail layers. Zhou and Tan¹⁴ combined infrared and visible images using wavelet transform (WT). Li¹⁵ proposed an infrared and visible image fusion algorithm based on target segmentation. First, a segmentation algorithm is used to extract target of an infrared image. Then, the target is fused with a visible image. This method preserves target information of infrared effectively. But the other information of the infrared image is lost.

A fused image retains most desirable information and characteristics of infrared and visible images. Generally, grayscale visible images in daytime are mostly used in traditional methods. Moreover, registration problems of infrared and visible images are not considered. Therefore, traditional fusion methods are not suitable for fusion of infrared and low-light visible images. To overcome these problems, a fusion technology is presented in this paper. It can be widely applied in various fields, such as target recognition, medical diagnosis, transportation, intelligent transportation, and video surveillance.

Necessary background information is provided in Sec. 2. In Sec. 3, the proposed image fusion method is described. Moreover, improved fusion rules are introduced. Experimental results are presented and discussed in Sec. 4. Finally, the conclusion of this study is presented in Sec. 5.

2. Background

Image fusion algorithms mainly include three categories, i.e., pixel-level fusion, feature-level fusion, and decision-level fusion. Compared with feature-level and decision-level fusion, pixel-level fusion directly combines the pixels of original images, which is applied more widely.

Multiscale transform methods have been demonstrated to be very useful in the pixel-level image fusion field. In previous research, pyramid methods and WT are most commonly used, such as Laplacian pyramid¹⁶^,¹⁷ and discrete wavelet transform (DWT).¹⁸^–²⁰ In addition to these, some multiscale transform methods are proposed, such as dual-tree discrete WT (DT-DWT),²¹ curvelet transform,²² contourlet transform,²³ and so on. DWT or DT-DWT often suffers from limited directions and cannot represent edges of images. Moreover, the traditional DWT-based fusion method can introduce a blocking effect to fused image.

For representing images accurately, Candès and Donoho²⁴ proposed curvelet transform. Unlike curvelet transform that first develops a transform in the continuous domain and then discrete for sampled data, contourlet transform starts with a discrete-domain construction directly.²⁵ Contourlet transform is composed of Laplacian pyramid and directional filter bank (DFB). Images can be decomposed into multiscales by Laplacian pyramid. Then DFBs are carried out to obtain directional components. The low-frequency part is always obtained by Laplacian pyramid decomposition. In addition, the high-frequency part is that the original image subtracts the low-frequency part, which is through upsampling. The Laplacian pyramid decomposition process is as follows:

Eq. (1)

L P_{k} = {\begin{cases} G_{k} - G_{k + 1}^{*} \begin{matrix} N - 1 \geq k \geq 0 \end{matrix} \\ G_{k} \begin{matrix} \begin{matrix}  \end{matrix} & \begin{matrix} k = N \end{matrix} \end{matrix} \end{cases},

Eq. (2)

G_{k} (m, n) = \sum_{i = - 2}^{2} \sum_{j = - 2}^{2} w (i, j) G_{k - 1} (2 m + i, 2 n + j),

Eq. (3)

G_{k}^{*} (m, n) = 4 \sum_{i = - 2}^{2} \sum_{j = - 2}^{2} w (i, j) G_{k - 1}^{'} (\frac{m - i}{2}, \frac{n - j}{2}),

where

k

represents the decomposition levels, (

m, n

) is the pixel position of

k

’th level image, and

w (i, j)

is the low-pass filter. When

(m - i) / 2

and

(n - j) / 2

are integer,

G_{k}^{'} = G_{k}

, otherwise

G_{k}^{'} = 0

.

The LP decomposition at each level generates a downsampled low-pass version of the original and the difference between the original and the prediction, resulting in a bandpass image.²⁵ And each bandpass image is further decomposed by an $l_{j}$ level DFB into $2^{l j}$ bandpass directional images. For the traditional contourlet transform-based fusion method, low-frequency coefficients are fused by average rule and high-frequency coefficients are fused by absolute value choosing max rule. In general, it is difficult to obtain high-contrast and high-definition fused images.

3. Proposed Method

Original images have been strictly matched in most image fusion algorithms. But the registration problem is important in the image fusion field. In addition, a low-light visible image has many dark regions. These dark regions result in reducing quality of the fused image. Furthermore, it is difficult to observe objects in infrared images captured directly. A low-light visible image should also be transformed to an intensity image in the fusion step.

To overcome these problems, the proposed fusion method mainly includes preprocessed sections (such as IHS transform, original images registration, and image inverse), contourlet transform, coefficients fusion, and inverse contourlet transform. Figure 1 shows the structure of the proposed image fusion method.

Fig. 1

Structure of proposed image fusion method.

According to the structure of the proposed image fusion method, the main steps are as follows:

1. Source images should be matched first. After that, matched images are enhanced to improve contrast. Then the luminance component of low-light visible image $I_{v}$ is extracted using IHS transform.
2. After preprocessing, infrared and luminance images ( $I_{IR}$ and $I_{v}$ are decomposed separately to low-frequency coefficients ( $I_{IR}^{J}, I_{v}^{J}$ ) and high-frequency coefficients ( $I_{IR}^{d, k}, I_{v}^{d, k}$ ) using contourlet transform, respectively. $J$ is the number of decomposition stages. $k$ shows directional number at $d$ ’th scale ( $J \geq d \geq 1$ ). Generally, more decomposition levels cost longer computation time. Therefore, two decomposition levels are chosen in this study. Generally, more directional subbands mean more high-frequency information. To reduce time consumption, two directional subbands are obtained in the first decomposition level and 16 directional subbands are obtained in the second level.
3. Considering different characteristics of low- and high-frequency coefficients, different fusion rules are chosen. Low-frequency coefficients are fused by the adaptive threshold-based fusion rule. High-frequency coefficients are fused by the PCNN-based rule.
4. Fused low-frequency coefficients and high-frequency coefficients ( $I_{F}^{low}, I_{F}^{high}$ ) are transformed into fused luminance image $I_{F}$ using inverse contourlet transform.
5. Considering the characteristics of low-light visible images, the final fused image is obtained by combining the fused luminance image and low-light visible image.

3.1.

Low-Frequency Coefficients Fusion

Low-frequency coefficients mainly contain approximate characteristics of infrared and low-light visible images. First, threshold value is adjusted based on maximum of the illuminance image. Then, current pixel value of the illuminance image is used as the fused pixel if the difference is bigger than threshold through comparing. Otherwise, the average rule is used for low-frequency coefficients fusion.

Threshold adjustment is one key step in low-frequency coefficients fusion. Mostly, there are only small amounts of brightest pixel values in low-light visible images. In low-light visible images, few pixels have high intensity value. These high intensity pixels are mostly from background light, such as car light, lamp light, and other light-emitting devices. After two levels of decomposition, low-frequency coefficients ( $I_{v}^{J}, I_{IR}^{J}$ ) are obtained. Through experiments, the top 0.13% largest values of coefficients difference are considered as background light components. Therefore, coefficients difference is sorted first. Then, the top 0.13% largest values are chosen to determine the threshold. In this paper, $w_{th} = 0.75$

Eq. (4)

I_{th} = w_{th} \times \max (I_{v}^{J} - I_{IR}^{J}),

Eq. (5)

I_{F}^{low} = {\begin{cases} I_{v}^{J} & I_{v}^{J} \geq I_{th} \\ (I_{IR}^{J} + I_{v}^{J}) / 2 & otherwise \end{cases},

where

I_{th}

is the threshold and

w_{th}

is the threshold weight.

I_{IR}^{J}

and

I_{v}^{J}

are corresponding low-frequency coefficients.

3.2.

High-Frequency Coefficients Fusion

High-frequency coefficients mainly contain detail characteristics of infrared and illuminance images. It represents texture and edge information of original images. The rule of maximum absolute value is usually chosen in most traditional fusion methods. But fused image quality is often bad.

PCNN is a simplified neural network model, which is constructed by a plurality of interconnected neurons. Each pixel datum is one neuron in image processing and each neuron mainly includes dendritic branch, connector, and pulse generator sections. For high-frequency coefficients fusion, the mathematical model based on PCNN can be described as follows. High-frequency coefficients are used to motivate PCNN

Eq. (6)

F_{i, j}^{d, k} (n^{'}) = I_{i, j}^{d, k},

where

F_{i, j}^{d, k}

is an output of feeding part,

I_{i, j}^{d, k}

is an input of feeding part,

n^{'}

shows iteration time,

k

is the decomposition level,

d

represents direction at

k^{'}

th level, and (

i, j

) shows pixel location.

Linking part is described as

Eq. (7)

L_{i, j}^{d, k} (n^{'}) = \exp (- a_{L}) L_{i, j}^{d, k} (n^{'}) + V_{L} \sum_{m, n} W_{i j, m n} Y_{i j, m n}^{d, k} (n^{'} - 1),

where

L_{i, j}^{d, k} (n^{'})

is an output of linking part,

m

and

n

are the scope of the connected neurons,

V_{L}

is a normalization coefficient, and

W_{i j, m n}

is the weight to connect other neurons

Eq. (8)

U_{i, j}^{d, k} (n^{'}) = F_{i, j}^{d, k} (n^{'}) [1 + β L_{i, j}^{d, k} (n^{'})],

Eq. (9)

θ_{i, j}^{d, k} (n^{'}) = \exp (- a_{θ}) θ_{i, j}^{d, k} (n^{'} - 1) + V_{θ} Y_{i, j}^{d, k} (n^{'} - 1),

where

U_{i, j}^{d, k} (n^{'})

is an internal state,

θ_{i, j}^{d, k} (n^{'})

is a threshold,

β

,

a_{θ}

, and

V_{θ}

are the constant parameters. In each iteration, output (also called firing time) is calculated as²⁶

Eq. (10)

Y_{i, j}^{d, k} (n^{'}) = {\begin{matrix} 1 & U_{i, j}^{d, k} (n^{'}) > θ_{i, j}^{d, k} (n^{'}) \\ 0 & otherwise \end{matrix},

Eq. (11)

T_{X, i j}^{d, k} = \sum_{n^{'} = 1}^{N} Y_{i, j}^{d, k} (n^{'}) X = v or IR,

where

X

represents the low-light visible or infrared image and

N

is the total number of iteration times.

After $N$ iterations, it is easy to obtain firing times of high-frequency coefficients. Then, fused high-frequency coefficients are obtained as

Eq. (12)

I_{F}^{high} = {\begin{cases} I_{v}^{d, k} & T_{v, i j}^{d, k} \geq T_{IR, i j}^{d, k} \\ I_{IR}^{d, k} & otherwise \end{cases} .

4. Experimental Results and Discussion

To verify practical value of the proposed method, fused image quality and fusion effect are analyzed in this section.

4.1.

Evaluation Criteria

Subjective criterion is mainly based on observation of human eyes. It is always hard to evaluate fused image quality. Objective criteria are based on characteristics of fused images and are often applied to evaluate image quality. To verify validity of the proposed method, subjective criterion and objective criteria are both analyzed in this section.

Mean value reflects average brightness of an entire image. Usually, it is hard to distinguish objects in a low luminance region for human eyes. Mean value represents average illumination of fused images. It is useful to evaluate the dynamic range of low-light fused images indirectly. For the human visual system, a high dynamic range of fused image brightness always indicates high definition. So larger mean value often shows better fused effect.

Standard deviation reflects gray-level distribution of a fused image. It describes the discrete degree between image pixels and mean value. Standard deviation is often used to evaluate contrast of fused images. The below equation shows the definition:

Eq. (13)

σ_{fused} = \sqrt{\frac{1}{m \times n} \sum_{i = 0}^{m - 1} \sum_{j = 0}^{n - 1} {[p (i, j) - μ]}^{2}},

where

δ_{fused}

is the standard deviation of the fused image. Size of the fused image is

m \times n

.

p (i, j)

is the probability of the position (

i, j

). Mean value of the fused image is

μ

.

Entropy describes average information carried by a fused image. If entropy of a fused image is large, it indicates that there is a lot of information in the fused image. Definition is shown as follows:

Eq. (14)

E_{fused} = - \sum_{j = 0}^{L - 1} p_{j} \log_{2} p_{j},

where

E_{fused}

is the entropy of the fused image and

p_{j}

is the probability of pixel value

j

.

L

is 255, generally.

Average gradient is also called image clarity. Generally, a larger value depicts better image quality. Equation (15) shows the definition of average gradient²⁷

Eq. (15)

g_{fused} = \frac{1}{(m - 1) \times (n - 1)} \sum_{i = 1}^{m - 1} \sum_{j = 1}^{n - 1} \sqrt{\frac{{[I (i + 1, j) - I (i, j)]}^{2} + {[I (i, j + 1) - I (i, j)]}^{2}}{2}},

where

g_{fused}

is the average gradient of the fused image and

I (i, j)

, is the pixel value at (

i, j

). Size of fused image is

m \times n

.

4.2.

Grayscale Images Fusion Analysis

To evaluate the proposed method, different fusion algorithms are chosen to compare. A fused image is mainly the fusion of intensity images and enhanced infrared images. Then, subjective criterion and objective criteria are used to analyze image quality. The first group experiment images are obtained from a website (Ref. 28). Figure 2 shows original images with resolution $640 \times 480$ . From the above red rectangle, clear words can be seen in Fig. 2(a) and nothing in Fig. 2(b). However, from the left red rectangle, a hidden person appears in Fig. 2(b) and nothing in Fig. 2(a).

Fig. 2

Original images: (a) original low-light visible image and (b) original infrared image.

There is different information in the original low-light visible image and infrared image observed in Fig. 2. To improve image quality, various methods are used to combine original images. Figure 3 shows fused images using different methods. Figure 3(a) represents the fused result using WT with two levels. Figure 3(b) shows a fused image using WT with four levels. Figure 3(c) is the fused result using traditional contourlet transform. Figure 3(d) shows a fused image of the OI-PCNN algorithm.¹¹ The fusion result of the GFF algorithm¹³ is shown in Fig. 3(e). Figure 3(f) shows the fused result of the proposed method.

Fig. 3

Fused grayscale images: (a) WT with two levels, (b) WT with four levels, (c) traditional contourlet transform, (d) OI-PCNN algorithm, (e) GFF algorithm, and (f) proposed method.

From Fig. 3, we can see that there is an obvious blocking effect in Figs. 3(a) and 3(b). And there is low contrast in Fig. 3(c). It is difficult to observe objects by human eyes. Figure 3(d) shows a number of fused errors in many areas, such as billboard and car lights. Figure 3(f) retains more information of original images than Fig. 3(e). The proposed method outperforms the OI-PCNN algorithm and GFF algorithm in comparison.

To analyze fused image quality more accurately, this study compares objective criteria of fused images in Table 1. The fused image of the OI-PCNN algorithm has the largest mean value and entropy value in Table 1. But from Fig. 3(d), we can clearly see that there are lots of fused errors, which impact on people’s subjective feelings seriously. Due to the effect of fused errors, it is difficult to observe objects for the human eye visual system. The fused image of the proposed method has a larger mean value and entropy than the others except for the OI-PCNN algorithm. Moreover, it has the largest standard deviation and average gradient. Its average running time is large but less than OI-PCNN. Considering all the characteristics, the fused image contains most detail information and highest contrast by using the proposed method than the others.

Table 1

Objective criteria of grayscale fused images.

Method	Criteria
Method	Mean	Standard deviation	Entropy	Average gradient	Average running time
WT with two levels	52.0438	22.7295	6.0219	3.0981	0.2473
WT with four levels	52.0872	28.4594	6.3381	3.9704	0.3150
Traditioal contourlet transform	51.9011	21.8508	5.9670	2.7671	0.8137
OI-PCNN algorithm¹¹	82.3657	35.7272	6.8181	3.3874	5.9449
GFF algorithm¹³	70.8443	32.4660	6.5351	3.2244	0.9360
Proposed method	74.0720	35.7955	6.7109	3.5111	5.7434

Note: Bold values are used to show the best quality of objective criterion.

4.3.

Color-Scale Images Fusion Analysis

In this section, a color-scale low-light image and infrared image are used for analysis. The original low-light visible image is captured by a Nikon camera. Moreover, the original infrared image is captured by a Xenics Bobcat-640 camera. Two group images are obtained at the south of the first school building in Changchun University of Science and Technology. Size of source image is $640 \times 480$ .

From Fig. 4(a), we can clearly see that there are many dark areas. Opposite, many targets are clear in the infrared image as shown in Fig. 4(b). In the proposed image fusion method, source images are matched first. Then matched images are enhanced to improve contrast based on the human eye vision system. In addition, matched visible images should be transformed to obtain intensity images using IHS transform.

Fig. 4

Original images: (a) original low-light visible image and (b) original infrared image.

Figure 5 shows a fused color image using different algorithms. In Fig. 5(a), the brightness of the fused color image is low. It is hard to observe the scene information for human eyes. Blocking effect reduces fused image quality in Fig. 5(b). In Fig. 5(c), image contrast still needs to be enhanced. In Fig. 5(d), there are obvious fusion errors in the sky area. In addition, there is a black edge around the lamps in Fig. 5(e). Moreover, brightness still needs to be improved. Figure 5(f) has better image quality than others subjectively. For example, the contour of cars is clearer than others, especially car wheels. In addition to that, there is still more edge information in the distant building and trees retained in Fig. 5(f) than others.

Fig. 5

Fused color image: (a) WT with two levels, (b) WT with four levels, (c) traditional contourlet transform, (d) OI-PCNN algorithm, (e) GFF algorithm, and (f) proposed method.

Table 2 shows objective criteria of various methods. By comparison, the proposed method has better mean value, standard deviation, entropy value, and average gradient value than others. Generally, larger objective criteria reflect better fusion results. In addition, its average running time is large but less than OI-PCNN. From Table 2, we can see the proposed method has better fusion results of color images than others. Moreover, objective evaluation and subjective evaluation reach the same conclusion. It is easy to conclude that the proposed method outperforms the OI-PCNN algorithm and GFF algorithm.

Table 2

Objective criteria of fused color images (experiment #1).

Method	Criteria
Method	Mean	Standard deviation	Entropy	Average gradient	Average running time
Original visible image	37.7766	70.3813	12.2569	3.1906
WT with two levels	99.7091	118.1735	18.3448	6.3606	0.3171
WT with four levels	100.3938	120.8256	18.4325	7.0295	0.3818
Traditioal contourlet transform	99.5396	117.8236	18.3726	6.2623	0.9145
OI-PCNN algorithm¹¹	126.9540	127.5001	19.6796	7.4368	8.0859
GFF algorithm¹³	117.8568	117.0916	19.3722	6.6016	1.1351
Proposed method	179.2643	158.0639	19.9771	9.0700	6.9468

Note: Bold values are used to show the best quality of objective criterion.

Figure 6 shows another group of source images. Both the low-light image and infrared image are $640 \times 480$ . In Fig. 6(a), lots of targets are not clear (such as people or trees) in the low-light image. However, it is clear in the infrared image as we can see in Fig. 6(b).

Fig. 6

Original images: (a) original low-light visible image and (b) original infrared image.

Figure 7 shows fused results using different methods. Figures 7(a) and 7(b) represent low contrast. Figure 7(c) has low brightness. From Fig. 7(d), we can see that there are many fusion errors. In Fig. 7(e), there are black pixels around lamps. In addition, it also has low brightness. Compared to others, the proposed method obtains high contrast and brightness as shown in Fig. 7(f).

Fig. 7

Fused color image: (a) WT with two levels, (b) WT with four levels, (c) traditional contourlet transform, (d) OI-PCNN algorithm, (e) GFF algorithm, and (f) proposed method.

From Table 3, we can see that the proposed method has the largest mean value, standard deviation, and entropy. Moreover, it also has larger average gradient value than the GFF algorithm and its average running time is less than the OI-PCNN algorithm. Therefore the proposed method has better fusion results than others by comparing objective criteria. Experimental results show that the proposed method outperforms the OI-PCNN algorithm and GFF algorithm.

Table 3

Objective criteria of second fused color Images (experiment #2).

Method	Criteria
Method	Mean	Standard deviation	Entropy	Average gradient	Average running time
Original visible image	78.9970	137.7899	15.6267	8.4931
WT with two levels	159.7776	177.9980	20.1652	13.1642	0.2925
WT with four levels	160.0983	180.9529	20.2653	13.9969	0.3323
Traditioal contourlet transform	159.7050	177.6528	20.1716	13.0353	0.9918
OI-PCNN algorithm¹¹	187.5629	183.5756	21.1463	13.8270	6.2495
GFF algorithm¹³	171.3346	172.2097	20.8379	12.9782	0.8938
Proposed method	296.6510	186.4172	21.4505	13.6713	6.0357

Note: Bold values are used to show the best quality of objective criterion.

5. Conclusion

In this paper, a fusion method of infrared and low-light visible images is proposed. First, original infrared and visible images are preprocessed. In addition, different fusion rules are chosen based on characteristics of low- and high-frequency information. The low-frequency component is fused using the adaptive threshold-based rule and the high-frequency component is fused based on PCNN. Moreover, the proposed method is also applicable for fusion of color visible images and infrared images. Finally, experimental results show that this method improves fused image quality through subjective and objective evaluations. Though the proposed method effectively improves image quality, some objects (such as people) are not clear enough. In the future, target extraction and enhancement can be introduced into the image fusion algorithm to improve fused image quality further.

Acknowledgments

This work was supported in part by grants from Project of Ministry of Science and Technology of People’s Republic of China (No. 2015DFR10670), Project of Jilin Province Development and Reform Commission (No. 2014Y109), and Projects of Jilin Province Science and Technology Department (Nos. 20140204045GX and 20140203014GX).

References

1.

C. L. Yao et al., “Research of multi-sensor images based on color fusion methods,” J. Networks, 8 (11), 2635 –2641 (2013). http://dx.doi.org/10.4304/jnw.8.11.2635-2641 Google Scholar

2.

B. Jin, G. Kim and N. I. Cho, “Wavelet-domain satellite image fusion based on a generalized fusion equation,” J. Appl. Remote Sens., 8 (1), 080599 (2014). http://dx.doi.org/10.1117/1.JRS.8.080599 Google Scholar

3.

G. Bhatnagar, Q. M. J. Wu and Z. Liu, “Directive contrast based multimodal medical image fusion in NSCT domain,” IEEE Trans. Multimedia, 15 (5), 1014 –1024 (2013). http://dx.doi.org/10.1109/TMM.2013.2244870 Google Scholar

4.

S. S. Bedi, J. Agarwal and P. Agarwal, “Image fusion techniques and quality assessment parameters for clinical diagnosis: a review,” Int. J. Adv. Res. Comput. Commun. Eng., 2 (2), 1153 –1157 (2013). Google Scholar

5.

G. Bebis and I. Pavlidis, “Infrared and visible image fusion for face recognition,” Proc. SPIE, 5404 585 –596 (2004). http://dx.doi.org/10.1117/12.543549 PSISDG 0277-786X Google Scholar

6.

F. Ouallouche et al., “Infrared and microwave image fusion for rainfall detection over northern Algeria,” Int. J. Image Graphics Signal Process., 6 (6), 11 –18 (2014). http://dx.doi.org/10.5815/ijigsp Google Scholar

7.

M. A. Smeelen et al., “Semi-hidden target recognition in gated viewer images fused with thermal IR images,” Inf. Fusion, 18 131 –147 (2014). http://dx.doi.org/10.1016/j.inffus.2013.08.001 Google Scholar

8.

J. Coddington, “Image fusion for art analysis,” Proc. SPIE, 5404 585 –596 (2011). http://dx.doi.org/10.1117/12.543549 Google Scholar

9.

S. Daneshvar and H. Ghassemian, “MRI and PET image fusion by combining IHS and retina-inspired models,” Inf. Fusion, 11 (2), 114 –123 (2010). http://dx.doi.org/10.1016/j.inffus.2009.05.003 Google Scholar

10.

D. K. Sahu and M. Parsai, “Different image fusion techniques-a critical review,” Int. J. Mod. Eng. Res., 2 (5), 4298 –4301 (2012). Google Scholar

11.

X. Qu, C. Hu and J. Yan, “Image fusion algorithm based on orientation information motivated pulse coupled neural networks,” in 7th World Congress on Intelligent Control and Automation (WCICA’08), 2437 –2441 (2008). http://dx.doi.org/10.1109/WCICA.2008.4593305 Google Scholar

12.

W. Kong, Y. Lei and X. Ni, “Fusion technique for grey-scale visible light and infrared images based on non-subsampled contourlet transform and intensity-hue-saturation transform,” IET Signal Proc., 5 (1), 75 –80 (2011). http://dx.doi.org/10.1049/iet-spr.2009.0263 Google Scholar

13.

S. Li, X. Kang and J. Hu, “Image fusion with guided filtering,” IEEE Trans. Image Process., 22 (7), 2864 –2875 (2013). http://dx.doi.org/10.1109/TIP.2013.2244222 Google Scholar

14.

Z. H. Zhou and M. Tan, “Infrared image and visible image fusion based on wavelet transform,” Adv. Mater. Res., 756–759 (2), 2850 –2856 (2014). http://dx.doi.org/10.4028/www.scientific.net/AMR.756-759.2850 Google Scholar

15.

W. L. Li, “Infrared and visible image fusion algorithm based on the target segmentation,” Comput. Simul., 31 (11), 358 –361 (2014). Google Scholar

16.

U. S. Kumar, B. Vikram and P. J. Patil, “Enhanced image fusion algorithm using Laplacian pyramid,” Int. J. Eng. Sci. Res., 4 (7), 525 –532 (2014). Google Scholar

17.

A. Sahu et al., “Medical image fusion with Laplacian pyramids,” in Proc. of IEEE Int. Conf. on Medical Imaging, M-Health & Emerging Communication Systems, 448 –453 (2014). http://dx.doi.org/10.1109/MedCom.2014.7006050 Google Scholar

18.

A. Krishn, V. Bhateja and A. Sahu, “Medical image fusion using combination of PCA and wavelet analysis,” in Int. Conf. on Advances in Computing, Communications and Informatics (ICACCI’14), 986 –991 (2014). http://dx.doi.org/10.1109/ICACCI.2014.6968636 Google Scholar

19.

Y. Yang et al., “Multi-focus image fusion using an effective discrete wavelet transform based algorithm,” Meas. Sci. Rev., 14 (2), 102 –108 (2014). http://dx.doi.org/10.2478/msr-2014-0014 Google Scholar

20.

H. Lin et al., “Remotely sensing image fusion based on wavelet transform and human vision system,” Int. J. Signal Process. Image Process. Pattern Recognit., 8 (7), 291 –298 (2015). http://dx.doi.org/10.14257/ijsip Google Scholar

21.

J. Saeedi and K. Faez, “Infrared and visible image fusion using fuzzy logic and population-based optimization,” Appl. Soft Comput., 12 (3), 1041 –1054 (2012). http://dx.doi.org/10.1016/j.asoc.2011.11.020 Google Scholar

22.

G. M. Taher, M. E. Wahed and G. E. Taweal, “New approach for image fusion based on curvelet approach,” Int. J. Adv. Comput. Sci. Appl., 5 (7), 67 –73 (2014). Google Scholar

23.

C. Zihong et al., “Visual and infrared image fusion based on contourlet transform,” in Proc. of Int. Industrial Informatics and Computer Engineering Conf. (IIICEC’15), 123 –126 (2015). Google Scholar

24.

E. J. Candès and D. L. Donoho, “New tight frames of curvelets and optimal representations of objects with piecewise C2 singularities,” Commun. Pure Appl. Math., 57 (2), 219 –266 (2004). http://dx.doi.org/10.1002/cpa.v57:2 CPMAMV 0010-3640 Google Scholar

25.

M. N. Do and M. Vetterli, “The contourlet transform: an efficient directional multiresolution image representation,” IEEE Trans. Image Process., 14 (12), 2091 –2106 (2005). http://dx.doi.org/10.1109/TIP.2005.859376 IIPRE4 1057-7149 Google Scholar

26.

G. S. El-Taweel and A. K. Helmy, “Image fusion scheme based on modified dual pulse coupled neural network,” IET Image Proc., 7 (5), 407 –414 (2013). http://dx.doi.org/10.1049/iet-ipr.2013.0045 Google Scholar

27.

Y. Na and K. M. Deng, “Fusion of multi-focus images based on a combination of the weighting fractal features and average gradient,” Electron. Sci. Technol., 28 (6), 68 –71 (2015). Google Scholar

28.

Y. Liu and Z. Wang, “Simultaneous image fusion and denoising with adaptive sparse representation,” IET Image Process., 9 (5), 347 –357 (2015). http://dx.doi.org/10.1049/iet-ipr.2014.0311 Google Scholar

Biography

Shuo Liu is currently pursuing his PhD in information and communications engineering at Changchun University of Science and Technology, Changchun, China. His research interests include digital signal processing, image processing, and field programmable gate array technology.

Yan Piao received her PhD in digital signal processing from Chinese Academy of Sciences, Changchun Institute of Optics, Changchun, China, in 2000. She is a professor at the School of Electronics and Information Engineering, Changchun University of Science and Technology, Changchun, China. Her major research interests include digital signal processing, image processing, and three-dimensional imaging technology.

Muhammad Tahir received his BS degree in telecommunication engineering and his MS degree in electrical engineering from Government College University Faisalabad and COMSATS Institute of Information Technology Islamabad in 2008 and 2013, respectively. His major research interests are RF propagation and energy optimization in wireless sensor networks, wireless body area networks, and underwater wireless sensor networks. Currently, he is pursuing his PhD in telecommunication engineering from the School of Electronics and Information Engineering, Changchun University of Science and Technology, Changchun, China.

CC BY: © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

Citation Download Citation

Shuo Liu, Yan Piao, and Muhammad Tahir "Research on fusion technology based on low-light visible image and infrared image," Optical Engineering 55(12), 123104 (8 December 2016). https://doi.org/10.1117/1.OE.55.12.123104

Published: 8 December 2016

Access the abstract

JOURNAL ARTICLE
9 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

CITATIONS

Cited by 13 scholarly publications and 2 patents.

Explore citations on Lens.org

KEYWORDS

Image fusion

Infrared imaging

Infrared radiation

Visible radiation

Image quality

Image enhancement

Optical engineering

1.

Introduction

2.

Background

Eq. (1)

Eq. (2)

Eq. (3)

3.

Proposed Method

Fig. 1

3.1.

Low-Frequency Coefficients Fusion

Eq. (4)

Eq. (5)

3.2.

High-Frequency Coefficients Fusion

Eq. (6)

Eq. (7)

Eq. (8)

Eq. (9)

Eq. (10)

Eq. (11)

Eq. (12)

4.

Experimental Results and Discussion

4.1.

Evaluation Criteria

Eq. (13)

Eq. (14)

Eq. (15)

4.2.

Grayscale Images Fusion Analysis

Fig. 2

Fig. 3

Table 1

4.3.

Color-Scale Images Fusion Analysis

Fig. 4

Fig. 5

Table 2

Fig. 6

Fig. 7

Table 3

5.

Conclusion

Acknowledgments

References

Biography

Show All Keywords

Keywords/Phrases

Search In:

Publication Years