1.

Introduction

An image captured by an imaging system represents the degraded version of an original image due to blurring and additive noise.^{1, 2} For a size $M\times N$ image, a typical degradation model can be written as

where the vectors $\mathbf{x}$ , $\mathbf{y}$ , and $\mathbf{n}$ are the lexicographically ordered original image, the observed image, and the additive noise of size $MN\times 1$ , respectively. In addition, $\mathbf{H}$ is the degradation matrix of size $MN\times MN$ to represent a spatially invariant or spatially varying point spread function (PSF).

Regularized constrained least squares (RCLS) have been widely used to obtain a solution for Eq. 1, and the solution is obtained by minimizing the following function with respect to $\mathbf{x}$ ,^{1, 2}

where $\alpha$ denotes the regularization parameter to control the tradeoff between fidelity to the data and smoothness, and $\mathbf{C}$ typically represents a high-pass operator.

The prior knowledge used in RCLS is that the original image is smooth. However, this is a global requirement and therefore not effective in terms of local smoothness. Also, when an iterative technique is used to obtain the solution of Eq. 2, the iterative solution may suffer from noise amplification after a certain iteration step when the additive noise is serious.^{2, 3, 4} Therefore, a more desirable solution can be obtained by imposing reasonable constraints into the solution space of RCLS.^{3, 4, 5} Projection onto convex sets (POCS) has been widely used in many areas, since it is effective to impose nonlinear local properties into the solution space.^{3, 6}

In this work, an iterative adaptive hybrid image restoration algorithm using a local smoothing constraint is presented. We follow the formulation represented by Eq. 2 and propose to bring knowledge about local properties of the original image into the restoration process, so that prior knowledge and spatial adaptivity are incorporated in the solution. The basic motivation is to constrain local ranges of values that the restored image can take, leading to increased convergence speed of the iterative algorithm and performance improvement compared to RCLS.

This work is organized as follows. In Sec. 2, the proposed hybrid gradient-projection restoration algorithm is explained. Experimental results and conclusions are presented in Sec. 3.

2.

Proposed Algorithm

The gradient iteration of the regularized smoothing functional in Eq. 2 can be written as

There exist various ways for determining the regularization parameter $\alpha$ .^{1, 2, 7} According to Ref. 7, we determine the regularization parameter bywhere $\theta ⩾2{\Vert \mathbf{y}\Vert }^2$ . The iterative solution in Eq. 3 reflects only global smoothing requirements and therefore it is not effective in terms of local smoothness. When a local smoothing constraint is imposed into the solution space of RCLS, the iteration in Eq. 3 can take the formwhere $P$ represents a projection operator of a signal onto a set with desirable local properties. The proposed projection set is defined by the local statistics of the partially restored image at each iteration step. For a pixel of the partially restored image ${\widehat{x}}_k(i,j)$ , the local mean and variance at coordinate $(i,j)$ with window $W$ are defined aswhere $W=(2U+1)\times (2V+1)$ is the extent of the analysis window that is symmetric about the point $(i,j)$ . In addition, the local maximum of the partially restored image at point $(i,j)$ is defined aswhere $S_{(i,j)}$ is the support region that determines the local maximum about the point $(i,j)$ . In this work, it is same with the analysis window used for the local mean and variance. Using the local information in Eqs. 6, 7, a parameter controlling the degree of local smoothing is defined asfor $L$ bits per pixel. In Eq. 8, $T_k$ is a threshold to be determined and $B(i,j)$ is defined asIn Eq. 9, $SW$ and $LW$ represent the relatively small and large window sizes centered at point $(i,j)$ .

In general, noise amplification contributes to the problem of restoration, and therefore desirable properties such as detection of noisy pixels and the degree of local activity should be incorporated into the restoration process to effectively suppress noise amplification. It is clear how the local statistics affect $B(i,j)$ . Let us assume that ${\widehat{x}}_k(i,j)$ is a noisy pixel belonging to an element of a homogeneous activity region. In such a case, ${\sigma }_{{\widehat{x}}_{k,SW}}^2(i,j)$ is greater than ${\sigma }_{{\widehat{x}}_{k,LW}}^2(i,j)$ , and therefore local smoothing is necessary to suppress the noise amplification. As the noise increases, the variance ratio decreases, leading to smaller $B(i,j)$ . In addition, when the same noise is added to low and high activity regions, the noise within the low activity region is more visible than that within the high activity region. Therefore, tighter bounds should be applied to the noisy pixel within the low activity region than to that within the high activity region. ${\widehat{x}}_{k,SW,\max }(i,j)∕m_{{\widehat{x}}_{k,SW}}(i,j)$ in Eq. 9 is used as a way to represent the degree of local activity in this work. As the local activity decreases, $B(i,j)$ decreases. A smaller $B(i,j)$ leads to tighter bounds, while looser bounds are for a larger $B(i,j)$ , so that the noise amplification is effectively suppressed. They are in agreement with the noise masking property in areas of high spatial activity of the human visual system.⁸

Local information has the limit to effectively suppress noise amplification. Therefore, global information about the noise can help to effectively reduce noise amplification. For example, tighter bounds are more desirable as the additive noise increases. $T_k$ in Eq. 8 is used to incorporate global information into the restoration process, so that it can control the degree of bounds that is computed at each iteration step, such as

where $Z$ represents a constant, and ${\mathbf{n}}_{k-1}=\mathbf{y}-\mathbf{H}{\mathbf{x}}_{k-1}$ .

The proposed algorithm is used to obtain a solution that is an element of the intersection set between a solution space using the gradient approach reflecting the global smoothing constraint, and a projection set incorporating the local smoothing constraint.

3.

Experimental Results and Conclusions

The proposed adaptive hybrid image restoration algorithm is tested with various noisy blurred images and degradation, and it is compared to typical RCLS. In the set of such experiments, the $256\times 256\text{pixels}$ Lena and Cameraman images are described here. The original images were blurred by $7\times 7$ motion blur and Gaussian blur with variance of 5. A Gaussian-distributed noise signal was added to the blurred images. We tested the proposed algorithm for various SNRs. In addition, a 2-D Laplacian operator was used for the high-pass operator in Eq. 2.² For evaluating the objective and subjective performance of the algorithm, the mean squared error (MSE) and structural similarity index (SSIM) in Ref. 9 were used, where SSIM takes a value between $-1$ (the worst value) and 1 (the best value). Also,

was used for terminating the iteration. In these experiments, $m=5$ was used for 5- and $10\text{-}\mathrm{dB}$ noises. Also, $m=6$ and $m=7$ were used for 20- and $30\text{-}\mathrm{dB}$ noises, respectively. In addition, $SW=3$ and $LW=7$ were used for small and large size windows in Eq. 9.

Figures 1, 1, 1 show the degraded Lena image ( $7\times 7$ Gaussian blur with variance of 5- and $10\text{-}\mathrm{dB}$ Gaussian noise), the restored image with RCLS, and the restored image with the proposed method, respectively. For $Z=0.07$ , the proposed algorithm converges after ten iterations ( $\mathrm{MSE}=260$ and $\mathrm{SSIM}=0.677$ ), while RCLS requires 39 iterations ( $\mathrm{MSE}=493$ and $\mathrm{SSIM}=0.432$ ). The results show that RCLS leads to noise amplification in the restored image. On the other hand, the proposed hybrid algorithm effectively suppresses the noise amplification. When tighter bounds (smaller $Z$ ) are used, the convergence becomes faster. However, tighter bounds result in oversmoothed images. On the basis of our experiments, $0.05⩽Z⩽0.1$ is a good range with convergence speed and performance.

Figures 2 and 2 show MSE and convergence speed comparisons as a function of iteration number for Lena image when $10\text{-}\mathrm{dB}$ Gaussian noise is added. The results show that the proposed algorithm has the capability to keep a minimum mean squared error, while RCLS is very sensitive to the terminating criterion, since the noise is amplified after a certain iteration number. Also, it is verified that the proposed algorithm is faster than RCLS for all cases.

Tables 1, 2 summarize the performance comparisons (MSE, SSIM, and iteration number) at convergence for Gaussian blur and motion blur, respectively. As expected, the performance gain of the proposed algorithm increases as the additive noise increases. However, as the additive noise decreases, looser bounds are necessary to avoid oversmoothed images. As shown in Tables 1, 2, the proposed algorithm performs similarly to RCLS for additive noise less than $30\mathrm{dB}$ .

Table 1

Performance comparisons at convergence (Lena).

Table 2

Performance comparisons at convergence (Cameraman).

The novelty of the proposed algorithm is that it leads to faster convergence speed, and subjectively and objectively better performance than typical RCLS without prior information of an original image by incorporating the local smoothing constraint.

We propose an iterative adaptive hybrid image restoration algorithm using a local smoothing constraint. Each pixel in an image is projected onto a local smoothing set determined by the local mean, variance, and maximum intensity value of the partially restored image. The ratio of these parameters is used to define the convex set, resulting in effective suppression of the noise amplification and faster convergence speed.