Although the successes of artificial intelligence in areas such as automatic translation are well known, the application of the powerful techniques of deep learning to current optics research is at a comparatively early stage. However, an area with particular promise for deep learning to accelerate both basic science and applications is in ultrafast optics, where nonlinear light-matter interactions lead to highly complex dynamics, including the emergence of extreme events. In the particular field of nonlinear fibre optics, we have recently reported a number of results that have shown how deep learning can both augment existing experimental techniques as well as provide new theoretical insights into the underlying physics. The objective of this paper is to review a selection of our work in this area.
KEYWORDS: Denoising, Digital mammography, Calibration, Sensors, Interference (communication), Image filtering, Digital breast tomosynthesis, Modulation transfer functions, Signal to noise ratio, Breast
Denoising can be used as a tool to enhance image quality and enforce low radiation doses in X-ray medical imaging. The
effectiveness of denoising techniques relies on the validity of the underlying noise model. In full-field digital
mammography (FFDM) and digital breast tomosynthesis (DBT), calibration steps like the detector offset and flat-fielding
can affect some assumptions made by most denoising techniques. Furthermore, quantum noise found in X-ray
images is signal-dependent and can only be treated by specific filters. In this work we propose a pipeline for FFDM and
DBT image denoising that considers the calibration steps and simplifies the modeling of the noise statistics through
variance-stabilizing transformations (VST). The performance of a state-of-the-art denoising method was tested with and
without the proposed pipeline. To evaluate the method, objective metrics such as the normalized root mean square error
(N-RMSE), noise power spectrum, modulation transfer function (MTF) and the frequency signal-to-noise ratio (SNR)
were analyzed. Preliminary tests show that the pipeline improves denoising. When the pipeline is not used, bright pixels
of the denoised image are under-filtered and dark pixels are over-smoothed due to the assumption of a signal-independent
Gaussian model. The pipeline improved denoising up to 20% in terms of spatial N-RMSE and up to 15% in
terms of frequency SNR. Besides improving the denoising, the pipeline does not increase signal smoothing significantly,
as shown by the MTF. Thus, the proposed pipeline can be used with state-of-the-art denoising techniques to improve the
quality of DBT and FFDM images.
Nonlocal image filters suppress noise and other distortions by searching for similar patches at different locations
within the image, thus exploiting the self-similarity present in natural images. This similarity is typically assessed
by a windowed distance of the patches pixels. Inspired by the human visual system, we introduce a patch foveation
operator and measure patch similarity through a foveated distance, where each patch is blurred with spatially
variant point-spread functions having standard deviation increasing with the spatial distance from the patch
center. In this way, we install a different form of self-similarity in images: the foveated self-similarity.
We consider the Nonlocal Means algorithm (NL-means) for the removal of additive white Gaussian noise as
a simple prototype of nonlocal image filtering and derive an explicit construction of its corresponding foveation
operator, thus yielding the Foveated NL-means algorithm.
Our analysis and experimental study show that, to the purpose of image denoising, the foveated self-similarity
can be a far more effective regularity assumption than the conventional windowed self-similarity. In the comparison
with NL-means, the proposed foveated algorithm achieves a substantial improvement in denoising quality,
according to both objective criteria and visual appearance, particularly due to better contrast and sharpness.
Moreover, foveation is introduced at a negligible cost in terms of computational complexity.
We propose an extension of the BM4D volumetric filter to the denoising of data corrupted by spatially nonuniform
noise. BM4D implements the grouping and collaborative filtering paradigm, where similar cubes of voxels
are stacked into a four-dimensional "group". Each group undergoes a sparsifying four-dimensional transform,
that exploits the local correlation among voxels in each cube and the nonlocal correlation between corresponding
voxels of different cubes. Thus, signal and noise are effectively separated in transform domain. In this work
we take advantage of the sparsity induced by the four-dimensional transform to provide a spatially adaptive
estimation of the local noise variance by applying a robust median estimator of the absolute deviation to the
spectrum of each filtered group. The adaptive variance estimates are then used during coefficients shrinkage.
Finally, the inverse four-dimensional transform is applied to the filtered group, and each individual cube estimate
is adaptively aggregated at its original location.
Experiments on medical data corrupted by spatially varying Gaussian and Rician noise demonstrate the
efficacy of the proposed approach in volumetric data denoising. In case of magnetic resonance signals, the
adaptive variance estimate can be also used to compensate the estimation bias due to the non-zero-mean errors
of the Rician-distributed data.
The block-matching and 3-D filtering (BM3D) algorithm is currently one of the most powerful and effective image denoising procedures. It exploits a specific nonlocal image modelling through grouping and collaborative filtering. Grouping finds mutually similar 2-D image blocks and stacks them together in 3-D arrays. Collaborative filtering produces individual estimates of all grouped blocks by filtering them jointly, through transform-domain shrinkage of the 3-D arrays (groups).
BM3D can be combined with transform-domain alpha-rooting in order to simultaneously sharpen and denoise the image. Specifically, the thresholded 3-D transform-domain coefficients are modified by taking the alpha-root of their magnitude for some alpha > 1, thus amplifying the differences both within and between the grouped blocks. While one can use a constant (global) alpha throughout the entire image, further performance can be achieved by allowing different degrees of sharpening in different parts of the image, based on content-dependent information.
We propose to vary the value of alpha used for sharpening a group through weighted estimates of the low-frequency, edge, and high-frequency content of the average block in the group. This is shown to be a viable approach for image sharpening, and in particular it can provide an improvement (both visually and in terms of PSNR) over its global non-adaptive counterpart.
We propose a powerful video denoising algorithm that exploits temporal and spatial redundancy characterizing
natural video sequences. The algorithm implements the paradigm of nonlocal grouping and collaborative filtering,
where a higher-dimensional transform-domain representation is leveraged to enforce sparsity and thus regularize
the data. The proposed algorithm exploits the mutual similarity between 3-D spatiotemporal volumes constructed
by tracking blocks along trajectories defined by the motion vectors. Mutually similar volumes are grouped
together by stacking them along an additional fourth dimension, thus producing a 4-D structure, termed group,
where different types of data correlation exist along the different dimensions: local correlation along the two
dimensions of the blocks, temporal correlation along the motion trajectories, and nonlocal spatial correlation
(i.e. self-similarity) along the fourth dimension. Collaborative filtering is realized by transforming each group
through a decorrelating 4-D separable transform and then by shrinkage and inverse transformation. In this way,
collaborative filtering provides estimates for each volume stacked in the group, which are then returned and
adaptively aggregated to their original position in the video. Experimental results demonstrate the effectiveness
of the proposed procedure which outperforms the state of the art.
The deblurring of images corrupted by radial blur is studied. This type of blur appears in images acquired during an
any
camera translation having a substantial component orthogonal to the image plane. The point spread functions (PSF PSF)
describing this blur are spatially varying. However, this blurring process does not mix together pixels lying on differen different
radial lines, i.e. lines stemming from a unique point in the image, the so called "blur center". Thus, in suitable pola polar
coordinates, the blurring process is essentially a 1-D linear operator, described by the multiplication with the blurrin blurring
matrix.
We consider images corrupted simultaneously by radial blur and noise. The proposed deblurring algorithm is base
based
on two separate forms of regularization of the blur inverse. First, in the polar domain, we invert the blurring matri matrix
using the Tikhonov regularization. We then derive a particular modeling of the noise spectrum after both the regularize regularized
inversion and the forward and backward coordinate transformations. Thanks to this model, we successfully use a denoisin denoising
algorithm in the Cartesian domain. We use a non-linear spatially adaptive filter, the Pointwise Shape-Adaptive DCT, i in
order to exploit the image structures and attenuate noise and artifacts.
Experimental results demonstrate that the proposed algorithm can effectively restore radial blurred images corrupted by additive white Gaussian noise.
We propose an image restoration technique exploiting regularized inversion and the recent block-matching and 3D
filtering (BM3D) denoising filter. The BM3D employs a non-local modeling of images by collecting similar image
patches in 3D arrays. The so-called collaborative filtering applied on such a 3D array is realized by transformdomain
shrinkage. In this work, we propose an extension of the BM3D filter for colored noise, which we use in
a two-step deblurring algorithm to improve the regularization after inversion in discrete Fourier domain. The
first step of the algorithm is a regularized inversion using BM3D with collaborative hard-thresholding and the
seconds step is a regularized Wiener inversion using BM3D with collaborative Wiener filtering. The experimental
results show that the proposed technique is competitive with and in most cases outperforms the current best
image restoration methods in terms of improvement in signal-to-noise ratio.
We propose a novel approach for joint denoising and interpolation of noisy Bayer-patterned data acquired from a digital imaging sensor (e.g., CMOS, CCD). The aim is to obtain a full-resolution RGB noiseless image. The proposed technique is specifically targeted to filter signal-dependant, e.g. Poissonian, or heteroscedastic noise, and effectively exploits the correlation between the different color channels. The joint technique for denoising and interpolation is based on the concept of local polynomial approximation (LPA) and intersection of confidence intervals (ICI). These directional filters utilize simultaneously the green, red, and blue color channels. This is achieved by a linear combination of complementary-supported smoothing and derivative kernels designed for the Bayer data grid. With these filters, the denoised and the interpolated estimates are obtained by convolutions over the Bayer data. The ICI rule is used for data-adaptive selection of the length of the designed cross-color directional filter. Fusing estimates from multiple directions provides the final anisotropic denoised and interpolated values. The full-size RGB image is obtained by placing these values into the corresponding positions in the image grid. The efficiency of the proposed approach is demonstrated by experimental results with simulated and real camera data.
A transform-domain fringe pattern denoising technique is presented. The Discrete Cosine Transform (DCT) is applied
in a sliding window manner to get an overcomplete image expansion, and then the transform coefficients are
thresholded to reduce the noise. We investigate the proper size of the sliding window and the proper threshold level. The
latter is determined individually for each window position using a local noise variance estimate. In order to deal with a
rather inadequate but simplified noise model, a proportionality factor, related with the speckle size, is found by
experiments with digitally simulated speckle fringes. Such a proportionality factor suggests that the technique could be
made fully automatic. We demonstrate promising results in denoising of real speckle fringe patterns, obtained through
an out-of-plane sensitive Digital Speckle Pattern Interferometry (DSPI) set-up in a process of non-destructive testing of
reinforced composite materials deformation.
The shape-adaptive DCT (SA-DCT) can be computed on a support of arbitrary shape, but retains a computational complexity comparable to that of the usual separable block DCT. Despite the near-optimal decorrelation and energy compaction properties, application of the SA-DCT has been rather limited, targeted nearly exclusively to video compression. It has been recently proposed by the authors8 to employ the SA-DCT for still image denoising. We use the SA-DCT in conjunction with the directional LPA-ICI technique, which defines the shape of the transform's support in a pointwise adaptive manner. The thresholded or modified SA-DCT coefficients are used to reconstruct a local estimate of the signal within the adaptive-shape support. Since supports corresponding to different points are in general overlapping, the local estimates are averaged together using adaptive weights that depend on the region's statistics. In this paper we further develop this novel approach and extend it to more general restoration problems, with particular emphasis on image deconvolution. Simulation experiments show a state-of-the-art quality of the final estimate, both in terms of objective criteria and visual appearance. Thanks to the adaptive support, reconstructed edges are clean, and no unpleasant ringing artifacts are introduced by
the fitted transform.
We present a novel approach to still image denoising based on effective filtering in 3D transform domain by combining sliding-window transform processing with block-matching. We process blocks within the image in a sliding manner and utilize the block-matching concept by searching for blocks which are similar to the currently
processed one. The matched blocks are stacked together to form a 3D array and due to the similarity between them, the data in the array exhibit high level of correlation. We exploit this correlation by applying a 3D decorrelating unitary transform and effectively attenuate the noise by shrinkage of the transform coefficients. The subsequent inverse 3D transform yields estimates of all matched blocks. After repeating this procedure for all image blocks in sliding manner, the final estimate is computed as weighed average of all overlapping blockestimates. A fast and efficient algorithm implementing the proposed approach is developed. The experimental
results show that the proposed method delivers state-of-art denoising performance, both in terms of objective criteria and visual quality.
We consider a signal restoration from observations corrupted by random noise. The local maximum likelihood technique allows to deal with quite general statistical models of signal dependent observations, relaxes the standard parametric modelling of the standard maximum likelihood, and results in flexible nonparametric regression estimation of the signal. We deal with the anisotropy of the signal using multi-window directional sectorial local polynomial approximation. The data-driven sizes of the sectorial windows, obtained by the intersection of confidence interval (ICI) algorithm, allow to form starshaped adaptive neighborhoods used for the pointwise estimation. The developed approach is quite general and is applicable for multivariable data. A fast adaptive algorithm implementation is proposed. It is applied for photon-limited imaging with the Poisson distribution of data. Simulation experiments and comparison with some of the best results in the field demonstrate an advanced performance of the developed algorithms.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.