KEYWORDS: Model based design, Data modeling, Mathematical optimization, Signal filtering, Tunable filters, Neural networks, Image processing, Deblurring, Process modeling
Co-design consists of optimizing the parameters of the lens and the processing together to obtain a gain in performance for the entire optical/processing chain, which requires new optimization tools, as traditional optical design ones can no longer be used easily. In the state of the art, joint optical/processing optimization methods based on statistical scene and blur models and analytical performance models have been proposed, and recently, new approaches based on the joint optimization of a neural network and optical components with a large database have been developed. However, to the best of our knowledge, no comparison of co-design results using either a model-based or a data-based approach for the same task have been conducted, which is the scope of this paper. We consider here the optimization of phase masks to extend the camera depth of field. We compare the optimization results using a performance model based on restoration error using either a generalized Wiener filter, or a neural network. We investigate the optimization trend depending on the neural network complexity, the starting point of the optimization and the possible interaction between the two approaches.
Optimizing freeform systems can encounter convergence difficulties due to the many degrees of freedom that these surfaces bring to optical systems. Moreover, the description of these freeform surfaces in a polynomial basis may impose prior knowledge on the shape of the surface. In this presentation, we will showcase a differential ray tracer with NURBS capabilities called FORMIDABLE. In contrast to available commercial optical design software, such as Zemax OpticStudio and Synopsys Code V, this library ican simulate and especially optimize Non-Uniform Rational B-Spline (NURBS) surfaces. The key advantage of NURBS lies in their ability to locally describe an optical surface, thereby minimizing preconceived notions about the surface shape, aside from the surface sampling determined by the density of the NURBS representation. The main drawback, however, is the significant increase in the degrees of freedom within the optical system, making the optimization of these surfaces a complex task with a conventional commercial optical design software. FORMIDABLE's implementation of differential ray-tracing capabilities allows faster convergence of systems described by many degrees of freedom and makes optimization with NURBS surfaces viable. The features of FORMIDABLE will first be described. Then its capabilities will be illustrated with the optimization of a classical non-reimaging Three-Mirror Anastigmat (TMA) by considering either a description of surfaces by NURBS or a description by the polynomial basis XY. Then, this optimized TMA will be compared with its equivalent optimized with Zemax OpticStudio. To enable this software comparison, we will use the same starting point and practically the same merit function. Standard metrics, such as Root Mean Square (RMS) spot size across the field of view (FOV) will be used to assess the imaging quality.
KEYWORDS: Wavefronts, Optical surfaces, Modulation transfer functions, Imaging systems, Zemax, Etching, Digital signal processing, Diffractive optical elements, Design and modelling
We compare different methods to extend the depth of focus of a fast infrared imaging system. Instead of using a phase mask for wavefront coding, we place this element directly on an optical surface.
We compare different methods to optimize end-to-end a hybrid optical/digital system for best and most uniform performance over the field-of-view with the Synopsys® CodeV® lens design software. We have extended the native optimization capability of this software by implementing different methods that leverage the deconvolution during the optimization process of the hybrid optical system as a whole, including simultaneously their optical and digital image processing parts. We show that the joint optimization of the lens and the processing through a true restored-image quality criterion significantly enhances the final post-processed image quality, and allows to fine-tune the residual balancing between on-axis and peripheral fields with simple weighting coefficients.
A phase mask in the aperture stop of an imaging system can enhance its depth of field (DoF). This DoF extension capacity can be maximized by jointly optimizing the phase mask and the digital processing algorithm used to deblur the acquired image. This method, introduced by Cathey and Dowski with a cubic phase mask, has been generalized to different mask models. Among them, annular binary phase masks are easy to manufacture, and can be co-optimized with a simple unique Wiener deconvolution filter. Their performance and robustness have been characterized theoretically and experimentally in the case of monochromatic illumination. We perform here a theoretical and experimental study of codesigned DoF enhancing binary phase masks in panchromatic imagers. At first glance, this configuration is not optimal for binary phase masks. Indeed, the binary phase masks are most often manufactured by binary etching of a dielectric plate, so dephasing depends on the wavelength. The π radians dephasing is reached for only one wavelength. How do phase masks optimized for a particular wavelength respond to a wide illumination spectrum? Is it possible to take into account the illumination spectrum in the co-optimization of phase masks? What impact does this have on the result? We analyze the behavior of DoF enhancing phase masks in panchromatic imagers in terms of Modulation Transfer Function and of final image quality. The results are experimentally validated with imaging experiments carried out with a commercial lens, a Vis-NIR CMOS sensor and co-optimized phase masks. We study different phase masks co-optimized for different spectrum of illumination. We show that masks specifically optimized for wide spectrum illumination perform better under this type of illumination than monochromatically optimized phase masks under monochromatic illumination, especially when the targeted DoF range is large.
Today, almost all imaging systems include both an optical part and an image processing part in order to improve the final image quality. It therefore seems natural to optimize them simultaneously to obtain the best possible result. However, even if this “co-design” approach is more and more recognized at a conceptual level, it is still rarely used in practice for designing complex lenses with many ajustable parameters and constraints. This is due to the fact that the contribution of image processing is currently difficult to take into account in optical design software. Until now, the field of co-design has thus mainly focused on simpler imaging systems, consisting for example of single co-optimized optical elements such as phase masks. More recently, Robinson and Stork have been working on the possibility of integrating the image processing criterion known as mean square error (MSE) in the optical software Zemax OpticStudio. It is also possible to consider surrogate criteria instead of this MSE, built from more classical optical criteria (modulation transfer function, point spread function, etc.) [Burcklen et al. (2018)]. In this study, we investigate the possibility of implementing the MSE criterion in the CodeV optical design software in a way that is easily usable by an optical designer. We compare systems co-designed with this approach to systems jointly optimized with surrogate criteria or conventionally optimized. We focus on the performance differences between these different approaches and on the opportunities offered by CodeV for co-design.
We investigate the depth of field (DoF) enhancing capacity of binary annular phase masks embedded in panchromatic imaging systems. We first demonstrate with numerical simulations and real-world imaging experiments that phase masks optimized for monochromatic illumination are somewhat robust to their use under wide spectrum illumination: they provide images that are slightly less sharp but less affected by deconvolution artifacts due to spectral averaging. Then, we show that masks specifically optimized for wide spectrum illumination perform better under this type of illumination than monochromatically optimized phase masks under monochromatic illumination, especially when the targeted DoF range is large. This interesting effect comes from the fact that deconvolution artifacts are significantly reduced by wide spectrum illumination. These results show that it is useful to take into account the illumination spectrum together with the scene characteristics and the targeted DoF range for effective co-design of DoF enhancing imaging systems.
We investigate the practical behavior of a co-optimized hybrid system involving a generic binary phase mask and digital deconvolution. We perform experiments with a case-study optical system with observed scene lighting by LED of different colors. By imaging a real scene and a depth of field (DoF) target, we show that the DoF reachable in practice matches with good accuracy the one predicted by simulation in case of monochromatic illumination. We also characterize the drop in performance when using this type of system with actual illumination wavelength departing from the nominal one.
We experimentally investigate the performance of co-optimized hybrid optical–digital imaging systems based on binary phase masks and digital deconvolution for extended depth-of-field (DoF) under narrow-band illumination hypothesis. These systems are numerically optimized by assuming a simple generic imaging model. Using images of DoF targets and real scenes, we experimentally demonstrate that in practice, they actually reach the DoF range for which they have been optimized. Moreover, they are shown to be robust against small mask manufacturing errors and residual spherical aberration in the optical system. These results demonstrate that the optical/digital optimization protocol based on generic imaging model can be safely used to design DoF-enhanced imaging systems aimed at real-world applications.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.