Standard imaging techniques do not get as much information from a scene as light-field imaging. Light-field (LF) cameras can measure the light intensity reflected by an object and, most importantly, the direction of its light rays. This information can be used in different applications, such as depth estimation, in-plane focusing, creating full-focused images, etc. However, standard key-point detectors often employed in computer vision applications cannot be applied directly to plenoptic images due to the nature of raw LF images. This work presents an approach for key-point detection dedicated to plenoptic images. Our method allows using of conventional key-point detector methods. It forces the detection of this key-point in a set of micro-images of the raw LF image. Obtaining this important number of key-points is essential for applications that require finding additional correspondences in the raw space, such as disparity estimation, indirect visual odometry techniques, and others. The approach is set to the test by modifying the Harris key-point detector.
Being able to identify defects is an essential step during manufacturing processes. Yet, not all defects are necessarily known and sufficiently well described in the databases images. The challenge we address in this paper is to detect any defect by fitting a model using only normal samples of industrial parts. For this purpose, we propose to test fast AnoGAN (f-AnoGAN) approach based on a generative adversarial network (GAN). The method is an unsupervised learning algorithm, that contains two phases; first, we train a generative model using only normal images, which proposes a fast mapping of new data into the latent space. Second, we add and train an encoder to reconstruct images. The anomaly detection is defined by the reconstruction error between the defected data and the reconstructed ones, and the residual error of the discriminator. For our experiments, we use two sets of industrial data; the MVTec Anomaly Detection Dataset and a private dataset which is based on thermal-wave and used for non-destructive testing. This technique has been utilized in research for the evaluation of industrial materials. Applying the f-AnoGAN in this domain offers high anomaly detection accuracy.
Light-field and plenoptic cameras are widely available today. Compared with monocular cameras, these cameras capture not only the intensity but also the direction of the light rays. Due to this specificity, light-field cameras allow for image refocusing and depth estimation using a single image. However, most of the existing depth estimation methods using light-field cameras require a prior complex calibration phase and raw data preprocessing before the desired algorithm is applied. We propose a homography-based method with plenoptic camera parameters calibration and optimization, dedicated to our homography-based micro-images matching algorithm. The proposed method works on debayerred raw images with vignetting correction. The proposed approach directly links the disparity estimation in the 2D image plane to the depth estimation in the 3D object plane, allowing for direct extraction of the real depth without any intermediate virtual depth estimation phase. Also, calibration parameters used in the depth estimation algorithm are directly estimated, and hence no prior complex calibration is needed. Results are illustrated by performing depth estimation with a focused light-field camera over a large distance range up to 4 m.
This work presents how deflectometry can be coupled with a light-field camera to better characterize and quantify the depth of anomalies on specular surfaces. In our previous work,1 we proposed a new scanning scheme for the detection and 3D reconstruction of defects on reflective objects. However, the quality of the reconstruction was strongly dependent on the object-camera distance which was required as an external input parameter. In this paper, we propose a new approach that integrates an estimation of this distance into our system by replacing the standard camera with a light-field camera.
Light-Field (LF) cameras allow the extraction not only of the intensity of light but also of the direction of light rays in the scene, hence it records much more information of the scene than a conventional camera. In this paper, we present a novel method to detect key-points in raw LF images by applying key-points detectors on Pseudo-Focused images (PFIs). The main advantage of this method is that we don’t need to use complex key-points detectors dedicated to light-field images. We illustrate the method in two use cases: the extraction of corners in a checkerboard and the key-points matching in two view raw light-field images. These key-points can be used for different applications e.g. calibration, depth estimation or visual odometry. Our experiments showed that our method preserves the accuracy of detection by re-projecting the pixels in the original raw images.
Widely used for surface slopes measurements and for three-dimensional shape reconstruction, deflectometry is a particularly powerful technique that can also be applied for defects detection on specular surfaces. In the visible domain, deflectometry is usually based on the projection of complex encoded light patterns and necessitates heavy processing that makes it not suitable for inline inspection. In this paper, A new deflectometry based approach that is more adapted for inline inspection of linearly moving parts (parts on conveyors) is proposed. Based on a more affordable and a simpler hardware setup, the new approach allows at the same time for a proper localization and a precise geometrical quantification of any defects on the scanned specular surfaces. The proposed approach uses a fast and simple processing algorithm that lends itself very well to real-time inspection. The new method is tested and validated in laboratory for the inspection of defects on specular surfaces of plastic parts.
During the last two decades the number of visual odometry algorithms has grown rapidly. While it is straightforward to obtain a qualitative result, if the shape of the trajectory is in accordance with the movement of the camera, a quantitative evaluation is needed to evaluate the performances and to compare algorithms. In order to do so, one needs to establish a ground truth either for the overall trajectory or for each camera pose. To this end several datasets have been created. We propose a review of the datasets created over the last decade. We compare them in terms of acquisition settings, environment, type of motion and the ground truth they provide. The purpose is to allow researchers to rapidly identifies the datasets that best fit their work. While the datasets cover a variety of techniques to establish a ground truth, we provide also the reader with techniques to create one that were not present among the reviewed datasets.
This paper proposes a fully automated vision system to inspect the whole surface of crankshafts, based on the magnetic particle testing technique. Multiple cameras are needed to ensure the inspection of the whole surface of the crankshaft in real-time. Due to the very textured surface of crankshafts and the variability in defect shapes and types, defect detection methods based on deep learning algorithms, more precisely convolutional neural networks (CNNs), become a more efficient solution than traditional methods. This paper reviews the various approaches of defect detection with CNNs, and presents the advantages and weaknesses of each approach for real-time defect detection on crankshafts. It is important to note that the proposed visual inspection system only replaces the manual inspection of crankshafts conducted by operators at the end of the magnetic particle testing procedure.
In computer vision, the epipolar geometry embeds the geometrical relationship between two views of a scene. This geometry is degenerated for planar scenes as they do not provide enough constraints to estimate it without ambiguity. Nearly planar scenes can provide the necessary constraints to resolve the ambiguity. But classic estimators such as the 5-point or 8-point algorithm combined with a random sampling strategy are likely to fail in this case because a large part of the scene is planar and it requires lots of trials to get a nondegenerated sample. However, the planar part can be associated with a homographic model and several links exist between the epipolar geometry and homographies. The epipolar geometry can indeed be recovered from at least two homographies or one homgraphy and two noncoplanar points. The latter fits a wider variety of scenes, as it is unsure to be able to find a second homography in the noncoplanar points. This method is called plane-and-parallax. The equivalence between the parallax and the epipolar lines allows to recover the epipole as their common intersection and the epipolar geometry. Robust implementations of the method are rarely given, and we encounter several limitations in our implementation. Noisy image features and outliers make the lines not to be concurrent in a common point. Also off-plane features are unequally influenced by the noise level. We noticed that the bigger the parallax is, the lesser the noise influence is. We, therefore, propose a model for the parallax that takes into account the noise on the features location to cope with the previous limitations. We call our method the “parallax beam.” The method is validated on the KITTI vision benchmark and on synthetic scenes with strong planar degeneracy. The results show that the parallax beam improves the estimation of the camera motion in the scene with planar degeneracy and remains usable when there is not any particular planar structure in the scene.
In the past few years, a new type of camera has been emerging on the market: a digital camera capable of capturing both the intensity of the light emanating from a scene and the direction of the light rays. This camera technology called a light-field camera uses an array of lenses placed in front of a single image sensor, or simply, an array of cameras attached together. An optical device is proposed: a four minilens ring that is inserted between the lens and the image sensor of a digital camera. This device prototype is able to convert a regular digital camera into a light-field camera as it makes it possible to record four subaperture images of the scene. It is a compact and cost-effective solution to perform both postcapture refocusing and depth estimation. The minilens ring makes also the plenoptic camera versatile; it is possible to adjust the parameters of the ring so as to reduce or increase the size of the projected image. Together with the proof of concept of this device, we propose a method to estimate the positions of each optical component depending on the observed scene (object size and distance) and the optics parameters. Real-world results are presented to validate our device prototype.
This work shows the interest of combining polarimetric and light-field imaging. Polarimetric imaging is known for its capabilities to highlight and reveal contrasts or surfaces that are not visible in standard intensity images. This imaging mode requires to capture multiple images with a set of different polarimetric filters. The images can either be captured by a temporal or spatial multiplexing, depending on the polarimeter model used. On the other hand, light-field imaging, which is categorized in the field of computational imaging, is also based on a combination of images that allows to extract 3D information about the scene. In this case, images are either acquired with a camera array, or with a multi-view camera such as a plenoptic camera. One of the major interests of a light-field camera is its capability to produce different kind of images, such as sub-aperture images used to compute depth images, full focus images or images refocused at a specific distance used to detect defects for instance. In this paper, we show that refocused images of a light-field camera can also be computed in the context of polarimetric imaging. The 3D information contained in the refocused images can be combined with the linear degree of polarization and can be obtained with an unique device in one acquisition. An example illustrates how these two coupled imaging modes are promising, especially for the industrial control and inspection by vision.
This work shows the interests of the refocusing technics in the domain of industrial vision. A prototype of light field camera is used for computing refocused images, which are calibrated in depth. These images are computed with a method previously presented, using a multi-view camera modeled following the “variable homography” principles. The camera prototype is composed by 4 mini-lenses placed behind a single CCD sensor, calibrated and able to perform 3D measurements. As the device is calibrated in depth, we link refocused images to a selected depth. Contrary to the conventional imaging, with refocused depth calibrated images, it is possible to highlight planes of interest to facilitate vision inspection. Objects can be distinguished from others according to their depths. We also show that a pixel metric scale can be estimated at different depths, avoiding the use of other measurement devices. Two standard vision examples are presented to illustrate the interests of this approach.
We propose a model of depth camera based on a four-lens device. This device is used for validating alternate approaches for calibrating multiview cameras and also for computing disparity or depth images. The calibration method arises from previous works, where principles of variable homography were extended for three-dimensional (3-D) measurement. Here, calibration is performed between two contiguous views obtained on the same image sensor. This approach leads us to propose a new approach for simplifying calibration by using the properties of the variable homography. Here, the second part addresses new principles for obtaining disparity images without any matching. A fast algorithm using a contour propagation algorithm is proposed without requiring structured or random pattern projection. These principles are proposed in a framework of quality control by vision, for inspection in natural illumination. By preserving scene photometry, some other standard controls, as for example calipers, shape recognition, or barcode reading, can be done conjointly with 3-D measurements. Approaches presented here are evaluated. First, we show that rapid calibration is relevant for devices mounted with multiple lenses. Second, synthetic and real experimentations validate our method for computing depth images.
In previous works, we have extended the principles of “variable homography”, defined by Zhang and Greenspan, for measuring height of emergent fibers on glass and non-woven fabrics. This method has been defined for working with fabric samples progressing on a conveyor belt. Triggered acquisition of two successive images was needed to perform the 3D measurement. In this work, we have retained advantages of homography variable for measurements along Z axis, but we have reduced acquisitions number to a single one, by developing an acquisition device characterized by 4 lenses placed in front of a single image sensor. The idea is then to obtain four projected sub-images on a single CCD sensor. The device becomes a plenoptic or light field camera, capturing multiple views on the same image sensor. We have adapted the variable homography formulation for this device and we propose a new formulation to calculate a depth with plenoptic cameras. With these results, we have transformed our plenoptic camera in a depth camera and first results given are very promising.
During foreign operations, Improvised Explosive Devices (IEDs) are one of major threats that soldiers may
unfortunately encounter along itineraries. Based on a vehicle-mounted camera, we propose an original approach
by image comparison to detect signicant changes on these roads. The classic 2D-image registration techniques
do not take into account parallax phenomena. The consequence is that the misregistration errors could be
detected as changes. According to stereovision principles, our automatic method compares intensity proles along
corresponding epipolar lines by extrema matching. An adaptive space warping compensates scale dierence in
3D-scene. When the signals are matched, the signal dierence highlights changes which are marked in current
video.
Fabric's smoothness is a key factor in determining the quality of finished textile products and has great influence on the functionality of industrial textiles and high-end textile products. With popularization of the zero defect industrial concept, identifying and measuring defective material in the early stage of production is of great interest to the industry. In the current market, many systems are able to achieve automatic monitoring and control of fabric, paper, and nonwoven material during the entire production process, however online measurement of hairiness is still an open topic and highly desirable for industrial applications. We propose a computer vision approach to compute epipole by using variable homography, which can be used to measure emergent fiber length on textile fabrics. The main challenges addressed in this paper are the application of variable homography on textile monitoring and measurement, as well as the accuracy of the estimated calculation. We propose that a fibrous structure can be considered as a two-layer structure, and then we show how variable homography combined with epipolar geometry can estimate the length of the fiber defects. Simulations are carried out to show the effectiveness of this method. The true length of selected fibers is measured precisely using a digital optical microscope, and then the same fibers are tested by our method. Our experimental results suggest that smoothness monitored by variable homography is an accurate and robust method of quality control for important industrial fabrics.
A fabric's smoothness is a key factor to determine the quality of textile finished products and has great influence on the
functionality of industrial textiles and high-end textile products. With popularization of the 'zero defect' industrial
concept, identifying and measuring defective material in the early stage of production is of great interest for the industry.
In the current market, many systems are able to achieve automatic monitoring and control of fabric, paper, and
nonwoven material during the entire production process, however online measurement of hairiness is still an open topic
and highly desirable for industrial applications.
In this paper we propose a computer vision approach, based on variable homography, which can be used to measure the
emergent fiber's length on textile fabrics. The main challenges addressed in this paper are the application of variable
homography to textile monitoring and measurement, as well as the accuracy of the estimated calculation. We propose
that a fibrous structure can be considered as a two-layer structure and then show how variable homography can estimate
the length of the fiber defects. Simulations are carried out to show the effectiveness of this method to measure the
emergent fiber's length. The true lengths of selected fibers are measured precisely using a digital optical microscope, and
then the same fibers are tested by our method. Our experimental results suggest that smoothness monitored by variable
homography is an accurate and robust method for quality control of important industrially fabrics.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.