|
1.INTRODUCTION AND MOTIVATIONThree-dimensional imaging of assembled printed circuit boards is the current state of the art with modern systems for inline optical inspection of surface mounted devices (SMD). However, the quality control of electronics made with through-hole technology (THT) is still mostly done with two-dimensional inspection systems. In contrast to SMD, the bulkier packages of THT components have a higher positioning tolerance and are soldered from the other side of the printed circuit board. The inspection of THT PCBs typically consists of a presence check for the right package and a polarity check of said components. A two-dimensional inspection system using an elaborated illumination system can detect misplaced or missing parts very reliable, and therefore, the use of three-dimensional inspection is not necessary for the component side. For the solder side of the THT-PCB, the inspection focuses on the solder joints. Depending on the desired quality, there are a lot of possible defects in the soldering connection to observe. A faulty connection always results in the board being marked as fail, but classifying the type of error helps to qualify and improve the production process. Examples for such soldering defects could be blowouts, unwetted pads, insufficient solder volume, and unsuitable wetting angles. Two-dimensional inspection can detect some but not all soldering defects reliably, so the need for fast and precise three-dimensional solder joint imaging arises. The typical measurement system used for SMD inspection struggle with the highly reflective and sloped surfaces of unoxidized THT solder joints. Mainly structured light based methods produce unsatisfying results when applied to fresh solder connections. Such methods rely on a more diffuse reflection of light and bright contrast of light and dark areas. A new method for three-dimensional imaging of THT solder joints to be used for fast and reliable inline inspection is proposed. Section 2 will lay out the current state of the art for related imaging technologies and the quality control of THT solder joints. Following this, section 3 describes the proposed approach and demonstrate its capability. Finally, section 4 gives a conclusion of the research and provides examples for future work on this topic. 2.STATE OF THE ARTThere is a wide variety of light-based 3d reconstruction methods used and fine-tuned for automatic optical inspection. Laser-triangulation, phase-shift profilometry based on structured light and shape from shading are amongst the most common ones, more exotic approaches include binocular stereo, chromatic confocal systems, or neural network based reconstructions using two-dimensional images. Subsection 2.1 shows current inspection technologies, and subsection 2.2 details the specific problems encountered when inspecting THT solder joints. 2.1Three-dimensional Optical InspectionLaser triangulation1,2 typically uses a line laser projection and triangulation to assign a height value to every observed pixel. The usage of line laser enables the system to cover one lateral dimension of the PCB with one observation. However, when inspecting only a portion of the PCB, most of the data is unused in the following classification. The main drawback of this approach is the overall slow inspection speed when using high lateral resolutions, and the observed interferences on shiny surfaces. Phase-shift profilometry1,3,4 seems to be the most widely spread method in the field of SMD assembled PCBs and solder-paste inspection. All well known AOI suppliers have at least one such system in their portfolio. An angled light source brings structured light onto the surface of the PCB. The geometry of the components and pins phase-modulates the structured light pattern in the observation. Based on this phase-modulation, the height of the object can be calculated. Like laser triangulation, this method reaches its limits with highly reflective surfaces. Binocular stereo, as described in [Luhmann],1 uses a calibrated pair of cameras to observe the object. The simultaneous observations then get processed into a disparity map, which can be triangulated into a depth map of the object using the calibrated poses of the cameras. The calculation of the disparity map relies on robust optical features, which can be identified in both observations. Such features are not present on unoxidized and high reflective solder joints. With the lack of features, binocular stereo and similar approaches are useless to inspect solder joints of THT components. Chromatic confocal microscopy5 uses stacked layers of focal planes to reconstruct the three dimensional surface of the inspected object. However, in contrast to z-axis-scanning methods, this approach uses color coding through intentional chromatic aberration. Although its faster than laser confocal microscopy, it lacks the speed and measurement depth of other techniques. Shape from shading6–10 is a group of methods using a constrained illumination. The known position of the observing camera and illuminations is used to calculate the surface profile from shadows and reflections of the inspected object. [Kim et al.]10 used three rings of LEDs to illuminate the surface of SMD solder joints. They extract two-dimensional features from the images taken of each illumination ring. If the classification based on the extracted features results in an ambiguous decision, additional three-dimensional features are calculated. This process is fine-tuned for SMD components and does not aim to give an overall three-dimensional visualization of the solder joints. [Loh, Lu]11 uses a sophisticated illumination setup to extract two-dimensional features of solder joints in an SMD process. Their main focus was to determine if the slant angle of the solder forms a convex or concave surface. As discussed in subsection 2.2, this cannot be used to discriminate good and bad solder joints in a THT-process. [Kong]8 proposed a new model for shape from shading specialized on solder joints. The influence of highly specular reflections on the state of the art approaches is reduced to give a realistic impression of the inspected solder joints. [Pang, Chu]7 uses an angled illumination setup to inspect dispensed solder paste. The segments of the illumination use different colors to extract the surface slant of the solder paste from a two-dimensional image. Different features are proposed to use the information from this approach. [Vitoriano, Amaral]6 described a method to estimate heightmaps of SMD solder joints based on a shape from shading approach. Similarly to the other approaches, rings of LEDs are used to illuminate the solder joint. Three rings with three different color are used to capture the slant of the solder in one two-dimensional image. By using assumptions about the geometry of the solder joints, the height is summed up from the pad to the component. This approach only works in one lateral direction and ignores tin metal wetting on the sides of the component. 2.2Optical Inspection of THT Solder JointsThe IPC standard “Acceptability of Electronic Assemblies”12 is a worldwide accepted standard for quality inspection of manufactured electronics. Examples for solder joint anomalies described in the document are excess solder, missing solder, holes or cracks in the solder connection and deformations resulting from thermal processes. Depending on the product class, these anomalies can lead to a fail classification of the connection. The IPC standard12 describes a good solder connection as an even shiny appearance with a concave surface and an angle to the solder pad of fewer than 90 degrees. A dull surface is only acceptable if the soldering process is unable to avoid them, like slow heat dissipation or lead-free solder. Specifically for THT solder joint inspection on the bottom side, the standard describes different quality metrics. This approach focuses on unbend pins, since they are the most common in automatic soldering processes, while bend pins are common with manual soldered connections. A quality through-hole solder connection should have a visible wetting of tin on the whole pad. The tin has to distribute around the pin evenly. Figure 1 illustrates an optimal concave through-hole solder joint. When inspecting THT solder joints using automatic optical inspection systems, two criterions are essential. The first is the length of the pin, and the second is the quality of the electrical and mechanical connection of the pin and pad. The amount of tin contacting the pin cannot be measured or estimated using pure optical inspection. An x-ray inspection system can be used to measure the overall volume of the tin and check for voids in the hole. [Fonseka, Jayasinghe]13 demonstrated a comprehensive algorithm to inspect THT solder joints in a two-dimensional color image. The pin height can give inside whether the components were placed correctly into the holes. Comparing pin heights with each other can be used to check components for coplanarity. Some manufacturers even want to inspect the correct trimming of the pins, since too long pins can contact the case of the finished product and lead to malfunctions. 3.PROPOSED APPROACHThis section proposes a method to estimate three-dimensional reconstructions of THT solder connections for automatic optical inspection. The reconstruction uses a multitude of two-dimensional orthogonal observations with a sequence of specialized illuminations. The algorithm used in the process leverages the radial symmetry of the solder connection, resulting in a fast and specialized new method. Subsection 3.1 demonstrates the illumination setup, while subsections 3.2, 3.3, and 3.4 detail the proposed algorithm. Subsection 3.2 details how the regions of pins are extracted and preprocessed. Subsection 3.3 shows how the preprocessed images are used to estimate the slant of the solder surface. Finally, subsection 3.4 explains how the surface slant is converted into a heightmap of the solder joint. 3.1IlluminationFigure 1 shows the illumination setup used in this method, consisting of four concentric rings of LEDs and one coaxial light source for the camera. The rings have individual distances to the camera. In this way, the overall appearance resembles a parabolic shape. The purpose of this setup is to illuminate the scene with discrete elevation angles. Every possible angle of the setup is observed in sequence by the camera. Since this method evaluates the reflections of the lights in shiny surfaces, the scene illumination does not have to be even. 3.2Extraction and Transformation of Pixel DataThe first step of the reconstruction is the extraction of the relevant pixel data. Only the pixels showing the pin and solder joint are useful for the reconstruction. When ignoring bend pins, the only possible location of the pin going through the PCB is the location of the drilled hole. Similarly, the only possible location of soldered tin metal is the region of the pad. Both pad and hole, are defined through the CAD data for the inspected PCB. A window with the size of at least the diameter of the pad is used to extract the pixel data around the drilled hole. Figure 2 shows such extracted regions for all the illuminations described in subsection 3.1. After extracting the regions, the algorithm transforms the images into a polar coordinate system using the rotational symmetry of the solder joints. By transforming the images, the surface angles of the sloped surface are all aligned along the same axis. This transformation breaks the following radial operations down into simpler operations along one image dimension. The transformation also affects the way the light sources are modeled in the following steps. The observed rings of LEDs now become axis-parallel strips of light sources. Since this method evaluates specular reflections, the light source model can be further simplified to use a single point light source along each polar angle. 3.3Estimation of Surface AnglesThe estimation algorithm relies on an illumination model capable of producing predictions of the observed scene depending on the surface angle, illumination parameters, and the observer position. For the experiments, the Phong illumination model14 was chosen, since it is robust and straightforward in its predictions. For each illumination angle in the list of observations, the model makes a prediction based on an assumed surface angle. The summed distance of the predictions and the observations overall illumination has to be minimized to find the surface angle matching the observation and prediction closely. This operation, as shown in the equation 1, is done for each pixel of the transformed images. α is the assumed surface angle used to minimize the distance of the observations Cn and the predictions over all N images. As the described operations run per pixel, the whole process can be sped up by using parallel processing. The result of this step is an image with an estimated surface angle per pixel in polar coordinates. The next step uses these angles to reconstruct a height map of the solder joint. 3.4Reconstruction of Height and RetransformationFor each pixel the tangent of the estimated surface angle tan(α) gives a height difference Δh. As it was previously assumed that all surfaces are angled away from the pin, the actual height of each pixel can be calculated by summing up all the Δh from the outer perimeter of the pad R towards the pin. This is shown in equation 2. After the calculation of the heights, the retransformation into the cartesian coordinate system gives a complete heightmap of the solder joint. The reconstructed heightmaps for the example solder joints are visualized in Figure 4. These heightmaps can be used to displace a mesh map for three-dimensional visualization, as shown in Subfigure 4c and 4d. 4.CONCLUSION AND FUTURE WORKA new method for three-dimensional imaging of THT solder joints was proposed. The new approach uses the specular reflection of a specialized illumination setup to reconstruct a heightmap of the inspected object based on estimated surface angles. The illumination setup described in this paper is problem-centered, yet cheap and straightforward to build in contrast to the other three-dimensional inspection systems. Because the proposed approach relies on specular reflections and not on diffuse reflections, it overcomes all the drawback of measuring highly reflective surfaces with other commonly used methods. REFERENCESLuhmann, T., Nahbereichsphotogrammetrie, Wichmann Verlag(2000). Google Scholar
Guerra, E., Manriquez, A., Schwartz, D., and Villalobos, J. R.,
“Three dimensional automated visual inspection of surface mounted devices,”
Computers and Industrial Engineering, 33
(1-2), 365
–368
(1997). https://doi.org/10.1016/S0360-8352(97)00113-7 Google Scholar
Lehmann, P. H., Bräuer-Burchardt, C., Osten, W., Breitbarth, A., Kühmstedt, P., Gastinger, K., Schmidt, I., Heinze, M., and Notni, G.,
“Fringe projection based high-speed 3d sensor for real-time measurements,”
Optical Measurement Systems for Industrial Inspection VII, Google Scholar
Osten, W., Kühmstedt, P., Gorecki, C., Munckelt, C., Heinze, M., Novak, E. L., Bräuer-Burchardt, C., and Notni, G.,
“3d shape measurement with phase correlation based fringe projection,”
Optical Measurement Systems for Industrial Inspection V, Google Scholar
Zint, M., Stock, K., Graser, R., Ertl, T., Brauer, E., Heyninck, J., Vanbiervliet, J., Dhondt, S., De Ceun-inck, P., and Hibst, R.,
“Development and verification of a novel device for dental intra-oral 3d scanning using chromatic confocal technology,”
Advanced Biomedical and Clinical Diagnostic and Surgical Guidance Systems, Xiii 9313
(2015). Google Scholar
Vitoriano, P. M. and Amaral, T. G.,
“3d solder joint reconstruction on smd based on 2d images,”
SMT Magazine, 31 83
–93
(2016). Google Scholar
Pang, G. K. and Ming-Hei., C.,
“Automated optical inspection of solder paste based on 2.5d visual images,”
in International Conference on Mechatronics and Automation,
https://doi.org/10.1109/ICMA.2009.5246351 Google Scholar
Kong, F.-h.,
“Reconstruction of solder joint surface based on hybrid shape from shading,”
in 2008 Fourth International Conference on Natural Computation,
593
–597 Google Scholar
H.-H., L. and M.-S., L,
“Printed circuit board inspection using image analysis,”
IEEE TRANSACTIONS ON INDUSTRY APPLICATIONS, 35
(2),
(1999). Google Scholar
Kim, T.-H., Cho, T.-H., Moon, Y.-S., and Park, S.-H.,
“Automatic inspection of solder joints using layered illumination,”
Google Scholar
P.M., N. and N.J., F,
“Using infrared sensors and the phong illumination model to measure distances,”
in International Conference on Robotics and Automation,
Google Scholar
IPC,
“Abnahmekriterium für elektronische baugruppen,”
Google Scholar
Fonseka, C. L. S. C. and Jayasinghe, J. A. K. S.,
“Implementation of an automatic optical inspection system for solder quality classification of tht solder joints,”
Ieee Transactions on Components Packaging and Manufacturing Technology, 9
(2), 353
–366
(2019). https://doi.org/10.1109/TCPMT.2018.2864666 Google Scholar
Phong, B. T.,
“Illumination for computer generated pictures,”
Communications of the ACM, 18
(6), 311
–317
(1975). https://doi.org/10.1145/360825.360839 Google Scholar
|