|
1.IntroductionThree-dimensional shape sensing plays an important role in machine vision, reversed engineering, automatic manufacturing, and other industrial applications. The use of a full-field technique, such as stereovision,1, 2 fringe projection,3, 4 and structured light illumination,5, 6 has been recognized as a promising method for the measurement of a surface profile. One of the major studies for 3-D sensing is the measurement of dynamic objects. For objects with fast-moving speeds, 3-D shape measurements always require that the observed images not be blurred by motion. High-speed cameras or stroboscopic illuminations are commonly used to obtain unblurred images. Unfortunately, when the speed of the objects still exceeds the temporal resolution of the sensor, the image is blurred. Of course, an ultra-short laser pulse can still be used to freeze the motion on the image. However, the illumination intensity might not be sufficient to perform large-scale measurements, and the cost of such light sources is generally high. In this paper, we show that the projected fringe profilometry4 does not need to avoid the blurred images. As a typical setup, we use a fringe pattern to illuminate the dynamic object and utilize a CCD camera to record the fringe distribution. Fringes on the obtained image are deformed by the topography of the object and also blurred by motion. Theoretical analysis shows that objects moving within one period of the projected fringe can be directly described by the projected fringe profilometry. Thus, the cost of the detection system is effectively reduced. 2.Theoretical AnalysisFigure 1 shows the system configuration. The plane is located in the figure plane, and the axis is normal to the figure plane. A fringe pattern is projected onto the inspected surface. Intensity of the fringes when propagating in space is represented as where is the background or dc intensity level, is the fringe contrast, and and are periods of the fringes in and axis, respectively. The depth value on a surface point is measured relative to the plane indicated in the figure. Thus, the reflected intensity on the surface is expressed aswhere is the reflectivity of the measured object.The projected fringes on the surface are observed by the image sensor array. The detection plane coordinate system is defined in the CCD detection plane with and axes parallel to the row and the column directions of the sensor array, respectively. The gray level on the recorded image corresponding to is described as where is the background or dc gray level, is the modulation amplitude, and is the measured absolute phase. For a telecentric system, the mapping transformation between the image plane and plane iswhere is the magnitude of the telecentric lens. A phase value sampled at an object point is assumed equal to that sampled at its image point. This assumption applies when the point spread function of the system is symmetric (coma-free). Thus, Eq. 3 can then be rewritten aswhere the constant identifies the linear relationship between the reflected light intensity and the image gray level .can be extracted with the phase-shifting technique or Fourier transform method. Since both phase evaluation techniques involve the arctangent operation, the extracted phases have discontinuities with phase jumps. Unwrapping is inevitable to recover the absolute phases.7 Once the unwrapped phase is obtained, depth on the surface point can be directly found from Eq. 5, as given by Now, consider this inspected object moving with speed in the world coordinates. Its depth profile is a function of time and is given by where is the object depth function at . and are unit vectors. ∇ is a 2-D gradient operator, which is denoted as .The image sensor array obtains a blurred image within the exposure time . The gray level of the blurred image with reference to Eq. 5 can be expressed as where is the measured phase from the blurred fringes, is the measured background or dc gray level, and is the modulation amplitude.For objects in which varies slowly with and , Eq. 8 can be represented as Substituting Eq. 7 into Eq. 9, the intensity of the blurred image is then simplified aswhere , and .According to Eq. 7, the depth profile at can be expressed as and therefore Eq. 10 is represented asComparing Eq. 8 with Eq. 12, it is found that can be fully identified from the blurred fringes, as given byThus, the 3-D shape of the dynamic object at can be directly retrieved from the blurred fringe. If the exposure time is close to zero, Eq. 13 becomes close to Eq. 5, and the fringes are not blurred.3.ExperimentsA ball with moving speed , , and was chosen as the dynamic sample. Its diameter was approximately . A sinusoidal fringe pattern was illuminated by a halogen lamp and then projected onto this dynamic sample. A CCD camera with at pixel resolution was used to record the fringe distribution. Fringes were blurred by linear motion. Figure 2 shows the fringe distribution, in which the exposure time was . Fig. 2(a) Fringes on the inspected object observed by a CCD camera when the object was shifting along a specific direction. (b) Phase distribution on the dynamic object. A gray-level bar is used to address the phase values. ![]() Phase-extraction was performed with the Fourier transform method.3 Figure 2 shows the computed phase , which was within the interval between and . Unwrapping was a necessary procedure to eliminate the discontinuities. In our experiment, we use Goldstein’s algorithm7 to restore the absolute phases. With Eq. 13, depth profile was determined. Figure 3 shows the retrieved profile. Its 1-D profile is shown in Fig. 3. Fig. 3(a) Retrieved profile for the inspected object. A gray-level bar is used to address the depth values. (b) One-dimensional surface plot of (a). ![]() A comparison when the sample was static was performed as well. Appearance of the projected fringes on the static sample is shown as Fig. 4. Equation 6 was employed to retrieve the 3-D shape. Figures 4 and 4 show the retrieved 3-D shape and its 1-D profile, respectively. Systematic accuracy for a static object was approximately . The errors were mainly from the spatially sampling density of the CCD camera and phase-extraction. The sampling resolution was approximately , which was determined by the field of view and the pixel numbers of the CCD camera. Fig. 4(a) Appearance of the fringe distribution when the object was static. (b) Retrieved profile for the inspected object. (c) One-dimensional surface plot of (b). ![]() The difference between one two profiles (the dynamic case and the static case) is depicted in Fig. 5, in which the shifting displacement has been compensated. It was found that accuracy at the center area of the sample could be achieved with the same order of the static case, implying that our theoretical analysis was correct. However, enormous errors occurred at the edge of the dynamic object. Fig. 5Difference between the profiles in Figs. 3 and 4. Shifting displacement between the two profiles has been compensated. ![]() There were two sources causing such errors. They were (1) variation of effective exposure time at the boundary area, and (2) ambiguity of phase-extraction for surfaces with large depth variation. The exposure time for image pixels that inspected the boundary area was unfortunately not a constant. It was highly dependent on the moving direction and the shape of the boundary. For example, a surface point on the boundary is observed by the image sensor array. Since this object is dynamic, the observed point is moving from point to point on the detection plane during the exposure time . As shown in Fig. 6, for a sensor pixel that is located within the interval between and , its effective exposure time is . The effective exposure time on the boundary area was therefore not . Equation 9 was not applicable for the boundary area. The example shown in Fig. 6 also indicates that the effective exposure time was corresponding to the shape of the boundary on the detection plane. Effective exposure time for pixel in Fig. 6 is different from that for pixel in Fig. 6. Fig. 6Examples of the observed boundary on the detection plane: (a) the circular boundary, and (b) the rectangular boundary. ![]() Errors from ambiguity of phase-extraction occur when the shifting amount of the projected fringes is larger than their periods. A displacement of the dynamic object directly causes the projected fringes to shift from one surface point to another. If the shifting amount is equal to the fringe period, the fringe contrast becomes zero. This phenomenon can be mathematically described when the sinc function in Eq. 10 is equal to zero, or say, is equal to . In such a situation, phase cannot be identified on that area. Moreover, aliasing occur when the shifting amount is larger than the fringe period, i.e., . This directly causes a phase offset when performing the phase unwrapping. Equation 10 for the aliasing area should be modified as and Eq. 13 is replaced asSince aliasing occurs with distribution of the aliasing area is varied with the moving direction, speed of motion, slope of the profile, and exposure time. We performed several measurements by changing parameters such as the moving direction, the moving speed, and the exposure time. An example is illustrated in Fig. 7. Figures 7 and 7 are appearances of the inspected sample moving along the and axis, respectively. The moving speed was , and the exposure time of the CCD camera was . The phase extracted by the Fourier transform method is depicted as Fig. 8. Aliasing areas are enclosed with dotted lines. Ideally, the phase offset on the aliasing area could be compensated by adding or reducing a phase value. Unfortunately, the signal-to-noise ratio (SNR) was too low around the zero fringe-contrast area, and therefore phase information was lost. This directly made performing the phase extraction uncertain when using the Fourier transform method. The phase distribution became continuous on the boundary between the aliasing area and the unaliasing area. If the phase on the aliasing area was compensated by adding or reducing a phase value, discontinuity with a phase jump appeared on the zero fringe-contrast area. Thus, it is impractical to robotically recover phases on the aliasing area by simply adding or reducing a phase value.Fig. 7Appearance of the fringe distribution when the object was shifting along (a) the axis and (b) the axis. ![]() Fig. 8Phase map computed by the Fourier transform method for the object shifting along (a) the axis and (b) the axis. Aliasing areas are enclosed with dotted lines. ![]() Figure 9 shows the recorded blurred image when the sample was moving along the axis. The moving speed of the sample was , and the exposure time was . The retrieved phase is shown in Fig. 9. Since and were zero, the value was only a function of . Fringe contrast over the whole image was therefore varied only with , not with or . Phase extraction did not encounter any ambiguity. Sources of errors corresponding to various moving directions are reported in Table 1. Fig. 9(a) Appearance of the fringe distribution when the object was shifting along the axis. (b) Phase distribution on the dynamic object. ![]() Table 1Sources of errors caused by the moving direction.
Systematic accuracy for a dynamic object can be illustrated as Fig. 10, in which a plate moving along the axis was inspected. The roughness of this plate was approximately . The moving speed was , and the exposure time was . A comparison was evaluated when this plate was static, as shown in Fig. 10. The retrieved 3-D shape for the dynamic case and the static case is depicted as Figs. 11 and 11, respectively. Even though the fringe contrast on the dynamic object was relatively low, its profile can be retrieved with accuracy as high as the static one. Fig. 10Appearances of the fringe distributions when (a) the plate was shifting along the axis and (b) the plate was static. ![]() Compared with other methods using deblurred algorithms to restore the observed information, the proposed method saves the computation time. For approaches using a high-speed camera or stroboscopic illuminations to freeze the object’s motion, the cost of the proposed system is relatively low. However, limitations are that the inspected object should be a rigid body, and this object should move linearly within one period of the projected fringes. If the displacement of the projected fringes shifts larger than one period of the fringes, aliasing will occur. In addition, errors also occur when the inspected object is rotating. Equation 9 is not applicable when the moving vector is time-dependent. 4.ConclusionsWe have presented a discussion on how to retrieve the 3-D shape from an image blurred by motion. With the fringe projection method, objects moving within one period of the projected fringes can be fully described. Thus, it is not necessary to avoid blurred images. Accuracy can be achieved that is as high as with the static image. This makes it desirable to reduce the cost of the detection system. We believe that applications to microelectromechanical systems (MEMS) and biomedical inspections can be realized. ReferencesN. A. Thacker and J. E. W. Mayhew,
“Optimal combination of stereo camera calibration from arbitrary stereo images,”
Image Vis. Comput., 9 27
–32
(1990). https://doi.org/10.1016/0262-8856(91)90045-Q 0262-8856 Google Scholar
R. A. Lane, N. A. Thacker, and N. L. Seed,
“Stretch-correlation as a real-time alternative to feature-based stereo matching algorithms,”
Image Vis. Comput., 12 203
–212
(1994). https://doi.org/10.1016/0262-8856(94)90074-4 0262-8856 Google Scholar
M. Takeda and K. Mutoh,
“Fourier transform profilometry for the automatic measurement of 3-D shaped object,”
Appl. Opt., 22 3977
–3982
(1983). https://doi.org/10.1364/AO.22.003977 0003-6935 Google Scholar
V. Srinivasan, H. C. Liu, and M. Halioua,
“Automated phase-measuring profilometry of 3-D diffuse objects,”
Appl. Opt., 23 3105
–3108
(1984). https://doi.org/10.1364/AO.23.003105 0003-6935 Google Scholar
W. H. Su,
“Color-encoded fringe projection for 3D shape measurements,”
Opt. Express, 15 13167
–13181
(2007). https://doi.org/10.1364/OE.15.013167 1094-4087 Google Scholar
W. H. Su,
“Projected fringe profilometry using the area-encoded algorithm for spatially isolated and dynamic objects,”
Opt. Express, 16 2590
–2596
(2008). https://doi.org/10.1364/OE.16.002590 1094-4087 Google Scholar
E. Zappa and G. Busca,
“Comparison of eight unwrapping algorithms applied to Fourier-transform profilometry,”
Opt. Lasers Eng., 46 106
–116
(2008). https://doi.org/10.1016/j.optlaseng.2007.09.002 0143-8166 Google Scholar
Biography
|