|
1.INTRODUCTIONDepth sensing technology has been intensively researched for applications in industry, medicine, computer science, entertainment, and other sectors. Structured light illumination (SLI) technology is a common approach among various methods for determining the depth of the object’s surface. In an SLI system, a projector projects pre-encoded fringe patterns onto the object, the camera synchronously captures the deformed pattern, and the phase is calculated from those patterns. After calibration, the depth of the surface can be calculated according to the principle of triangulation [1]. In this paper, a simple and practical square-wave phase-shift encoding strategy and the corresponding decoding method are proposed. Compared with the traditional sinusoidal phase-shift encoding strategy, the proposed method is immune to the effects of gamma distortion. Moreover, the invalid phase is removed by the epipolar constraints. Experiments are conducted to validate the proposed method. The proposed approach produces a more accurate point cloud than traditional SLI according to the results. 2.RELATED WORKThe projection of a single pattern or a set of patterns onto the surface of the scanned object, which is then synchronously captured by a single camera or a series of cameras, is the basic process of a structured light system. The patterns have been generated in such a way that codewords are assigned to a group of pixels. There is a direct mapping from the codewords to the corresponding coordinates of the pixel in the pattern since each coded pixel has its own codeword [2]. Timemultiplexing, spatial neighborhood codification, and direct codification are the three encoding strategies used in projection techniques [2]. The spatial neighborhood codification tends to employ a single pattern for reconstruction. The codeword that labels a point of the pattern is obtained from a neighborhood of the points around it. In some spatial encoding strategy, such as pseudo-noise sequence [3], M-array [4], color encoding [5], grid encoding [6] et al, a single pattern is generated and projected, whose advantage is that it requires less time for projection and is much easier to achieve real-time 3D reconstruction. However, the decoding stage may become difficult because the spatial neighborhood cannot always be identified, which causes reconstruction errors. The temporal encoding strategy is one of the most often employed methods in structured light systems. A series of patterns are projected onto the surface of the scanned object in this case. This encoding scheme is capable of achieving higher measurement precision than the spatial neighborhood encoding strategy. In these encoding strategies, the most common schemes include phase shift [7], grayscale [8], and so on. Because multiple patterns are needed to project, it is not suitable for dynamic scenes. 3.PROPOSED METHODBased on the triangulation between the camera and the projector, SLI is a widely used, high-precision, and active measurement method. Figure 1 shows the basic process of SLI. In this paper, we propose a square-wave encoding and decoding scheme. Moreover, with the epipolar constants, the invalid phase can be removed and the accuracy of the point cloud is improved. 3.1Classical structured light illuminationThe classical phase-shift SLI usually adopts the encoding scheme in which the intensity changes sinusoidally along the horizontal or vertical direction. The coding pattern can be described by the following equation where where With phase unwrapping operation, the absolute phase can be obtained. However, the classical sinusoidal encoding scheme is sensitive to gamma nonlinearity, noise, and multiple patterns are required to be projected, which takes a much longer time. Liu et al. [9] deduced the gamma distortion model in detail, and quantitatively described the relationship between gamma and phase error. The gamma distortion model is based on a one-parameter gamma function, which is described as where where Gm is As can be seen from the above equation, the phase error is related to γ. Therefore, the classical sinusoidal encoding scheme suffers from gamma distortion. In the next section, a gamma-free encoding scheme is proposed. 3.2The proposed bidirectional pattern encoding schemeIn this section, a square-wave encoding strategy for bidirectional scanning SLI is proposed for 3D reconstruction, which overcomes the shortcomings of the traditional sinusoidal encoding scheme. The projected pattern along the vertical direction can be described as where Similarly, the projected pattern along the horizontal direction is expressed as where W is the width of projector resolution. Figure 2 illustrates the difference between the square-wave encoding scheme and the sinusoidal encoding strategy. As we can see, the square-wave pattern only needs two levels of intensity. Figure 2.The comparison of the sinusoidal coding scheme and the square wave coding scheme. ![]() After projecting the square wave pattern, the pattern captured by the camera can be described as where Correspondingly, the phase distorted by gamma nonlinearity is derived as With equation (11), the equation (12) can be simplified to As we can see from the above equation, the phase ϕ has nothing to do with the gamma distortion coefficient γ. In conclusion, it is not affected by gamma distortion. Then, the original range of the phase ϕ, which is obtained with the equation (13), is[–π, π]. For convenience, we map its range to[0,2π]. In Figure 3, we visualize one column of the wrapped phase. In phase-shift structured light illumination, high frequency is more effective for suppressing noise, nonlinearity, and other errors, so the high-frequency phase is generally used to reconstruct 3D point clouds. The phase ϕ calculated by equation (3) needs to be unwrapped according to its fringe order. For the wrapped phase above, its fringe order can be obtained by the following equation where ϕl is the phase of base frequency, round(•) is the numeric rounding operation, and k is the fringe order. For multi-frequency phase-shift structured light technology, when the base frequency is used to unwrap the phase of a high frequency, then the result is used as the new base frequency phase to unwrap the phase of a higher frequency until the highest frequency is obtained. After the fringe order is computed, the absolute phase can be calculated by the following equation We obtain the absolute phase at a sub-pixel level by fitting. The absolute phase is shown in Figure 4. 3.3Removal of invalid phaseThe calibration matrices of the camera and projector are expressed as To obtain a more accurate point cloud, we remove the invalid phase resulting from several factors. In the Liu’s research [10], the epipolar line is derived into a much simpler description: where where the where The The description of epipolar line in equation (17) does not require a fundamental matrix between the camera and the projector and can be transformed into where A, B and C are According to the epipolar constraint, the point (xc, yc) corresponding to the point (xp, yp) on the projection plane should be located on the epipolar line, that is, in an ideal situation, the distance from the point to the epipolar line should be 0. The distance from the point to the epipolar line is When errors are caused by various reasons such as noise, lens distortion, and image saturation in the imaging process, the above relationship is not satisfied. When the distance between the phase point and the corresponding epipolar line exceeds the threshold value, it indicates that the point is too impacted by external and internal factors to reconstruct an accurate point cloud. As a result, the region including these points should be excluded. The method is simple and practical. Moreover, it also works for the classical sinusoidal encoding SLI system. To obtain the point cloud, we need to calibrate the structured light system to get the projector and camera parameters. There are various calibration techniques for calibrating the structured light system. In this paper, we adopt the least-square solution method, which is easier to implement and is close to other methods in accuracy [11]. With the calibration matrices of the camera and projector, the point cloud of the scanned surface is computed by where C and D are 4.EXPERIMENTAL RESULTSTo validate the effectiveness of the proposed encoding scheme, we conducted two experiments in this section. In the first experiment, the invalid phase area is determined and labelled when scanning a sculpture. In the second experiment, a vertical plane is scanned, which consists of black and white grids to simulate the high contrast region. The invalid phase area is determined and removed. Then, the measurement errors of the plane are calculated for the proposed method and the classical method. The structured light system consists of a Casio XJ-M140 projector, an AVT Prosilica GC650C camera, and a computer. The resolutions of the projector and camera are 800×600 and 640×480 respectively, and the camera works in grayscale mode. Before conducting the experiment, we first calibrate the projector and camera as described to obtain the calibration parameter matrices of the camera and projector. The system suffers from the gamma distortion and we don’t remove it. First, we generate a set of square wave patterns for projection, the frequencies are 1, 8, 64, 128, and the total number of phase shifts for each group is 8. We first scan the sculpture and compute the phase (xp, yp) according to the captured patterns. The result is shown in Figures 5 and 6. Then, we compute the In the second experiment, in order to further quantify the accuracy improvement of the proposed method for 3D reconstruction, a vertical plane is reconstructed with the proposed and classical methods. Moreover, the reconstruction errors are shown in Table 1. Figure 8 demonstrates a vertical plane with a checkerboard image in the central part. The black and white area of the plane simulates different adjacent high contrast regions. The proposed method can effectively detect the regions with large phase errors in the phase-shifted structured light. Figure 9 shows the reconstruction error of the vertical plane after removing the invalid point area by the proposed method. Meanwhile, we calculate the mean reconstruction error, maximum reconstruction error and RMS of the reconstruction error with the two methods, and the results are shown in Table 1. Table 1.The error comparison.
From Table 1, we can see that the proposed method can effectively remove the points with large errors in the 3D reconstruction, and then improve the accuracy of the reconstruction in the gamma distorted systems. Moreover, the proposed method is simple and practical without complicated calculations and additional hardware. With the proposed method of removing invalid phase area, we can easily locate the area and reconstructs much more accurate point clouds. 5.CONCLUSIONA square wave encoding strategy, which is gamma-free, is proposed for an SLI system. A framework including projection pattern design, practical decoding strategy, and removal of invalid phase is developed. Besides, gamma analysis for the square wave is also presented, which proves that it is not affected by gamma distortion. Moreover, a simple and practical method, which can remove the invalid phase and also works for the classical sinusoidal encoding SLI system, is proposed. Compared with the sinusoidal phase-shifting method, the proposed method has a much better performance in the low SNR scenes and is easy to implement. With the proposed method, we can obtain a more accurate point cloud. ACKNOWLEDGMENTSThis work was supported by Science and Technology Project of State Grid Sichuan Electric of Power Company (5219622000GW). REFERENCESGorthi, S. S. and Rastogi, P.,
“Fringe projection techniques: Whither we are?,”
Opt. Lasers Eng, 48
(2), 133
–140
(2010). https://doi.org/10.1016/j.optlaseng.2009.09.001 Google Scholar
Salvi, J., Pagès, J. and Batlle, J.,
“Pattern codification strategies in structured light systems,”
Pattern Recogn, 37
(4), 827
–849
(2004). https://doi.org/10.1016/j.patcog.2003.10.002 Google Scholar
Sachs, J., Peyerl, P., Wöckel, S., Kmec, M., Herrmann, R. and Zetik, R.,
“Liquid and moisture sensing by ultra-wideband pseudo-noise sequence signals,”
Meas Sci Technol. Papers, 18
(4), 1074
(2007). https://doi.org/10.1088/0957-0233/18/4/016 Google Scholar
Yang, L., Bengtson, K. R., Li, L. and Allebach, J., P.,
“Design and decoding of an M-array pattern for low-cost structured light 3D reconstruction systems,”
in Proc. 2013 IEEE International Conference on Image Processing,
2168
–2172
(2013). Google Scholar
Chen, X. C., Lu, C., Ma, M. C., Mao, X. D. and Mei, T.,
“Color-coding and phase-shift method for absolute phase measurement,”
Opt. Commun., 298-299, 54
–58
(2013). https://doi.org/10.1016/j.optcom.2013.02.013 Google Scholar
Buendía, M., Salvador, R., Cibrian, R., Laguia, M. and Sotoca, J. M.,
“Determination of the object surface function by structured light: application to the study of spinal deformities,”
Phys. Med. Biol, 44
(1), 75
–86
(1999). https://doi.org/10.1088/0031-9155/44/1/007 Google Scholar
Li, E. B., Peng, X., Xi, J., Chicharo, J. F., Yao, J. Q. and Zhang, D. W.,
“Multi-frequency and multiple phase-shift sinusoidal fringe projection for 3D profilometry,”
Opt. Express, 13
(5), 1561
–1569
(2005). https://doi.org/10.1364/OPEX.13.001561 Google Scholar
Song, Z.,
“Absolute phase retrieval methods for digital fringe projection profilometry: A review,”
Opt. Lasers Eng, 107
(1), 28
–37
(2018). Google Scholar
Liu, K., Wang, Y. C., Lau, D. L., Hao, Q. and Hassebrook, L. G.,
“Gamma model and its analysis for phase measuring profilometry,”
J. Opt. Soc. Am. A, 27
(3), 553
–562
(2010). https://doi.org/10.1364/JOSAA.27.000553 Google Scholar
Liu, K., Zhang K. K., Wei, J. H., Song, J. W., Lau, D. L., Zhu, C. and Xu, B.,
“Extending epipolar geometry for real-time structured light illumination,”
Opt. Lett, 45
(12), 3280
–3283
(2020). https://doi.org/10.1364/OL.390212 Google Scholar
Feng, S. J., Zuo, C., Zhang, L., Tao, T. Y., Hu, Y., Yin, W., Qian, J. M. and Chen, Q.,
“Calibration of fringe projection profilometry: A comparative review,”
Opt. Lasers Eng, 143
(1), 106622
(2021). https://doi.org/10.1016/j.optlaseng.2021.106622 Google Scholar
|