1.

Introduction

Laser range-gated imaging can penetrate a strong scattering medium and capture the image of an object at a certain distance by applying active laser illumination, changing the gate delay time, and varying the gate width. It has been widely used in areas such as military reconnaissance, monitoring, underwater detection, and search and rescue. For cases when the line of sight is blocked by corners of walls or other building obstructions within the environment of city streets and buildings (as shown in Fig. 1), traditional imaging techniques and equipment are unable to achieve imaging beyond the direct field of view. Non-line-of-sight imaging can bypass the corners of walls or other obstructions to achieve imaging beyond the direct field of view, which can be used to reduce casualties in modern urban street fighting, and to increase battlefield awareness and operational efficiency.

Fig. 1

Representation of the line of sight being blocked by the corner of a structure.

In 2006 and 2007, researchers at the Research Institute for Optronics and Pattern Recognition of Germany and Swedish Defence Research Agency (FOI)^1,2 of Sweden carried out joint research on short-wavelength infrared laser range-gated imaging for ground applications in monostatic and bistatic configurations. A number of test targets (test panels, vehicles, and soldiers with weapons) were included in the tests, and windows in buildings and cars, traffic signs, and vehicle surfaces were utilized as the intermediary reflective surfaces. In 2011, the FOI group³ proposed the loss factor model through further study of the effect of the characteristics (including the shape and the reflection characteristics) of the intermediary reflective surface on laser illuminance attenuation and measured the loss factors with different intermediary reflective surfaces. Though their work was insightful, their model is not effective for describing the imaging process. Since 2007, the Media Lab at the Massachusetts Institute of Technology (MIT)⁴ has been studying transient imaging based on femtosecond laser range-gated imaging technology and proposed an algorithm for extracting object information from multireflection, realizing a means of seeing around the corner. In 2012, researchers at MIT⁵ realized a three-dimensional reconstruction of hidden targets utilizing diffuse reflections of ultrashort laser pulses using an intermediary reflective surface. Since 2012, researchers at the Beijing Institute of Technology (BIT)^6,7 have been studying non-line-of-sight imaging. The BIT group uses a nanosecond laser range-gated imaging system to realize non-line-of-sight imaging of an USAF 1951 standard target at different distances with glass and ceramic tile as the respective intermediary reflective surfaces. These existing researches show the potential application prospects of non-line-of-sight imaging in areas such as urban warfare, counter-terrorism, and disaster relief.

Since non-line-of-sight imaging based on laser range-gated imaging is affected not only by the parameters of the laser range-gated imaging system but also by the reflection characteristics of the intermediary reflective surface and the scene, construction of an optical imaging model is important to the development and application of non-line-of-sight imaging. This paper describes the construction of an image contrast model of non-line-of-sight imaging from the viewpoint of a two-dimensional illumination distribution on the photosensitive surface, in order to analyze the performance of a non-line-of-sight imaging system utilizing a laser range-gated imaging system and to provide a theoretical basis for actual non-line-of-sight imaging system design.

2.

Principle of Non-Line-of-Sight Imaging

The key to non-line-of-sight imaging is to make it possible to see a street-corner target or an indoor target by utilizing an intermediary reflective surface, as illustrated in Fig. 2. A is the observer, B is the target under observation, and C is the intermediary reflective surface. B is outside the range of objects that can be directly imaged by A, so that C is utilized to realize non-line-of-sight imaging of target B.

Fig. 2

Non-line-of-sight imaging and modeling of the intermediary reflective surfaces.

A laser range-gated imaging system, consisting of a nanosecond laser and an intensified charged coupled device (ICCD) imaging system, is located at location A. The ICCD imaging system is focused on location B. The beam from the nanosecond laser from location A experiences first-order reflections (consisting of first-order specular reflection and first-order diffuse reflections, where the first-order diffuse reflection contains first-order backscattering and first-order forward-scattering) at the intermediary reflective surface C. The first-order specular reflection and first-order forward-scattering illuminate target B and then are reflected at target B in a Lambertian fashion (the second-order reflection). First-order backscattering is received by the ICCD imaging system, forming a defocused image and reducing the image contrast of target B. The second-order Lambertian reflection strikes the intermediary reflective surface C and experiences third-order reflections (consisting of third-order specular reflection, third-order backscattering, and third-order forward-scattering) at location C. Third-order specular reflection is received by the ICCD imaging system, forming a focused image of target B. Like the first-order backscattering, the third-order forward-scattering is also received by the ICCD imaging system, reducing the contrast of the image of target B.

For certain intermediary reflective surfaces (like ceramic tile), the effective target signal illuminance (third-order specular reflection) may be overwhelmed by the first-order backscattering and third-order forward-scattering, resulting in the target being undetectable. The first-order backscattering is apparently larger than the third-order forward-scattering, so the first-order backscattering is the dominant factor reducing the contrast of the image of target B. Thus, a better non-line-of-sight image of target B can be obtained by filtering out the first-order backscattering, which can be realized by laser range-gated imaging. Meanwhile, an image intensifying technique is also needed because the effective target signal illuminance (third-order specular reflection) is typically weak. The intermediary reflective surface is also a very important factor in non-line-of-sight imaging, which can be generalized to include a full specular reflector like glass (1), a partial specular reflector like ceramic tile (2), and an ideal diffuse surface (3). The study in this paper is mainly carried out on intermediary reflective surface types 1 and 2.

Construction of an optical imaging model of the non-line-of-sight imaging process, which is based on the parameters of the intermediary reflective surface, the parameters of the laser range-gated imaging system, and the properties of the target, is vital for the theoretical analysis and system design of non-line-of-sight imaging based on laser range-gated imaging. This model assumes that the imaging will occur at night or in low-light conditions to reduce the impact of environmental illuminance.

3.

Image Contrast Model of Non-Line-of-Sight Imaging

As shown in Fig. 3, the laser beam is projected onto an intermediary reflective surface forming the light spot $S_1$ and experiences first-order reflections; then the first-order reflections are projected onto the target forming light spot $S_2$. The distance from the laser to the geometric center of $S_1$ is $l_1$. The distance from the geometric center of $S_1$ to the geometric center of $S_2$ is $l_2$. The vector tracing the direction of the incident light from $S_1$ to $S_2$ makes an angle $\alpha$ with ${\mathbf{n}}_1$ (the vector normal to $S_1$) and an angle $\beta$ with ${\mathbf{n}}_2$ (the vector normal to $S_2$). The optical axis of the laser and the optical axis of the ICCD are assumed to be coincident, and the divergence angle of the laser and the field view angle of the ICCD are also assumed to be coincident. In this section, the laser transmission characteristics (attenuation and distribution) in the non-line-of-sight imaging process will be analyzed.

Fig. 3

Diagram of laser propagation.

3.3.

Target Signal Illuminance and Disturbance Caused by the Intermediary Reflective Surface under CW-Laser Illumination

3.3.1.

Illuminance on the target plane $S_2$

The illuminance on the target plane $S_2$ is denoted as $E_2$, which consists of two parts: $E_{2s}$ is caused by the first-order specular reflection of the laser and $E_{2d}$ is caused by the first-order forward-scattering. The illuminance on $S_1$ should be obtained first.

Illuminance on $S_1$

According to Eq. (1), the illuminance on $S_1$ is given by

Eq. (6)

$$E_1=\frac{I_0·\cos \alpha }{l_1^2}=\frac{\mathrm{\Phi }·\cos \alpha }{t_p·{\mathrm{\Omega }}_{\text{laser}}·l_1^2}.$$

First-order specular reflection

The area illuminated on $S_2$ by the laser first-order specularly reflected by $S_1$ can be written as

Eq. (7)

$$s_2=\frac{{(l_1+l_2)}^2·{\mathrm{\Omega }}_{\text{laser}}}{\cos \beta }.$$

Therefore, $E_{2s}$ can be calculated as

Eq. (8)

$$E_{2s}=\frac{P·{\rho }_s}{s_2}=\frac{P·{\rho }_s}{{(l_1+l_2)}^2·{\mathrm{\Omega }}_{\text{laser}}}·\cos \beta .$$

The target distance $l$ is equal to the sum of $l_1$ and $l_2$. If $l$ is much larger than the square root of $s_2$, and $S_2$ is illuminated by the vertically incident laser beam ($\cos \beta =1$), Eq. (8) can be further simplified.

First-order forward-scattering

To facilitate the analysis and calculations, Fig. 3 is expanded along the optical path. Assuming that the optical axis of the laser coincides with the optical axis of the ICCD imaging system, a Cartesian coordinate system is established with the geometric center of the lens as the Cartesian coordinate origin. The optical axis is along the $z$ axis, and the $y$ axis and the $x$ axis are, respectively, parallel to the detector array’s length and width directions, as is shown in Fig. 4. $\phi$ is the angle between the $x$ axis and the projection of ${\mathbf{n}}_1$ on the $xoy$ plane, and ${\mathbf{n}}_2$ points in the direction opposite to the direction of the $z$ axis. $S_1$ and $S_2$ are, respectively, projections of the laser beam on the intermediary reflective surface and on the target.

Fig. 4

Diagram illustrating the propagation of the laser beam expanded along the optical path.

According to the geometric relationship, the projection equations for $S_1$ and $S_2$ on the $xoy$ plane are, respectively, given by

Eq. (9)

$$x^2+y^2-{\mathrm{tg}}^2\theta ·{[\mathrm{tg}\alpha ·(x\cos \phi -y\sin \phi )+l_1]}^2\le 0,$$

Eq. (10)

$$x^2+y^2-{\mathrm{tg}}^2\theta ·{(l_1+l_2)}^2\le 0.$$

$P_1(x_1,y_1,z_1)$ and $P_2(x_2,y_2,z_2)$ are random points, respectively, on $S_1$ and $S_2$, and $d_{S1}$ is a microsurface element around $P_1(x_1,y_1,z_1)$ (as shown in Fig. 5).

Eq. (11)

$$d{E}_{2d}=\frac{L_1·d{s}_1}{l_{p2}^2}·\cos {\gamma }_2·\cos {\eta }_2=\frac{E_1·{\rho }_d·\cos {\eta }_2·\cos {\gamma }_2}{\pi ·l_{p2}^2·\cos \alpha }\mathrm{d}{x}_1\mathrm{d}{y}_1.$$

$d{E}_{2d}$ is the illuminance on $P_2(x_2,y_2,z_2)$ caused by $d_{S1}$. The illuminance is inversely proportional to the square of the distance as in Eq. (11), where ${\gamma }_2$ is the angle between vectors $\overrightarrow{P_1{P}_2}$ and ${\mathbf{n}}_1$ and ${\eta }_2$ is the angle between vectors $\overrightarrow{P_1{P}_2}$ and ${\mathbf{n}}_2=(0,0,-1)$.

Eq. (12)

$$\cos {\gamma }_2=\frac{\mathrm{tg}\alpha \cos \phi (x_2-x_1)+\mathrm{tg}\alpha \cos \phi (y_2-y_1)-(z_2-z_1)}{\sqrt{{(x_2-x_1)}^2+{(y_2-y_1)}^2+{(z_2-z_1)}^2}\sqrt{1+{\mathrm{tg}}^2\alpha }}(0\le {\gamma }_2\le \frac{\pi }{2}),$$

Eq. (13)

$$\cos {\eta }_2=\frac{-(z_2-z_1)}{\sqrt{{(x_2-x_1)}^2+{(y_2-y_1)}^2+{(z_2-z_1)}^2}}(0\le {\eta }_2\le \frac{\pi }{2}),$$

Eq. (14)

$$l_{p2}^2={(x_2-x_1)}^2+{(y_2-y_1)}^2+{(z_2-z_1)}^2.$$

Fig. 5

Diagram of the microsurface element source.

Therefore,

Eq. (15)

$$E_{2d}={\iint }_{S_1}d{E}_{2d}(\alpha +\theta <\frac{\pi }{2}).$$

In summary,

Eq. (16)

$$E_2=E_{2s}+E_{2d}.$$

3.3.2.

Target signal illuminance on the photosensitive surface

Eq. (17)

$$E_{su}=\frac{1}{4}·\pi ·L_u·{\tau }_o·{\rho }_s·{\left(\frac{D_0}{f^{\prime }}\right)}^2/[1+\frac{1}{4}·{\left(\frac{D_0}{f^{\prime }}\right)}^2]=\frac{\pi ·L_u·{\tau }_o·{\rho }_s}{1+4F^2}.$$

Assuming that the target plane and the photosensitive surface of the laser range-gated imaging system are conjugate object plane and image plane, and that $l$ is much larger than $f^{\prime }(l\gg f^{\prime })$, the illuminance on the photosensitive surface caused by the scene can be expressed as Eq. (17), where $L_u={\rho }_u{E}_2/\pi$ is the luminance on $S_2$ (${\rho }_u$ is the reflectance of the target or background, for $u=t$ or $b$), $F=f^{\prime }/D_0$ denotes the F-number, $D_0$ is the aperture diameter, and ${\tau }_o$ is the spectral transmittance of the objective lens.

3.3.3.

Disturbance illuminance on the photosensitive surface caused by scattering of the intermediary reflective surface

The disturbance illuminance on the photosensitive surface caused by scattering of the intermediary reflective surface consists of two parts: $E_d$ (third-order forward-scattering) and $E_{\text{back}}$ (first-order backscattering). Since the intermediary reflective surface is not on the target plane, each point on $S_1$ corresponds to a diffuse plaque on the photosensitive surface, and the diffuse plaque can be expressed by a point spread function. A Cartesian coordinate system is established with the geometric center of the lens as a Cartesian coordinate origin. $P(0,0,l_1)$ is the intersection of the intermediary reflective surface and the $z$ axis, and $P$’ is the image point of $P$, as shown in Fig. 6.

Fig. 6

Imaging relation of a point on the intermediary plane.

According to geometrical optics, the center of the diffuse plaque for point $(x_1,y_1)$ on $S_1$ is given by

Eq. (18)

$$(x_0^{\prime },y_0^{\prime })=(-n{l}^{\prime }{x}_1/l_1,-n{l}^{\prime }{y}_1/l_1),$$

where

Eq. (19)

$$l^{\prime }=\frac{l·f^{\prime }}{l-n·f^{\prime }}.$$

The radius of the diffuse plaque can be written as

Eq. (20)

$$r_0(z)=\frac{1}{2}\left\{\right[1-\frac{(z-n·f^{\prime })·l^{\prime }}{z{f}^{\prime }}]·D_0\}=\frac{1}{2}\left\{\right[1-\frac{(l_1+\mathrm{tg}\alpha ·x_1-n·f^{\prime })·l^{\prime }}{(l_1+\mathrm{tg}\alpha ·x_1)f^{\prime }}]·D_0\},$$

where $D_0$ is the aperture diameter and $n=1$ is the refractive index of the object space.

The point spread function of the diffuse plaque for point $(x_1,y_1)$ can be expressed as

Eq. (21)

$$h(x^{\prime },y^{\prime },x_1,y_1,l_1)=\mathrm{circ}\left[\frac{\sqrt{{(x^{\prime }-x_0^{\prime })}^2+{(y^{\prime }-y_0^{\prime })}^2}}{r_0(z)}\right],$$

with

Eq. (22)

$$\mathrm{circ}(r)=\mathrm{circ}\left(\sqrt{x^2+y^2}\right)=\{\begin{array}{cc}1& \sqrt{x^2+y^2}\le 1\\ 0& \text{otherwise}\end{array}.$$

First-order backscattering illuminance on the photosensitive surface

According to Eq. (21), the first-order backscattering $E_{\text{back}}$ can be expressed as

Eq. (23)

$$E_{\text{back}}=\frac{E_1·{\rho }_d·{\tau }_o·{\mathrm{\Omega }}_p\cos \alpha }{{\pi }^2}·{\iint }_{S_1}\frac{1}{r_0^2(z)}\mathrm{circ}\left[\frac{\sqrt{{(x^{\prime }-x_0)}^2+{(y^{\prime }-y_0)}^2}}{r_0(z)}\right]\mathrm{d}{s}_1,$$

with

Eq. (24)

$${\mathrm{\Omega }}_p=\pi D_0^2/4{(l_1+x_1·\tan \alpha )}^2.$$

${\mathrm{\Omega }}_p$ is the solid angle defined by the rays emanating from point $P_1(x_1,y_1,z_1)$ to the aperture defined by the lens. In fact, since the distances between the camera and every point on $S_1$ are not the same, the value of ${\mathrm{\Omega }}_p$ for each of the points is not the same. As the laser divergence angle $\theta$ is very small, an approximate treatment for ${\mathrm{\Omega }}_p$ is made here.

Third-order forward-scattering illuminance on the photosensitive surface

Eq. (25)

$$d{E}_1^{\prime }=\frac{L_2·d{s}_2}{l_{p_1}^2}·\cos {\eta }_1·\cos {\gamma }_1=\frac{E_2·{\rho }_{\max }·\cos {\eta }_1·\cos {\gamma }_1}{\pi ·l_{p_1}^2}\mathrm{d}{x}_2\mathrm{d}{y}_2.$$

In Fig. 5, $d_{S2}$ is a microsurface element around $P_2(x_2,y_2,z_2)$. $d{E}_1^{\prime }$ is the illuminance on $P_2(x_2,y_2,z_2)$ caused by $d_{S2}$. The illuminance is inversely proportional to the square of the distance as expressed in Eq. (25), where ${\gamma }_1$ is the angle between vectors $\overrightarrow{P_1{P}_2}$ and ${\mathbf{n}}_1$ and ${\eta }_1$ is the angle between vectors $\overrightarrow{P_1{P}_2}$ and ${\mathbf{n}}_2$. Then we obtain

Eq. (26)

$$E_1^{\prime }={\iint }_{S_2}d{E}_1^{\prime }(\alpha +\theta <\pi /2).$$

Since $I_p$ can be calculated as

Eq. (27)

$$I_p(x,y,z)=L_p·\cos \alpha ·d{s}_1^{\prime }=E_1^{\prime }·{\rho }_d·\cos \alpha ·d{s}_1^{\prime }/\pi ,$$

the illuminance on the photosensitive surface caused by $P_1(x_1,y_1,z_1)$ can be written as

Eq. (28)

$$d{E}_d=\frac{E_1^{\prime }·{\rho }_d·{\tau }_o·{\mathrm{\Omega }}_p·\cos \alpha }{{\pi }^2·r_0^2(z)}d{s}_1^{\prime }.$$

In summary,

Eq. (29)

$$E_d={\iint }_{S_1}h(x^{\prime },y^{\prime },x_1,y_1,l_1)·d{E}_d.$$

3.4.

Image Contrast Model of Non-Line-of-Sight Imaging

Non-line-of-sight imaging can be achieved with laser range-gated imaging with a pulsed laser; therefore it is necessary to make a distinction between third-order reflection and first-order backscattering.

3.4.1.

Illuminance on the photosensitive surface under pulsed laser illumination

Under pulsed laser illumination, the illuminance on the photosensitive surface caused by a single pulse is a function of time $t$.

Eq. (30)

$$E_{su}(t)=E_{su}·\mathrm{rect}\left\{\frac{1}{t_p}\right[t+\frac{t_p}{2}-\frac{2(l_1+l_2)}{c}\left]\right\}(u=t,b),$$

Eq. (31)

$$E_{\text{back}}(t)=E_{\text{back}}·\mathrm{rect}\left[\frac{1}{t_p}\right(t+\frac{t_p}{2}-\frac{2l_1}{c}\left)\right],$$

Eq. (32)

$$E_d(t)=E_d·\mathrm{rect}\left\{\frac{1}{t_p}\right[t+\frac{t_p}{2}-\frac{2(l_1+l_2)}{c}\left]\right\}.$$

3.4.2.

Signal and noise under pulsed laser illumination

Considering the time sequence of the laser range-gated imaging system and the discrete nature of the detector unit, the illuminance on the detector unit $(i,j)$ caused by the target and background within a pulse period can be calculated as

Eq. (33)

$$Q_u(i,j)=ab·E_{su}·{\int }_{t_d}^{t_d+t_g}\mathrm{rect}\left\{\frac{1}{t_p}\right[t+\frac{t_p}{2}-\frac{2(l_1+l_2)}{c}\left]\right\}\mathrm{d}t,$$

where $a$ and $b$ are the length and width of the detector unit, respectively.

The illuminance on the detector unit $(i,j)$ caused by scattering of the intermediary reflective surface can be expressed, respectively, as

Eq. (34)

$$Q_{\text{back}}(i,j)={\iint }_{ds}{E}_{\text{back}}[(i+1/2)a,(j+1/2)b]\mathrm{d}{x}^{\prime }\mathrm{d}{y}^{\prime }{\int }_{t_d}^{t_d+t_g}\mathrm{rect}\left[\frac{1}{t_p}\right(t+\frac{t_p}{2}-\frac{2l_1}{c}\left)\right]\mathrm{d}t,$$

Eq. (35)

$$Q_d(i,j)={\iint }_{ds}{E}_d[(i+1/2)a,(j+1/2)b]\mathrm{d}{x}^{\prime }\mathrm{d}{y}^{\prime }·{\int }_{t_d}^{t_d+t_g}\mathrm{rect}\left\{\frac{1}{t_p}\right[t+\frac{t_p}{2}-\frac{2(l_1+l_2)}{c}\left]\right\}\mathrm{d}t.$$

We simplify the imaging system noise and environmental noise by combining them into an equivalent incident illuminance $E_{\mathrm{eq}}$, which is used as shown in the following equation:

Eq. (36)

$$Q_{\mathrm{eq}}(i,j)=E_{\mathrm{eq}}·a·b·t_g.$$

The total disturbance of the non-line-of-sight imaging system can be expressed as

Eq. (37)

$$Q_n=Q_{\mathrm{eq}}+Q_d+Q_{\text{back}}.$$

3.4.3.

Apparent contrast of the image on a photosensitive surface

The apparent contrast of the image on a photosensitive surface can be described by

Eq. (38)

$$C(i,j,l_1,l_2)=\left|\frac{(Q_t+Q_n)-(Q_b+Q_n)}{Q_t+Q_b+2Q_n}\right|=\left|\frac{{\rho }_t-{\rho }_b}{{\rho }_t+{\rho }_b}\right|·\left|\frac{1}{1+Q_n/Q_{tb}}\right|=C_0(i,j)T_{sh}(i,j,l_1,l_2),$$

Eq. (39)

$$Q_{tb}(l_1,l_2)=[Q_t(l_1,l_2)+Q_b(l_1,l_2)]/2,$$

Eq. (40)

$$T_{sh}(i,j,l_1,l_2)=\left|\frac{1}{1+Q_n/Q_{tb}}\right|<1,$$

where $Q_{tb}$ $(l_1,l_2)$ denotes the averaged signal of the target and background, $C_0(i,j)$ denotes the inherent contrast ratio, and $T_{sh}(i,j,l_1,l_2)$ is the contrast transfer function.

3.4.4.

Analysis and discussion

In addition to systematic noise, the scattered disturbance illuminance from the intermediary reflective surface is the main factor in image quality degradation.

• 1. For photoelectric imaging systems based on human eyes, detection and identification of the extended source target depend on the target contour brought out by the apparent contrast of the image. Although people simply believe that 3 to 5% is the contrast threshold for target detection with human eyes, the more effective approaches are based on minimum resolvable contrast (MRC)^8,9 or target task performance (TTP).¹⁰ Enhancing the apparent contrast of the image is conducive to improving the performance of non-line-of-sight imaging system.

• 2. It is impossible to separate $E_d(t)$ from $E_{st}(t)$ because they are synchronized with each other. However, $E_{\text{back}}$ may be reduced or eliminated by time gating, as shown in Fig. 7.

  • • $l_2\ge c·t_p/2$: $E_{\text{back}}$ is separable from $E_{st}(t)$ [as in Fig. 7(a)]. $E_{\text{back}}$ may be filtered out completely without reducing the laser illuminance efficiency, and thus, the highest contrast image is obtained.

  • • $l_2<c·t_p/2$: When the gate opens upon the return of the target signal, part of the first-order backscattering is accepted by the detector, and the image contrast is decreased to some extent. To this end, the image contrast can be improved by adjusting $t_d$ [as in Fig. 7(b)].

Fig. 7

Temporal relationship between pulse propagation and gated detection. (a), (b)

$E_{\text{back}}$ can be totally filtered out by setting the strobe gating delay time to $t_d=2l_2/c+t_p$. An optimum contrast image can be obtained if the gate width is simultaneously reduced to $t_g=2l_2/c$. However, utilization rate of the laser illuminance decreases with increase of $t_d$. The laser range-gated imaging system cannot receive the target signal illuminance with $t_d=2(l_1+l_2)/c+t_p$, and the efficiency decreases to 0. Since the laser range-gated imaging itself is low-signal-to-noise ratio (SNR) imaging, sacrificing the laser illuminance efficiency significantly reduces the SNR of the image. Therefore, practical application of non-line-of-sight imaging with laser range-gated imaging requires careful balance of the overall system.

4.

Imaging Characteristic Analysis of Non-Line-of-Sight Imaging

Based on the image contrast model of non-line-of-sight imaging constructed above, this section analyzes the non-line-of-sight imaging results with different intermediary reflective surfaces. Typical parameters of non-line-of-sight imaging with a laser range-gated imaging system are shown in Table 1. The imaging depth of field is 36 m with a laser pulse width of 120 ns and a gate width of 120 ns; 40 laser pulses are received by the photodetector within a frame when the imaging frame rate is 25 fps and the laser pulse frequency is 1000 Hz, so that a laser illuminance equivalent to 4.8 μJ is received by the photodetector per frame.

Table 1

Parameters of non-line-of-sight imaging with the laser range-gated imaging system.

4.1.

Impacts of Reflection Characteristics of the Intermediary Reflective Surface on Non-Line-of-Sight Imaging

The impacts of the reflection characteristics of the intermediary reflective surface on non-line-of-sight imaging are considered with $l_1=5\mathrm{m}$, $l_2=20\mathrm{m}$, and $\alpha =45\mathrm{deg}$. Since $l_2\ge c$·$t_p/2=18\mathrm{m}$, the target signal illuminance and first-order backscattering are separable.

Figure 8 shows the curves of the target signal illuminance $E_{st}$, the first-order backscattering $E_{\text{back}}$, and the third-order forward-scattering $E_d$ for different values of the diffuse reflectivity ${\rho }_d$ of the intermediary reflective surface with a CW laser. Since $E_{\text{back}}$ and $E_d$ are unevenly distributed over the imaging field, the $E_{\text{back}}$ and $E_d$ in Fig. 8 are the respective means of $E_{\text{back}}$ and $E_d$ in the center area of the imaging field. The curves in Fig. 8 indicate the following conclusions:

• 1. Under a CW laser, the target signal illuminance $E_{st}$ monotonically decreases with ${\rho }_d$, and first-order backscattering $E_{\text{back}}$ monotonically increases with ${\rho }_d$. $E_{st}$ is larger than $E_{\text{back}}$ only when ${\rho }_d$ is $<0.05$ (i.e., diffuse reflection takes up only 5% of the total reflection illuminance, and the specular reflection illuminance dominates), that is to say that non-line-of-sight imaging can be realized with only a CW laser when the intermediary reflective surface has very strong specular reflection characteristics. This conclusion indicates that practical realization of non-line-of-sight imaging must use a pulsed laser range-gated imaging mode in order to filter out $E_{\text{back}}$. In the instance indicated in Fig. 8, $E_{\text{back}}$ can be totally filtered out without reducing the laser illuminance efficiency.

• 2. The target signal illuminance $E_{st}$ spans more than three orders of magnitude from $4.6\times {10}^{-3}\mathrm{W}/{\mathrm{m}}^2$ for a completely specular reflection (${\rho }_d=0$) to $5.1\times {10}^{-7}\mathrm{W}/{\mathrm{m}}^2$ for a nearly completely diffuse reflection (${\rho }_d=0.99$). $E_d$ is several orders of magnitude smaller than $E_{st}$, and $E_d$ is two orders of magnitude smaller than $E_{st}$ even when ${\rho }_d$ is equal to 0.99. Figure 9 shows that the distribution of $E_d$ on the photodetector for $l_1=5\mathrm{m}$, $l_2=20\mathrm{m}$, $\phi =0\mathrm{deg}$, $\theta =2\mathrm{deg}$, and ${\rho }_d=0.5$ is within the same order of magnitude for $\alpha$ (the tilt angle of the intermediary reflective surface) equals 20 or 60 deg. The image quality is severely disturbed by quantum noise when the target signal illuminance is too low, so ${\rho }_d\le 0.7$ is selected as an effective non-line-of-sight imaging condition. $E_{st}$ with ${\rho }_d=0.7$ is one order of magnitude smaller than $E_{st}$ with ${\rho }_d=0$. $E_d$ is always two orders of magnitude smaller than $E_{st}$ when ${\rho }_d$ changes from 0 to 0.7. In view of the above discussion, $E_d$ is often negligible.

Fig. 8

Illuminance on the photosensitive surface versus ${\rho }_d$.

Fig. 9

Distribution of $E_d$ on a photosensitive surface ($\phi =0\mathrm{deg}$, $\theta =2\mathrm{deg}$, and ${\rho }_d=0.5$). (a) $\alpha =20\mathrm{deg}$, (b) $\alpha =60\mathrm{deg}$.

4.2.

Impacts of the Location of the Intermediary Reflective Surface and the Gate Width on Non-Line-of-Sight Imaging

The parameter $t_g$ is chosen to be equal to $t_p$ to fully use the laser illuminance and to reduce the noise for a typical laser range-gated imaging system, in which case the imaging depth of field is $c$·$t_p$. The gate opens when the echo signal reaches the detector, so that the gate delay time $t_d$ equals $2(l_1+l_2)/c$. The following discussion is based on the assumption that $Q_{\mathrm{eq}}$ is equal to 0.

Figure 10 shows a plot of the image contrast $C$ versus distance $l_2$ with $l_1=5\mathrm{m}$ and $t_d=2(l_1+l_2)/c$. The plot shows that $C$ first decreases with $l_2$ and then increases with $l_2$ to a certain constant value at $l_2=18\mathrm{m}$ when $E_{\text{back}}$ is totally separated from $E_{st}$ and that finally $C$ is nearly fixed with increasing $l_2$. Figure 11 shows a plot of the image contrast $C$ versus distance $t_d$ with $l_1=5\mathrm{m}$ and $l_2=10\mathrm{m}$. The plot shows that $C$ increases with $t_d$ and that $C$ obtains its maximum value when $E_{\text{back}}$ is totally filtered out at $t_d=140\mathrm{ns}$.

Fig. 10

$C$ versus $l_2$ [$l_1=5\mathrm{m}$, $t_d=2(l_1+l_2)/c$].

Fig. 11

$C$ versus $l_2$ ($l_1=5\mathrm{m}$, $l_2=10\mathrm{m}$).

5.

Conclusions

This paper describes the construction of an image contrast model of non-line-of-sight imaging with a laser range-gated imaging system on the basis of the modeling of an intermediary reflective surface. The image contrast model analyzes three factors affecting the image contrast of non-line-of-sight imaging with different intermediary reflective surfaces, where the three factors are target signal illuminance, first-order backscattering, and third-order forward-scattering. A non-line-of-sight imaging experiment simulation was carried out with the parameters of an actual laser range-gated imaging system. The simulation results show that (1) the first-order backscattering is the dominant disturbance to non-line-of-sight images and can be effectively filtered out by laser range-gated imaging and (2) the effect of the third-order forward-scattering on non-line-of-sight images is often negligible since the third-order forward-scattering is insignificant compared to the target signal illuminance in most cases. The image contrast model of non-line-of-sight imaging with a laser range-gated imaging system gives insights into the theoretical analysis and system design of non-line-of-sight imaging.

The simulation results also show that the reflection characteristics of the intermediary reflective surface have a significant effect on non-line-of-sight imaging. The bidirectional reflectance distribution function (BRDF) is a more comprehensive description of surface reflection characteristics; thus, BRDF modeling of typical construction material surfaces is a direction for further research. Meanwhile, since non-line-of-sight imaging with laser range-gated imaging system is usually low-SNR imaging, estimating the range of non-line-of-sight imaging using MRC or TTP is another direction for further research.