The development of a simple, robust high-resolution fluorescence endomicroscope is driven by preclinical and clinical needs.¹ Standard wide-field techniques are hampered by their inability to to reject out-of-focus background, generally leading to low signal contrast. Strategies to reduce out-of-focus background have been based on confocal detection^{2, 3, 4, 5, 6} or two-photon excitation,^{7, 8} both requiring some sort of scanning mechanism. Alternatively, out-of-focus background can be rejected in a nonscanning wide-field endoscope by use of structured illumination microscopy (SIM),⁹ which we have implemented with a flexible fiber bundle.¹⁰ While SIM is effective at optical sectioning, we have found that it is highly susceptible to sample motion, the difficulty being that high-resolution image information in SIM is distributed over a series of at least three raw images, meaning that any misregistration between the raw images leads to artifacts in the final processed SIM image. Recently, we have developed a novel hybrid-illumination technique to address this problem.¹¹ In this technique, two raw images are required, only one containing high-resolution information and the other containing low-resolution information. Hence, the name of our new imaging technique: HiLo microscopy.

The two raw images required for HiLo microscopy are based respectively on uniform and nonuniform (or structured) illumination. In our initial implementation of HiLo microscopy, the nonuniform illumination was obtained with laser speckle.¹¹ However, HiLo microscopy is more general than this and can be implemented with any type of nonuniform illumination. In particular, we demonstrate here the implementation of HiLo endomicroscopy with nonuniform illumination in the form of a grid pattern, of the same type as used in SIM. Indeed, our HiLo endomicroscope setup is identical to our previous SIM endomicroscope setup (see Fig. 1 ), except that for HiLo, the spatial light modulator now toggles between two illumination patterns, grid and uniform, whereas for SIM, it sequentially produced three grid patterns of incrementing phase.¹⁰

The principle of HiLo microscopy was described in Ref. 11. In brief, a final optically sectioned HiLo image is constructed from the fusion of complementary in-focus high- and low-frequency image components. High-frequency components in the uniform illumination image are inherently in focus and are extracted with a high-pass filter. In-focus low-frequency components, on the other hand, must be extracted in a more complicated manner, since the simple application of a low-pass filter to the uniform illumination image does not reject out-of-focus background. To reject low-frequency out-of-focus background, we evaluate the local contrast in the fluorescence image obtained with nonuniform illumination. This local contrast is higher for in-focus image components than for out-of-focus components and hence is axially resolved. A multiplication of the local image contrast with the original uniform illumination image then provides an optically sectioned image, although at low resolution. A fusion of this in-focus low-resolution image with the complementary high-resolution image (inherently in focus) then leads to a full-resolution image that is axially resolved over all spatial frequencies within the microscope passband.

A key step in HiLo microscopy is the extraction of local image contrast from the nonuniform illumination image. In our previous implementation where the nonuniform illumination consisted of laser speckle, this local image contrast was evaluated by calculating the standard deviation of the nonuniform illumination image intensity over local, coarse-grained resolution areas. In our current implementation with a grid pattern, we will adopt a slightly different, although essentially equivalent, approach based on single-sideband demodulation. To understand this approach, let us phenomenologically decompose our uniform illumination image $I_u\left(\vec{\rho }\right)$ into in-focus and out-of-focus components. That is, we write

where $\vec{\rho }=\{x,y\}$ are spatial coordinates in the image plane. Our final goal is to isolate $I_{\mathit{\text{in}}}\left(\vec{\rho }\right)$ .

The nonuniform image can be decomposed similarly into

where the (imaged) grid illumination is modeled as a sinusoidal pattern of spatial frequency ${\kappa }_g$ in the $x$ direction, with arbitrary phase $\phi$ and (imaged) modulation contrast $M$ . Note that only the in-focus image component appears to be modulated, whereas the out-of-focus component does not, precisely because the latter is out of focus.

$I_n\left(\vec{\rho }\right)$ and $I_u\left(\vec{\rho }\right)$ are the two raw images required for HiLo microscopy. The ratio $R\left(\vec{\rho }\right)=I_n\left(\vec{\rho }\right)∕I_u\left(\vec{\rho }\right)$ of these two images leads to

where $C\left(\vec{\rho }\right)=I_{\mathit{\text{in}}}\left(\vec{\rho }\right)∕[I_{\mathit{\text{in}}}\left(\vec{\rho }\right)+I_{\mathit{\text{out}}}\left(\vec{\rho }\right)]$ is the local image contrast that we have set out to derive. The Fourier transform of $R\left(\vec{\rho }\right)$ —namely, $\mathcal{R}\left(\vec{\kappa }\right)$ —is illustrated in Fig. 2, where we observe that $\mathcal{C}\left(\vec{\kappa }\right)$ [the Fourier transform of $C\left(\vec{\rho }\right)$ ] and its complex conjugate reside in sidebands centered at $\pm {\kappa }_g$ . The more highly contrasted the image, the taller and narrower these sidebands become. While several techniques may be used to extract $\mathcal{C}\left(\vec{\kappa }\right)$ from $\mathcal{R}\left(\vec{\kappa }\right)$ , we adopt the standard technique of single-sideband demodulation. In the frequency domain, this corresponds to isolating only a single sideband ${\mathcal{R}}_{+}\left(\vec{\kappa }\right)$ by applying a one-sided high-pass filter to $\mathcal{R}\left(\vec{\kappa }\right)$ (dashed line in Fig. 2), followed by an inverse Fourier transform to retrieve $R_{+}\left(\vec{\rho }\right)$ . Provided the sidebands are well separated from one another (i.e., they do not overlap), then the local image contrast is given by $C\left(\vec{\rho }\right)={\left[R_{+}\left(\vec{\rho }\right)R_{+}^{*}\left(\vec{\rho }\right)\right]}^{1∕2}∕M$ . From $C\left(\vec{\rho }\right)$ , derived in this manner, and the uniform illumination image $I_u\left(\vec{\rho }\right)$ , we can then infer $I_{\mathit{\text{in}}}\left(\vec{\rho }\right)=C\left(\vec{\rho }\right)I_u\left(\vec{\rho }\right)$ . That is, we can extract $I_{\mathit{\text{in}}}\left(\vec{\rho }\right)$ from $I_u\left(\vec{\rho }\right)$ , thereby obtaining an optically sectioned image containing only in-focus contributions.

There are two difficulties with the preceding procedure. First, the sidebands ${\mathcal{R}}_{-}\left(\vec{\kappa }\right)$ and ${\mathcal{R}}_{+}\left(\vec{\kappa }\right)$ may be so wide as to overlap. This problem can be alleviated by choosing a one-sided, high-pass filter cutoff profile that helps suppress the overlap region. The second problem is that, in general, $M$ is not known a priori. We thus define a new parameter $I_{su}\left(\vec{\rho }\right)={\left[R_{+}\left(\vec{\rho }\right)R_{+}^{*}\left(\vec{\rho }\right)\right]}^{1∕2}{I}_u\left(\vec{\rho }\right)$ that is independent of $M$ . Moreover, we purposefully restrict $I_{su}\left(\vec{\rho }\right)$ to spatial frequencies smaller than ${\kappa }_g$ by applying a low-pass filter to $I_{su}\left(\vec{\rho }\right)$ , with user-defined cutoff frequency ${\kappa }_c⩽{\kappa }_g$ , obtaining $I_{\mathit{lp}}\left(\vec{\rho }\right)=\mathrm{LP}\left[I_{su}\left(\vec{\rho }\right)\right]$ . In practice, $\mathrm{LP}\left[I_{su}\left(\vec{\rho }\right)\right]$ is performed by convolving $I_{su}\left(\vec{\rho }\right)$ with a square window of size $2\pi ∕{\kappa }_g$ (or integral multiple thereof, to minimize the possibility of aliasing). In addition to confining $I_{su}\left(\vec{\rho }\right)$ to a well-defined bandwidth, such filtering helps suppress potential artifacts arising, for example, from a nonperfectly sinusoidal illumination pattern.

Finally, the low-resolution image $I_{\mathit{lp}}\left(\vec{\rho }\right)$ is combined with complementary high-resolution information $I_{\mathit{hp}}\left(\vec{\rho }\right)$ obtained by applying a high-pass filter directly to the uniform illumination image, such that $I_{\mathit{hp}}\left(\vec{\rho }\right)=\mathrm{HP}\left[I_u\left(\vec{\rho }\right)\right]=I_u\left(\vec{\rho }\right)-\mathrm{LP}\left[I_u\left(\vec{\rho }\right)\right]$ . The final processed HiLo image is given by

where $\eta$ is a scaling factor introduced to ensure a seamless transition of the frequency content of $I_{\mathit{hilo}}\left(\vec{\rho }\right)$ across the cutoff frequency ${\kappa }_c$ . [In effect, the introduction of $\eta$ compensates for the fact that $M$ is unknown—see Ref. 11 for an explanation of how $\eta$ is derived from $I_{\mathit{lp}}\left(\vec{\rho }\right)$ and $I_{\mathit{hp}}\left(\vec{\rho }\right)$ , based on their complementary nature.] We emphasize that $I_{\mathit{hilo}}\left(\vec{\rho }\right)$ is axially resolved over all spatial frequencies, both high and low, within the endomicroscope passband and therefore constitutes an optically sectioned image. Moreover, it provides a faithful rendition of the sample since, to high accuracy, $I_{\mathit{hilo}}\left(\vec{\rho }\right)$ is directly proportional to the in-focus fluorophore concentration in the sample.

Figure 3 provides comparisons of standard wide-field and HiLo endomicroscope images through a fiber bundle. It should be noted that a longer grid period leads to a stronger imaged modulation depth $M$ —however, at the expense of weaker HiLo optical sectioning capacity. A grid period of $30\mu \mathrm{m}$ was found to provide a reasonable compromise between grid pattern contrast and HiLo optical sectioning capacity. A rough measure of this sectioning capacity can be inferred from a measurement of the detected signal strength from a uniform fluorescent half-space whose interface is scanned through the focal plane. A comparison of the signal strengths acquired with wide-field and HiLo endomicroscopy is illustrated in Fig. 3d, where we observe that the HiLo signal strength decays much more precipitously than the wide-field signal strength. The corresponding HiLo axial resolution for a laterally uniform plane is inferred from the derivative of the HiLo signal strength, and is found, in this case, to be about $30\text{-}\mu \mathrm{m}$ FWHM. Note that for a laterally uniform sample, $I_{\mathit{hp}}\left(\vec{\rho }\right)$ vanishes, and the HiLo image is comprised solely of $I_{\mathit{lp}}\left(\vec{\rho }\right)$ .

Finally, a comparison of wide-field and HiLo endomicroscopic imaging of colon tissue in motion is illustrated in the multimedia movies ( Videos 1 and Videos 2 ). Note the absence of a residual grid pattern or other motion-related artifacts. Our net HiLo imaging acquisition rate was about $2\mathrm{Hz}$ (i.e., $250\mathrm{ms}$ per raw image). This rate was software limited and will be improved in future versions of our apparatus. Our imaging resolution is about $2.6\mu \mathrm{m}$ , limited by the Nyquist frequency associated with the (magnified) fiber core quasiperiodicity (see Ref. 10).

10.1117/1.3130266.110.1117/1.3130266.2

In conclusion, we have successfully demonstrated the capacity of HiLo microscopy to provide optically sectioned imaging of fluorescently labeled colon tissue through a flexible optical fiber bundle. We anticipate that this will open new possibilities in high-resolution fluorescence endomicroscopy.