2.1.

Imaging System and Instrumentation

A schematic of a 3D NIR-II fluorescence rotational stereo imaging system is shown in Fig. 1.

Fig. 1

Home-built 3D NIR-II fluorescence rotational stereo imaging system: (a) schematic diagram of the system and (b) photograph of the experimental setup of in vivo optical imaging.

The optical system includes an SWIR camera (InGaAs camera, NIRvana 640; $640\times 512\text{pixels}$, response 900 to 1700 nm) with a 35-mm C-mount zoom lens (LM35HC-SW, Kowa, Tokyo, Japan) and a 793-nm continuous-wave fiber laser (CNI laser, FC-W-793). The final measured optical spatial resolution results is given in Sec. S1 in the Supplementary Material. In air, the measured spatial resolution of the camera is consistent with the theoretical spatial resolution limit of 0.15 mm for the camera’s hardware (Sec. S2 in the Supplementary Material). A ground glass diffuser (Thorlabs DG10-220-MD) was utilized to homogenize the laser beam, creating a uniform light that illuminated objects. The laser power for absorption/fluorescence measures was $50\mathrm{mW}/{\mathrm{cm}}^2$. For blood vessel imaging, a 1300 long-pass (LP) filter (FELH1300, Thorlabs) and a 1000 LP filter were adopted to eliminate light interference below 1000 nm. We employed a 1300 LP filter (FELH1300, Thorlab) on the filter wheel to capture fluorescence above 1300 nm with lower scattering, as the longest emission tail of the NIR-II dye that we utilized could reach this range. Additionally, using a 1000 LP filter (MIDOPT LP1000) mounted on the lens, we were able to filter out scattered light and reduce background noise, thus achieving a high-contrast NIR-II fluorescence image. A 360-deg scanning module consisting of a mouse holder and a rotational stage was adopted to capture the NIR-II fluorescence stereo images of the mice. The homemade animal holders kept the mouse stable during the imaging process. Moreover, the fluorescence images obtained by 360 deg scanning and the 3D mouse’s contour were constructed by stacking different angles. The scanning angle of each section was $\pm 3.5\mathrm{deg}$ from vertical.

2.2.

NIR-II Fluorescence Rotational Stereo Imaging

The rotational stereo vision imaging process is displayed in Fig. 2(a). The first step measured the camera parameters. The second step regulated the epipolar lines in the left and right eye views. Stereo matching is the key step in which pixel-wise correspondences were found between two eye views to generate a disparity map. A semi-global matching algorithm in MATLAB was used to calculate the disparity map. The final step reconstructed 3D information from the disparity map using the triangulation technique. For 3D contour imaging, we acquired 2D mouse contour images with different camera viewing angles of $+\theta$ and $-\theta$ under incandescent lamp illumination. We rotated the mouse and obtained 2D mouse contour images at different viewing angles (0 deg, 90 deg, 180 deg, and 270 deg). Finally, the 2D surface contour images were processed using MATLAB (MathWorks Inc., Natick, Massachusetts, United States) to obtain 3D contour images. Figure 2(b) shows the pipeline of our NIR-II fluorescence rotational stereo imaging. Both 3D vessels and contour imaging followed the same procedures for generating 3D images. Only one camera and a rotational stage were employed in building 3D vessel images from a left-eye view and a right-eye view. The different camera viewing angles of the objects result in displaced object locations on the images. The advantage of using a rotational stage instead of a moving plate is that the rotational stage has a larger stereo baseline. Thus the 3D whole-body vasculature images and contour images can be reconstructed with NIR-II rotational stereo vision.

Fig. 2

(a) Overview of the stereo imaging system for 3D vascular imaging. (b) The pipeline of the NIR-II fluorescence rotational stereo imaging method.

A two-camera stereo vision technique calculates the depth of objects by utilizing two cameras with a stereo baseline length of $b$ to capture images of the same scene from two different perspectives [Fig. 3(a)]. A single-camera stereo vision technique captures two images of the same scene from slightly different angles, allowing for the calculation of the depths of objects [Fig. 3(b)]. The setup of the rotational stereo imaging is shown in Fig. 3(c). The rotational stereo imaging can be transformed into a standard stereovision problem. A rotational stereo technique captures two images of the same scene from slightly different rotational angles, thus enabling the calculation of the depths of objects present in the scene. A rotational angle was based on the baseline length and the distance between the camera and imaging plate, $D$. The different camera viewing angles of the objects cause a stereoscopic disparity in the sensor. The disparity is the displacement of a particular object between the left and right images. Figure 3(d) displays the distance measurement of the stereo imaging. The black and brown images are two images with different camera viewing angles $+\theta$ and $-\theta$ acquired from Fig. 3(c). The baseline $b$ of the stereo coordinate system is the distance between the two eye-view images with their respective optical centers $O_L$ and $O_R$. The focal length $f$ of a lens is the distance from the lens to the digital camera sensor and is much smaller than the object distance $Z$. Two eye-view images with optical centers $O_L$ and $O_R$ are separated by a baseline $b$. $I_L$ and $I_R$ are the image locations on the left and right eye views. We used the following stereo triangulation formula, which is based on a similar triangle theorem, to calculate the object distance $Z$ in a stereo vision system:³⁶

Fig. 3

(a) Two-camera stereo system principle; (b) one-camera stereo system principle; (c) a top view of a rotational stereo vision setup; and (d) schematic of the rotational stereo vision imaging coordination system.

The depth resolution $D_r$ is calculated with the triangulation Eq. (2) as

2.3.

Experimental Animals

The animal study protocol was approved by the Institutional Animal Care and Use Committee of the National Yang Ming Chiao Tung University (1100509, approved date May 9, 2021). BALB/c nude female mice (5 to 6 weeks, 18 to 22 g) were purchased from the National Laboratory Animal Center, Taipei, Taiwan. The mice were housed under normal laboratory conditions ($22\pm 2°\mathrm{C}$) and a regular 12 h light/dark cycle with free access to food and water. To transfer the tumor cells to the mice, a total of $1\times {10}^6$ 4T1 cells with a volume of $200\mu \mathrm{L}$ were injected into the right hindlimb of BALB/c female nude mice subcutaneously.

2.4.

Tumor Model

The 4T1 triple-negative murine breast cancer cell line was purchased from ATCC (American Type Culture Collection, Manassas, Virginia, United States). Cell lines were maintained in RPMI (GIBCO^® invitrogen Inc., Carlsbad, California, United States) medium with 10% fetal bovine serum (HyClone^® Thermo, Waltham, Massachusetts, United States), $50\mu \mathrm{g}/\mathrm{mL}$ of penicillin/streptomycin (Sigma-Aldrich Co., St. Louis, Missouri, United States), and 2 mM of l-glutamine (Sigma-Aldrich Co., St. Louis, Missouri, United States) and were incubated at a 37°C in a humidified incubator with 5% ${\mathrm{CO}}_2$ and passaged every two days. The 4T1 mammary carcinoma was selected in this research due to its highly tumorigenic and invasive characteristics. In addition, the primary tumor could easily be surgically removed, which helps conduct an ex vivo tumor observation.

2.5.

Fluorescent Agent

Our NIR-II imaging system can work with NIR-II fluorescent probes with emission peaks that are in the NIR-II region (1000 to 1700 nm), such as indocyanine green, IR-1061, IR-783, and IR-TPE Pdots. The NIR-II fluorescent agent, IR-TPE Pdots (MW: 2.4 kDa), which was applied in this research, was synthesized by the procedure of our recent study.¹⁹ The emission peak of the IR-TPE Pdot fluorescence spectrum is about 1000 nm and extends over 1200 nm, characterized by fluorescence spectroscopy when excited by a 793 nm NIR laser. The hydrodynamic diameter of the IR-TPE Pdot is about 45 nm, analyzed by dynamic light scattering. The imaging probe was administered at a dosage of 40 mg/kg body weight, and no adverse health effects were observed. All animals exhibited normal behavior and had normal health statuses throughout the study. This dose was selected to prevent signal saturation in the NIR-II fluorescence imaging while still allowing for adequate integration time. The mice were intravenously injected with IR-TPE Pdots ($200\mu \mathrm{L}$, $5\mathrm{mg}/\mathrm{mL}$) ($n=3$) for imaging at different time points. All mice for imaging were anesthetized with 2% isoflurane in air during the tail vein injection of the agent and image acquisition period.

3.1.

NIR-II 3D Rotational Stereo Imaging of Tumor-Bearing Mice

An in vivo test was performed to validate the feasibility of the system. A subcutaneous tumor was implanted in the right leg/hindlimb, and the experiment started when the diameter of the tumor reached 12 mm ($n=3$). An NIR-II fluorescence rotational stereo imaging system collected an NIR-II fluorescence image [Fig. 4(a)]. Then the system was used to acquire a left-eye camera view angle (green) and a right-eye view (blue), which resulted in displaced object locations on the images [Fig. 4(b)].In addition, we applied a vessel-enhancing algorithm based on the Hessian matrix to increase the image contrast, extract vascular structures, and filter non-vascular networks.³⁹ A depth map [Fig. 4(c)] was then calculated from the disparity map according to Eq. (1). The result shows that the tumor vascular network (imaging depth 0 to $-5\mathrm{mm}$) was correctly reconstructed through system performance estimation. The different blood vessel depths are indicated with a color bar. In addition, the blood vessels were displayed in different colors for different depths in the tumor area of the mice.

Fig. 4

NIR-II rotational stereo image of a tumor blood vessel network: (a) NIR-II 2D image (scale bar: 10 mm); (b) two camera views; and (c) depth map of the blood vessels in the tumor region.

3.2.

NIR-II 3D Rotational Stereo Imaging of Whole-Body Vasculature

In this test, we performed blood vessel imaging of mice following intravenous injection of IR-TPE Pdots ($5\mathrm{mg}/\mathrm{ml}$) ($n=3$). 2D fluorescence imaging of blood vessels was captured at 10 min post-injection with a 1300 nm LP filter [Fig. 5(a)]. We acquired a left-eye view (green) and a right-eye view (blue) with a customized NIR-II 3D rotational stereo imaging system with a rotation of 3.5 and 6 deg for 3D NIR-II imaging. The distance between the camera and imaging plate $D$ of 440 mm. Figure 5(b) shows a left-eye camera view angle (green) and a right-eye view (blue). A disparity map can be turned into a depth map from two images taken from distinct camera viewpoints. Figures 5(c) and 5(d) show that the depth map of abdominal vessels (imaging depth 0 to $-5\mathrm{mm}$) was correctly reconstructed through system performance estimation. There are four main blood vessels with two layers. Major abdominal blood vessels were presented in multiple colors at different depths. Specifically, rotational stereo imaging with a larger stereo baseline length can distinguish the different layers of the blood vessels in the abdominal region.

Fig. 5

NIR-II rotational stereo image of a mouse’s abdominal region: (a) NIR-II 2D image (scale bar: 10 mm); (b) two camera views; (c) depth map of the abdominal blood vessels with a small viewing angle; and (d) depth map with a large viewing angle.

3.3.

NIR-II 3D 360-deg Contour Imaging of Phantom

This section displays the reconstructed 3D point clouds of a cylinder phantom with vessel patterns. To validate the feasibility of the system, we performed experiments using an experimental phantom. The phantom was a circular cylinder with a radius of 20 mm and a height of 50 mm. The experiments were performed under incandescent lamp illumination, which could be detected by an NIR-II camera. Later, we used the imaging system to acquire left and right eye views with 8 camera-view angles, totaling 16 images. Each of the two left and right eye-view images were used to reconstruct a depth map. Then all of the depth maps with different viewing angles were transformed into a 3D model. The results of applying the proposed method are shown in Fig. 6. The first row shows the 2D NIR-II fluorescence images captured by a 1000 nm LP filter. The second row shows the 3D point cloud with a texture of different camera view angles (0 deg, 90 deg, and 180 deg). In other words, the rotational stereo imaging system can reconstruct the contour of the phantom.

Fig. 6

3D contour of a cylinder phantom with a blood vessel pattern using NIR-II rotational stereo imaging.

3.4.

NIR-II 3D Rotational Stereo Imaging of Mice Contour and Whole-Body Blood Vessels

Finally, we further experimented on nude mice in our 360-deg NIR-II rotational stereo imaging system. Mice were anesthetized with 2% isoflurane saturated with ${\mathrm{O}}_2$ and were placed on a rotational stage. First, we acquired the 2D NIR-II fluorescence images of whole-body blood vessels under laser illumination. Second, the 2D NIR-II mice contour images were collected under incandescent lamp illumination. The 3D models were reconstructed from eight camera-view angles. Figures 7(a) and 7(d) show the obtained NIR-II 2D gray-level images and fluorescence images of the whole-body blood vasculature in mice. Figures 7(b) and 7(e) present the NIR-II images of the 3D model of the mouse contour with texture and the 3D whole body blood vessels in a color bar in different viewing angles (0 deg, 90 deg, and 180 deg). Figures 7(c) and 7(f) and Video 1 present the 3D depth map of mice contour and blood vessels in a color bar.

Fig. 7

3D mice contour and whole-body vasculature using NIR-II rotational stereo imaging: (a) NIR-II surface images; (b) 3D object of mice contour with texture; (c) 3D depth map of contour; (d) NIR-II fluorescence blood vessel images; (e) 3D object of whole body blood vessels; and (f) 3D depth map of blood vessels and contour (Video 1, MP4, 2.93 MB [URL: https://doi.org/10.1117/1.JBO.28.9.094807.s1]).