1.

Introduction

Assessment of pulmonary functions is important for respiratory disease diagnosis and therapy evaluation. The methods currently used in clinical practice for pulmonary function tests, such as the spirometer and pneumotachography,¹ enable evaluation and direct measurement of breathing capacity in real time. However, because these techniques measure exhaled and inhaled air volume or airflow speeds, they lack the ability to assess regional chest wall motion, which is essential for kinetic analysis of respiratory physiology² and for diagnostic evaluation of the pulmonary diseases with abnormalities in chest wall movement, such as pulmonary edema and thoracoabdominal asynchrony.³ Additionally, the necessary use of the nose nip, mouth connectors, or masks that these techniques employ commonly interferes with subjects’ normal respiration, and is very difficult for the patients with breathing difficulties.

Konno and Mead⁴ established that respiratory function could be evaluated by measuring volume changes in the whole chest wall, including the rib cage and abdomen, and many techniques of chest wall measurement have been subsequently developed for respiratory volume estimation, which can provide noninterfering assessments of respiration. Some simple respiratory sensors, such as inductive plethysmography, strain gauges,³ magnetometers,⁵ long-period gating sensors,⁶ and fiber optic sensors,⁷ used to measure area, perimeter, diameter, or curvature variation of one or a few particular cross sections of the chest wall, have been applied in respiratory monitoring and analysis of chest wall motion. These techniques, however, mainly measure parameters in specific positions or sections, rather than whole surface motion. This results in technical difficulties when estimating respiratory volume, such as the need for complicated physiological models and additional calibration procedures with other instruments for each subject.

Unlike the preceding techniques, optical methods have the advantages of noncontact and whole-field measurement, and provide opportunities to accurately assess respiratory volume by measuring morphological changes in the chest wall. Saumarez⁸ used an automatic light-sectioning system to continuously measure movement of the torso surface during breathing. This method, however, was not time-efficient and was not feasible for routine use. Recently, Ferrigno ⁹ presented a method using an optical reflectance motion-analysis system to estimate volume changes in the chest wall by measuring 3-D coordinates of multimarkers placed on the rib cage and abdomen. The data acquisition speed of the technique was very high and the position determination was direct with high accuracy. The method could also assess compartmental breathing¹⁰ to provide capability for respiratory kinetic analysis.^{2, 11} However, the spatial resolution was dependent on the number of markers placed on the thoracoabdominal wall, whose distribution was sparse, and the marker attachment procedure is inconvenient.

Recently, progress in using structured light to measure surface topography^{12, 13, 14} has provided opportunities for 3-D surface shape acquisition with high spatial and temporal resolutions, which enable us to explore new ways to measure chest wall movement in respiration. Unlike the optical reflection system, which uses sparsely distributed markers, structured light can offer direct, full-field information about the chest wall surface with high spatial resolution. Aoki ¹⁵ used a near-IR multiple slit-light projection to monitor the respiration of subjects at supine posture. The volume change estimation in that study, however, may include bias due to variable projection coverage on the chest wall for different respiratory states.

In this paper, we present a newly developed technique to evaluate respiratory volumes by combining color structured light for 3-D shape measurement with the use of a small number of active light markers for chest wall region determination. The device is simple and convenient, using only a projector to illuminate the structured light over the chest wall with color stripes and a video camera to record the height-modulated images. The positions of the active markers recorded by the same camera define the measurement boundaries of the moving chest wall to offer a consistent region for volume evaluation in respiratory processes. With this system, the measurement can be performed dynamically from a single side of the human body when positioned against a wall, from two sides when the subject is in standing posture, and from one side with the subject in supine posture. The respiratory volume estimation using this method was validated by comparing the measured results with those from pneumotachography performed at the same time, during deep and spontaneous breathing by the subjects. Isovolume tests¹⁰ were also carried out to show the measurement variance of the optical system. Moreover, the proposed method can not only assess whole respiratory volume with high accuracy, but also estimates regional pulmonary function by measuring regional motions of the chest wall. After validation by isovolume tests, an experiment was conducted to demonstrate function estimations for thoraxes of a patient before and after clinical treatment.

2.

Optical Measurement

2.1.

Projection Pattern

To acquire a 3-D shape of the whole chest wall in real time, a color pattern consisting of parallel straight stripes encoded with the De Bruijn sequence¹⁶ was used to illuminate the object. Such a sequence of order $m$ over an alphabet of $n$ symbols is a circular string of length $n^m$ , providing a window property that the subsequence with length $m$ appears only once, which is critical for identifying each stripe in the pattern. The sequence is obtained by searching a Hamiltonian circuit over a De Bruijn graph whose vertices are the words of length¹⁶ $m+1$ . To enhance the robustness of the recognition, an additional rule that the adjacent symbols must be different is also employed in the De Bruijn sequence design.^{13, 14} With that rule, a sequence of length $l=n{(n-1)}^{m-1}$ is generated from an altered De Bruijn graph produced by deleting all the vertices with same adjacent symbols from the traditional De Bruijn graph. Figure 1 shows a pattern generated by five colors ( $n=5$ , red, yellow, green, cyan, magenta) with a three-order $(m=3)$ De Bruijn sequence, where each stripe can be identified with a unique code in the whole sequence, composed by the colors of its two adjacent stripes and itself.

Fig. 1

Pattern with 120 strips generated by five colors with a three-order De Bruijn sequence. (Color online only.)

2.2.

Triangulation Measurement

The surface measurement of the chest wall is based on triangulation of the structured light system, which consists of a projector for pattern illumination and a video camera for image recording. Figure 2 shows a typical arrangement of the system for surface height measurement. The pattern is projected onto the chest wall by the projector at $O_p$ from an inclined direction, and the video camera (Sony Digital HandyCam, TRV60E) placed at $O_c$ captures the pattern from the normal direction to the reference plane. The optical axis of the camera is perpendicular to the reference plane, and the connection line between the optical centers of the camera and projector $\left(O_c{O}_p\right)$ is parallel to the reference plane. In this geometry, the pattern projected at point $B$ on the reference plane is intersected by the chest wall with height $h$ , causing a projection movement to the point $C$ on the chest wall surface, which results in a corresponding shift $d$ on the image plane of the camera. Because the triangle $\Delta ABC$ is similar to $\Delta O_c{O}_{p}C$ , the height $h$ can be obtained by measuring the lateral shift $d=\left|AB\right|∕k$ ( $k$ is the ratio of the length of real object and its image) on the recorded images, with triangulation of¹⁴

Eq. 1

$$h=\frac{Hdk}{dk+D},$$

where $H$ and $D$ are the distances of the camera from the reference plane and from the projector, respectively, and $k$ can be obtained by the system calibration.

Fig. 2

Optical geometry of projection and imaging for triangulation measurement.

2.3.

Calibration

The planes of several boxes with different sizes are used as varied height references to determine the system parameters in calibration. The lateral shift $d_i$ corresponding to each known height $h_i$ is acquired from the images of the box planes, which are composed of a set of $(h_i,d_i)$ (seven pairs in our experiment) for the system. To calculate the surface height $h$ from image shift $d$ using Eq. 1, three parameters should be obtained in the system calibration: $H$ , $D$ , and $k$ . To simplify the calibration procedure, a linear equation is derived from Eq. 1: $Hd-hD∕k=hd$ with two unknown parameters ( $H$ and $D∕k$ ), which can be calculated by the linear least-squares method. Figure 3 shows an example of the calibration result as the nonlinear relationship between and $d$ and $h$ .

Fig. 3

Example of the relation between the image lateral shift $d$ obtained by calibration and the object height $h$ . Stars present the data collected in calibration.

2.4.

Measuring Region

With the obtained 3-D surface topology, the volume between the reference plane and the surface can be calculated by volume integration (to be described in Sec. 2.6). However, there is a technical obstacle to volume change evaluation. For different respiratory states, the chest wall surface changes are mainly in the normal direction (forward and backward movement of the chest wall). In the proposed inclined illuminating system, the coverage of the projected pattern on the chest wall is variable due to these surface changes, which cause the integrating region to be anatomically inconsistent for different respiratory states. As a consequence, bias would be included in the volume change evaluation. Additionally, the necessity to measure all chest wall movements, including the motion in the direction parallel to the reference plane, also requires an anatomically consistent measurement region.

To overcome these obstacles, a small number of active markers (white LEDs) are attached to the chest wall to correct the volume change assessments from the same anatomic region. These markers form a closed boundary by connecting them with straight lines, which defines a measurement region to cover the major part of the moving chest wall. These markers can be conveniently traced in the images captured by the camera during respiration, as shown in Fig. 4 .

Fig. 4

Image processing (a) the pattern captured in measurement, (b) the region automatically determined by the segmented markers, (c) the cropped image, (d) the center locations of each color stripe, (e) the recognized color for each stripe, and (f) the calculated depth map. (Color online only.)

2.6.

Chest Wall Volume Calculation

After the height map of the chest wall surface is obtained, a topographic mesh composed of small triangles is constructed. An integration process to obtain the whole chest wall volume is implemented by calculating the sum of the small columns with the triangles on the topographic mesh and their projection on the reference plane as the top and bottom surfaces, given by $\mathit{Vol}={\sum }_{i=1}^n{V}_i$ , where $V_i$ is the volume of the $i$ ’th column unit calculated by $V_i=S\cos \left(\alpha \right)\left(\mathit{\text{height}}\right)$ , whereas $\alpha$ is the angle between the triangle and the reference plane, height is the distance between the centroid of the triangle and its projection, and $S$ is the area of the triangle (Fig. 5 ).

Fig. 5

Illustration of one column $\left(V_i\right)$ between a triangle $\left(\Delta O_1{O}_2{O}_3\right)$ on the topographic mesh and its projection $\left(\Delta P_1{P}_2{P}_3\right)$ on the reference plane: height is the distance between the centroid of $\Delta O_1{O}_2{O}_3$ and $\Delta P_1{P}_2{P}_3$ , $\alpha$ is the angle between $\Delta O_1{O}_2{O}_3$ and the reference plane, and $S$ is the area of $\Delta O_1{O}_2{O}_3$ . The volume of this column can be calculated by $V_i=S\cos \left(\alpha \right)\left(\text{height}\right)$ .

Notably, the chest wall volume calculation is done inside the measurement region confined by the active markers attached to anatomic locations, as described in Sec. 2.4. As a result, the measurement includes not only the chest wall movements in the direction normal to the reference plane, but also the motion components in the direction parallel to the reference plane, providing a full-scale evaluation of the chest wall volume change.

3.

Validation Experiments and Results

3.1.

Three Types of Measurement Setups

Based on this structured light 3-D measurement technique, we set up three types of illuminating and recording devices for different postures or positions to evaluate respiratory volumes. As shown in Fig. 6 , the single-sided setup is for the standing posture of a subject positioned against a wall and asked to not move their back from the wall during the experiment [Fig. 6]. The inclined single-sided setup for the supine posture, on the other hand, is used for subjects who are lying down on a bed with adjustable obliquity $\left(0\text{to}60\mathrm{deg}\right)$ . In this position, the subject’s back should naturally be kept in contact with the bed surface [Fig. 6]. For subjects having no contact with a reference plane, the double-sided illuminating-recording setup [Fig. 6] is suitable, using a standing posture with two projector-camera pairs to cover the front and the back of the trunk. In this case, the rigid movement of the human trunk can be compensated by depth maps measured from both sides. For each projector-camera pair in these devices, the projector is placed below the camera at a distance of $0.8\mathrm{m}$ to illuminate the subject from the bottom, and the camera is positioned $2.1\mathrm{m}$ in front of the subject’s trunk.

Fig. 6

Illuminating-recording setups for (a) single-sided standing posture, (b) inclined single-sided supine posture, and (c) double-sided standing posture.

For single-sided measurements, 12 active markers are attached to skin on the anterior side of the subject’s trunk using adhesive tape, after system calibration, as shown by the cross stars in Fig. 7 . For the double-sided setup, another 12 markers are attached on the posterior side of the trunk [Fig. 7]. The subject puts on a white close-fitting T-shirt to cover the body before testing begins.

Fig. 7

Positions of the active markers on the (a) anterior and (b) posterior sides of the trunk. Surface anatomic landmarks for the horizontal rows are $1\mathrm{cm}$ under the clavicular line, the nipples, the lower costal margin, umbilicus, and anterior superior iliac crest. The vertical columns are the midline, axillary lines, and the midpoints of the interval between the midline and the axillary lines.

3.4.

Results

3.4.1.

Single-Sided Setup for Standing Posture

Respiration volumes $(N=1800)$ extracted from the volume curves of 23 subjects (s1 to s23) showed a high correlation of 0.998 $(P<0.001)$ between the results from the chest wall motion measurement using structured light method $\left(Vo\right)$ and pneumotachography measurements $\left(Vp\right)$ , as shown in Fig. 8 , with the Bland-Altman plot shown in Fig. 8. The mean difference between the two methods is 5.60% [mean of $\left(Vo\text{-}Vp\right)∕Vp$ ] with a standard deviation of 8.49% [standard deviation of $\left(Vo\text{-}Vp\right)∕Vp$ ]. The comparison indicates that the proposed method is valid for respiratory volume estimation, with values higher than those from pneumotachography. As an example, Figs. 9 and 9 present the data from a female of age 24 (s3), showing the volume curves measured by pneumotachography and by the optical system. Those curves coincide very well, with high correlations of 0.991 $(P<0.001)$ for spontaneous respiration and 0.997 $(P<0.001)$ for deep respiration, respectively. For the isovolume test, Fig. 10 shows the volume curve of a subject (s11) performing isovolume maneuver, showing a standard deviation of $0.0805\mathrm{L}$ in the breath-holding period $\left(15\text{to}24\mathrm{s}\right)$ .

Fig. 8

(a) Comparison of the volumes obtained by pneumotachography $\left(Vp\right)$ and one-side setup for standing posture $\left(Vo\right)$ . The dashed line is the linear fitting, $y=1.065x\text{-}0.004$ . (b) Bland-Altman plot of the volumes for the two methods.

Fig. 9

Examples of volume curve comparison: (a) curves of spontaneous breaths and (b) curves of deep breaths.

Fig. 10

Volume curve obtained by the proposed method during the subject (s11) performing the isovolume maneuver.

3.4.2.

Inclined Single-Sided Setup for Supine Posture

Two healthy subjects (s11 and s24) were recruited for comparison between the inclined single-sided setup and pneumotachography. The respiration volumes $(N=65)$ calculated from the volume curves showed a high correlation of 0.998 $(P<0.001)$ . The measurement difference $[\left(Vo\text{-}Vp\right)∕Vp]$ between the two methods was $1.76\pm 4.03\mathrm{\%}$ ( $\text{mean}\pm \text{standard}$ deviation). Also, a healthy subject (s12) who performed the isovolume maneuver, showed a standard deviation of $0.0764\mathrm{L}$ in the breath-holding period.

3.4.3.

Double-Sided Setup for Standing Position

The test results for two subjects (s11 and s25) using the double-sided setup show a correlation of 0.999 $(P<0.001)$ in comparison with the measurement results from pneumotachography. Measurement difference $[\left(Vo\text{-}Vp\right)∕Vp]$ between the two methods was $4.25\pm 3.49\mathrm{\%}$ ( $\text{mean}\pm \text{standard}$ deviation). For another subject (s13) who performed the isovolume test, the standard deviation in the breath-holding period was $0.0493\mathrm{L}$ .

4.

Study of Regional Chest Wall Change

Compared with the techniques of measuring air flow to assess respiratory volume, such as using a spirometer or pneumotachography, the proposed optical measurement has the advantage of not only estimating whole lung volume changes without any contact on the human body or limitation of normal breathing, but also can evaluate regional pulmonary function by measuring the movement of specified regions of the chest wall. In this paper, we present a single-sided illuminating-recording setup for supine posture to evaluate regional lung function in a subject with different functions of the left and right lung.

To validate the method, a measurement was initially performed on a healthy subject (s13) using the isovolume test. Besides the 12 active markers positioned on the front side of chest wall as already shown, an additional 5 markers, as shown in Fig. 7 by the sun stars, were attached on the trunk to split it into the regions of the left thorax, right thorax, left abdomen, and right abdomen, respectively. During the tests, the subject sequentially performed spontaneous breathing, deep breathing, and then breath holding without air flow but exchanging his internal volume between thorax and abdomen. The volume changes in different regions are presented in Fig. 11 . For the volume curve of the whole chest wall [Fig. 11], the last part $\left(31\text{to}45\mathrm{s}\right)$ is almost a straight line (standard deviation: $0.823\mathrm{L}$ ), showing that the total volume during breath holding remained in the whole trunk, as expected from the isovolume maneuver. On the other hand, the volume curves of the whole thorax and the whole abdomen [Fig. 11] change with opposite phases in this period [correlation coefficient of the volume curves is $-0.991$ $(P<0.001)$ ], reflecting the volume exchanges between them, which cannot be revealed by air flow measurement. Moreover, the volume changes are compared for the left and right abdomen [Fig. 11] and for the left and right thorax [Fig. 11], respectively. They have almost identical volume curves with nearly symmetrical functions of the healthy subject, with correlation coefficient of the volume curves at 0.991 $(P<0.001)$ and 0.982 $(P<0.001)$ , respectively.

Fig. 11

Volume curves of a healthy subject performing isovolume test: (a) whole chest wall; (b) whole thorax region and whole abdomen region, the correlation coefficient between their volume curves is $-0.991$ $(P<0.001)$ $\left(31\mathrm{s}\text{to}45\mathrm{s}\right)$ ; (c) comparison of the right and left abdomen, the correlation coefficient between their volume curves is 0.991 $(P<0.001)$ ; and (d) comparison of the right and left thorax, the correlation coefficient between their volume curves is 0.982 $(P<0.001)$ .

After the study procedure and consent forms were reviewed and approved by the institutional ethics board, the optical method of color structured light was applied to the analysis of regional pulmonary function for a subject (s26: male; age, 15; height, $1.77\mathrm{m}$ , weight, $62\mathrm{kg}$ , BMI, $19.79\mathrm{kg}∕{\mathrm{m}}^2$ ) with pleural effusion in the left thorax (detected by X-ray photography). We measured the volume change of different chest wall regions during spontaneous and deep breathing with the optical system. The left thorax respiratory function was evaluated by the ratio of the volume value of the left thorax to that of the whole thorax. The test was initially performed before scheduled drainage to obtain the comparative functions for the left thorax. After drainage, the left thorax was measured again on the same day to evaluate the regional pulmonary function change after treatment. In this test, the contribution of the left thorax to the whole thorax changes from 39.23 [Fig. 12 ] to 41.88% [Fig. 12] for the spontaneous breath; and from 37.60 [Fig. 12] to 42.53% [Fig. 12] for the deep breath, respectively. The results provide the evidence of respiratory function improvement of the diseased side of the lung after therapy, and prove the capability of the optical method to estimate regional pulmonary functions.

Fig. 12

Volume curves of a patient with the left thorax (solid line), right thorax (dashed line), and the whole thorax (dashdotted line) for (a) spontaneous breath before drainage, (b) deep breath before drainage, (c) spontaneous breath after drainage, and (d) deep breath after drainage.

5.

Discussion and Conclusion

A new method using projecting structured light and tracking markers was proposed to evaluate pulmonary function by measuring chest wall motion in respiration. The structured light technique provides accurate 3-D topology of the chest wall surface with high resolution, and the traced markers define a consistent region for chest wall volume change estimation. With the advantage of noncontact measurement, the proposed method has no interference with the natural respiration of the subjects and is comparable to the functions of the traditional pneumotachography/spirometer systems for pulmonary function evaluation. Compared to other optical methods for evaluating the perimeter or curvature of the chest wall, this technique has the advantage of whole-field measurement with high spatial resolution, which enables the evaluation of exact respiratory volume during respiration without any individual registration procedure. Of course, the portability of the proposed method is not as good as some optical methods, such as the fiber optic sensor, but in this technique, convenience is sacrificed for whole-field measurement.

To demonstrate the applicability of the proposed method in different situations, three types of measuring setups were established. The single-sided illuminating-recording setup is suitable for most subjects with standing posture. The inclined single-sided setup, on the other hand, is more suitable for the subjects who must be in a supine posture. The double-sided setup can compensate for body movement and offers better accurate respiratory volume assessment than the single-sided setups, showing smaller variation during isovolume test than the single-sided setups.

In this paper, pneumotachography was used as the “gold standard” or “reference standard” to validate the new technique. This is reasonable as the accuracy of pneumotachography has been proven to have less than 3% error in volume assessment.¹⁷ All three setups of this method provide measurements highly correlated with those from pneumotachography [correlation coefficient is $R>0.99$ $(P<0.001)$ ], with small positive errors. The Bland-Altman plot also presented good agreement between the two methods. Another trend that can be seen from the Bland-Altman plot is that the absolute difference is relatively large when the respiration volume is large. A potential source of the measurement difference may come from the rigid forward movement of the subjects. For the single-sided setup designed to evaluate volume changes between the chest wall surface and the reference plane, it is not easy for subjects to keep their back close against the reference plane at all times during respiration, especially during deep breathing in the standing posture. In practice, the inaccuracy of the single-sided setup for supine posture and the double-sided setup has much less influence from the rigid movement of subjects. This was proved in the tests of the inclined single-sided setup and the double-sided setup for standing posture, which showed smaller differences than the single-sided setup for standing posture compared with pneumotachography (1.76 and 4.25%, compared with 5.60%). Also, a similar trend can be seen in the isovolume tests.

In the exploration of regional measurement, the isovolume test on the volunteers presents high negative correlation between the volume changes of the abdomen and thorax; whereas the volume change of the whole chest wall is nearly zero. The comparison of left and right regions (abdomen and thorax) shows almost the same volume change curves with high positive correlation. The results indicate that the new method can measure not only whole chest wall volume change, but regional pulmonary volume as well. As an application, the regional measurement of the left and right thorax movement of a patient with pleural effusion shows the proposed technique can really exhibit the difference of regional pulmonary functions. The result presents the improvement of regional pulmonary function after treatment in the diseased side. Although that regional volume change may be not significant due to the limited sample size, the experimental demonstration shows the potential application of the optical system.

In conclusion, this paper proposed a chest wall motion measuring technique for pulmonary function evaluation by combining projecting structured light and tracking markers. In the validation tests of the three established setups, the agreement with pneumotachography and the small variance in the isovolume maneuver proved the accuracy of the proposed technique. More significantly, its ability to evaluate regional chest wall volume was validated by the isovolume test, and the regional respiratory volume evaluation on the patient with pleural effusion shows promise in future clinical applications.