1.

Introduction

Recent years have witnessed the promising development of three-dimensional (3-D) display techniques.^1,2 Levoy and Hanrahan proposed a light field (LF) rendering technique for three-dimensional (3-D) scene reconstruction.³ The major idea is a representation of the LF, which aims to reconstruct the LF of a 3-D scene in a pixel-based rather than a view-based perspective through designed imaging process. Compared to other conventional methods, it generates a higher density of viewing zones such that it provides observers a more comfortable viewing experience.

Substantial efforts have been conducted to achieve a 3-D display using LF rendering. A Hungarian company named Holografika proposed the first commercial LF display system.⁴ Researchers from MIT proposed several display prototypes using multilayer attenuators.^5,6 Zhejiang University proposed a liquid-crystal-display-based LF 3-D display system.⁷ Moreover, Samsung announced a large-scale system using 300 projectors in the 2013 Society for Information Display technical symposium.⁸ From this, it can be concluded that a large-scale system employed with multiprojections is becoming the trend of 3-D display and has commercial prospects.

In a multiprojector-type 3-D display, geometrical calibration is definitely crucial in adjusting the projection accuracy. Alternative methods have been proposed for two-dimensional (2-D) image calibration.^9–12 Mainstream methods mentioned in the references are derived from a combination of a standard camera and several uncalibrated projectors, by which excellent calibration results have been achieved. However, our previous studies mainly focus on calibrating a projected image onto a 2-D plane, for example, making one or more projectors display a stitched and undistorted image on a surface of unknown geometry.^13–15 Meanwhile, there are few reports in the literature of flexible and efficient calibration dedicated for multiprojector-type 3-D displays so far. Different from the calibration of a 2-D image, the light rays in a 3-D scene are considered as vectors such that projected images are interpreted as a combination of vectors. Therefore, the calibration in multiprojector-type LF 3-D display requires that more factors be taken into consideration. Moreover, it shall be implemented more accurately and efficiently by taking the integral imaging process into consideration.

In this study, an automatic geometrical calibration framework for a multiprojector-type LF 3-D display is investigated. Different from the conventional methods, a precise rotary table is added into the calibration framework to enable the usage for a large panoramic viewing angle with a large-scale curved screen. Together with the robust calibration algorithm, which transfers the calibration of a 3-D scene into the calibration of a 2-D image on the diffuser interface, the display performance can be calibrated automatically and accurately. The experimental results verify the effectiveness of the proposed calibration framework. Given that the properties of rays have been taken into consideration, we believe that the calibration algorithm can be applied to most projector-type 3-D displays, not only based on LF reconstruction, but also on a multiview reconstruction.

2.

Principle of Multiprojector-type LF 3-D Display

By convention, rays can be parameterized in a couple of ways in computer graphics.^3,16 Here, we configure it by a point ($x$, $y$, and $z$) and a direction $(u,v)$ as Fig. 1 shows. We define the LF rendering of an object in the following form using homogeneous coordinates:

Fig. 1

The representation of rays in space.

When trying to reconstruct the light filled rendering of an object by multiprojection or other techniques, it is important to guarantee the accuracy of the parameters.

Figure 2 shows a general configuration of a multiprojectors-type LF display, which consists of a series of projectors arranged in a circle and a directional diffuser. Images are projected upward to the opposite side of the screen with the same height. Because of the angle-interval between the projectors, the special diffuser provides a large diffuse angle in the vertical direction for image perspective, while providing a small angle in the horizontal direction for pupil jointing.¹⁷ The vertical diffuse angle of the diffuser depends on the interval angle between the adjacent projectors.

Fig. 2

Illustration of display principle. The two circles show the images seen from views $V_1$ and $V_2$.¹⁸

Consequently, observers will see the jointed stripe images as Fig. 2 shows. At viewing points $V_1$ and $V_2$, the two spatial points $A$ and $B$ are represented by light rays emitted from different projectors and can be seen at different positions. By adding sufficient projectors with appropriate intervals, the stripe images will join together to make up an integral image.¹⁸

3.

Framework of Geometrical Calibration

The image generation algorithm for the display system is deduced under a theoretical situation. In reality, as the number of projectors increases, the installation and adjustment of all projectors become a huge task. It is rather difficult and inefficient to calibrate each projector manually and separately. If any projector is not adjusted well, it results in additional distortion and a defective 3-D reconstruction. To solve this problem, our research has been conducted on automatic geometrical calibration. Taking one projector as an example, the framework of geometrical calibration is presented below.

First, capture the printed standard spot diagram (s-image) and the projector spot diagram (p-image), and calculate the mapping relationship Matrix 1 ($M_1$) between the p-image in the camera space and the original spot diagram (o-image) in the projector space

The calibrated spot diagram (c-image) in the projector space can be obtained by multiplying the s-image with Matrix 1

Then calculate the mapping relationship Matrix 2 ($M_2$) between the o-image and c-image

A transformation matrix $M_2$ is then applied for any image generated from the LF rendering algorithm for the projector. This calibration approach can be naturally extended to a system containing many projectors. Repeat this process for every projector and all projections in the system will finally have a uniform standard.

Without loss of generality, the main module of autocalibration can be summarized as Fig. 3 shows.

Fig. 3

Flowcharts of autocalibration approach.

Figure 4 presents the captured photos taken during the calibration proceeding mentioned above. The spot diagram in Figs. 4(a) and 4(b) is identified and marked with red circles during image processing. Also, Figs. 4(c) and 4(d) are the spot diagram in the projector space before and after the calibration.

Fig. 4

Pictures taken in calibration proceeding: (a) s-image, (b) p-image, (c) o-image, and (d) c-image.

The validity of this method can be demonstrated through discussing several different scenarios. Without loss of generality, the deviation of the projector is divided into two main types, angle deviation and position deviation.

3.1.

Angle Deviation

We interpret the projected image as a combination of vectors. Due to the display principle illustrated in the above paragraphs, the image observed at any viewing position is derived from certain rays emitted from a series of projectors. On the other hand, the voxel in the constructed 3-D scene is also composed of rays from different projectors. If one projector is fixed in the right place, what is left is projecting a correct image onto the correct place in a fixed 2-D plane.

In Fig. 5, the solid lines represent the practical light rays projected from projectors, and the dashed lines represent the extension of rays connecting the viewing position to a virtual 3-D voxel. Note that herein blue lines stand for the incorrect rays before calibration, while red lines stand for the correct rays after calibration. As Fig. 5(a) shows, we assume a projector is placed at the correct place but toward an incorrect direction. In the diffuser interface, the pixel of the projection image which should be located at $A$ is wrongly located at $A^{\prime }$. Also, the observer will see an incorrect voxel $p^{\prime }$ from this projector, resulting in the observer seeing an incorrect 3-D scene. The major idea behind the calibration framework is to calculate the mapping relationship between $A$ and $A^{\prime }$ and assign the information of light ray ${\mathrm{OA}}^{\prime }$ back to OA. In this way, the incorrect voxel p′ will be replaced with the correct voxel $p$, and a correct projection image can be regenerated after calibration.

Fig. 5

(a)–(c) Illustration of angle deviation, vertical and horizontal position deviations.

4.

Experiments and Results

To verify the principle and calibration algorithm, a prototype is constructed as Fig. 6 shows, which consists of a series of circularly arranged projectors and a cylindrical directional diffuser. All projectors are arranged in a circle under the diffuser and staggered horizontally to condense the pupil array. The specifications are shown in Table 1.

Fig. 6

Schematic of the 360 deg display system.¹⁸

Table 1

Specifications of the 360 deg display system.

As Fig. 7 shows, the calibration prototype system used in the experiment employs a precise rotary table set in the center. Meanwhile, a CCD camera is fixed onto the table and a printed standard spot diagram is pasted on the diffuser interface. To be clear, the printed standard spot diagram is generated from the LF reconstruction algorithm based on an ideal prototype.

Fig. 7

Structure of an automatic calibration system: 1. diffuser; 2. printed standard spot diagram; 3. projects array; and 4. CCD camera fixed on precise rotary table.

Figures 8 and 9 show the comparison of the projected spot diagram before and after the calibration. Figures 8(b) and 8(d) are the partial zoomed views of Figs. 8(a) and 8(c). The autocalibration result of spot diagram is illustrated in Fig. 8(d), which is much better than the result for the manual adjustment showed in Fig. 8(c).

Fig. 8

Comparison of projected spot diagram before and after calibration: (a) projected spot diagram before calibration, (b) partial zoomed views of (a), (c) projected spot diagram after calibration, (d) partial zoomed views of (c).

Fig. 9

Comparison of projected three-dimensional scene before and after calibration: (a–b) viewed from left position, (c–d) viewed from center position, (e–f) viewed from right position.

To prove the effectiveness of the calibration framework, we take five spots in different areas marked with yellow circles in Fig. 8(c) to perform the calculation. The calibration accuracies of the spots are listed in Table 2. From the measurement result, the maximum mean deviation is 0.575 pixels.

Table 2

Measurement calibration accuracy analysis.

Figure 9 shows that a better 3-D scene is achieved after calibration from the left, center, and right viewing positions, which proves the effectiveness of the calibration framework. Figures 9(a), 9(c), and 9(e) are the projected 3-D scene before calibration viewed from the left, center, and right view positions, respectively. Figures 9(b), 9(d), and 9(f) are the projected 3-D scene after calibration viewed from left, center, and right view positions, respectively. The performance enhancement is obvious.

5.

Conclusion

This paper presents an automatic geometrical calibration approach for a multiprojector-type LF 3-D display. The calibration framework is proposed and then detailed for calibrating the geometrical deviation and distortion of a projected 3-D scene. Experimental results demonstrate that the projectors in the display system can be adjusted automatically and accurately. The validity of this method has been discussed and we argue that this approach has the robustness to be extended to most existing multiprojector-type displays.

Therefore, with the aid of this automatic geometrical calibration, multiprojector-type LF 3-D displays will see great potential in commercial applications in the near future because of its better performance and easier calibration.