1.

Introduction

Holographic display is a true 3-D display technology that has the ability to spontaneously present all depth cues from multiple views. The technology does not depend on any visual aids and promises no visual fatigue to the human visual system. As such, it is considered the ultimate 3-D display technology with market potential for various applications in the near future.¹

According to Stanley ,² the displayed image size of 3-D objects depends proportionally on the number of pixels of the hologram. As such, increasing the hologram pixel count for a specific field of view and wavelength enable us to increase the image size of reconstructed 3-D objects.

With this in mind, for a full parallax holographic 3D display with a $7\text{-}\mathrm{in.}$ diagonal image size and a $20\mathrm{deg}$ field of view will require about $4.5\times {10}^9\text{pixels}$ . This is still a few orders of magnitude beyond the pixel count of single spatial light modulators (SLMs) currently available in the market. As an example, one of the advanced projectors by NHK and JVC contains a liquid-crystal-on-Silicon (LCOS)-based SLM that can demonstrate³ $35\mathrm{Mpixels}$ $(8192\times 4320\text{pixels})$ , which is still about 120 times less than the requirement to produce a holographic 3-D display system with the displayed image size already mentioned.

Various implementations of holographic 3-D display technology have been achieved by different groups all over the world. To solve the problem of the limited pixel count of a single SLM, one approach is to replicate SLM itself by using Qinetiq’s active tiling (AT) system.² The AT system utilizes a set of replication optics to produce multiple images of an electrically addressed SLM on an optically addressed SLM (OASLM). The display of a $100\text{-}\mathrm{Mpixel}$ computer-generated hologram (CGH) was demonstrated with their holographic 3-D display system.

Another approach is to physically increase the number of SLMs, as proposed by a research group in Japan called Telecommunication Advancement Organization^{4, 5} (TAO). They used five LCDs arranged horizontally, which resulted in a total pixel count of about $15\mathrm{Mpixels}$ $(\mathrm{16,000}\times 960\text{pixels})$ . In these displays, a large image and a wide viewing zone were obtained, but the vertical parallax was discarded.

Research institutes such as the Massachusetts Institute of Technology⁶ (MIT) and the Institute of Symbiotic Science and Technology, Tokyo University of Agriculture and Technology,⁷ have each developed a holographic 3-D display system using the horizontal-parallax-only (HPO) approach. Their approach involves scanning a 1-D horizontally limited parallax line hologram using galvanometric scanners to form a 2-D hologram on a vertical diffuser. These displays sacrifice vertical parallax to reduce the resolution requirement for the SLM. Large-size images with wide viewing angles were also achieved using the HPO approach.

It is observed that the approaches employed by Qinetiq² and TAO (Refs. 4, 5) involve tiling of subholograms predivided from a reconfigurable CGH with a high pixel count before reconstructing the tiled CGH, whereas the HPO approach requires scanning 1-D holograms onto a vertical diffuser to form a 2-D hologram, discarding vertical parallax.

In this paper, we introduce a new approach whereby subholograms computed for the subobjects that were predivided from a single 3-D object are reconstructed before tiling them in space. In our system, full-parallax binary digital subholograms instead of linear HPO holograms are first launched onto a high-speed digital micromirror device (DMD). A true holographic 3-D subobject is be reconstructed by illuminating the subhologram on the DMD with a red laser.⁸ The reconstructed subobjects are finally time-sequentially tiled using a two-axis scanning mirror device. Our presented system only uses one DMD and does not require an OASLM for hologram tiling or a vertical diffuser. We also present a new tiling sequence and a shutterless system design to enable effective tiling of subobjects reconstructed from subholograms to achieve a $50\text{-}\mathrm{Mpixel}$ display.

2.

System Overview

Our holographic 3-D display system consists mainly of a DMD, a two-axis scanning mirror device, a $50\text{-}\mathrm{mW}$ continuous wave (cw) red laser diode at $655\mathrm{nm}$ , telecentric $f$ -theta scanning lenses, and mirrors. The DMD device from Texas Instruments consists of a DMD chip ( $0.95\text{-}\mathrm{in.}$ diagonal, $10.8\mu \mathrm{m}$ , $1920\times 1080$ ) connected to a Digital Light Processing (DLP) Discovery 4000 starter kit (Digital Controller Board DCB4000). An Accessory Light modulator Package 4 (ALP-4) software suite from Vialux provides an application programming interface to control the parameters of the DMD. One set of filtering optics is used after the DMD to filter out twin image, zero and higher order diffractions due to SLM pixel boundaries. The ultrahigh frame rate $\left(6\mathrm{kHz}\right)$ of DMD with an on-board memory of $1\mathrm{Gbyte}$ enables us to render holograms at video rate. A computer is used to provide the processing power to predivide a 3-D object into subobjects, from which corresponding subholograms (i.e., CGHs) are computed with our new algorithm.^{1, 9} The cw red laser light source is used to reconstruct the subholograms. The two-axis scanning mirror device is used to tile the reconstructed subholograms launched from the DMD. The device contains two mirrors, which are mounted on actuators (Cambridge Technology, galvanometer optical scanner model 6231C), and the movements of the actuators are driven by electrical servo drivers (Cambridge Technology driver model 67723). These drivers are controlled by a controller board (LEC—1 Embedded Controller Card from Lanmark Controls) whereby users can upload user-defined scanning tasks called jobs into the controller. The mirror scanning sequence and timing parameters are input into the jobs to control the movement of the scanning mirrors. The high-speed DMD is used to launch the subholograms to be reconstructed and tiled onto a set of telecentric $f$ -theta scanning lens (Sill Optics ST-S4LFT0093/123). The lens is used together with the two-axis scanning mirrors to ensure that the projection of the reconstructed subobjects is always parallel for the purpose of better alignment.

Figures 1, 1, 1, 1 show the sequence for our approach. First, a single 3-D object is divided into subobjects. Figures 1 and 1 illustrate an example whereby a computer-generated 3-D cube is predivided into four parts, from which four subholograms are computed respectively [see Fig. 1]. As schematically shown in Fig. 1, subhologram 1 is first launched onto the DMD. The laser light source illuminates the surface of the DMD, which reconstructs subobject 1. The reconstructed subobject is then projected onto the two-axis scanning mirrors. Based on its required position, the controller inputs voltage signals to the scanning mirror drivers to actuate the mirrors to the desired position. After projecting subobject 1 onto the telecentric $f$ -theta scanning lens, the mirror is moved to the next position before projecting and tiling reconstructed subobject 2 in space. This sequence is repeated until all reconstructed subobjects have been tiled to form a single 3-D object reconstructed from all four subholograms. The complete projection and tiling of all the reconstructed subobjects in space were done within $40\mathrm{ms}$ , corresponding to a refresh rate of $25\text{frames}∕\mathrm{s}$ (fps). With our current prototype, a full reconstruction of a holographic 3-D object without obvious flickering could be obtained.

Fig. 1

(a) Single 3-D object, (b) subobjects predivided, (c) subholograms computed from subobjects, and (d) schematic drawing showing our approach to tile reconstructed subholograms time sequentially.

Our holographic 3-D display system is not restricted to static 3-D objects because dynamic holographic 3-D objects can also be produced by launching the subholograms for a series of 3-D subobjects to be tiled, which are preloaded onto the DMD onboard memory and reconstructed in sequence to resemble the animated 3-D objects through our approach. However, memory and data transfer become issues when we increase the number of subholograms for a single tiled 3-D object and the animation time.

3.

System Control and Synchronization Using LabVIEW Software

A program was developed using LabVIEW software to control and synchronize the DMD with the scanning mirror device. The interface for the program is a graphical user interface (GUI) (see Fig. 2) that enables users to simply select CGHs to be displayed. A flowchart for controlling and operating our holographic 3D display system is shown in Fig. 3.

Fig. 2

User-friendly interface developed using LabVIEW software.

Fig. 3

Flowchart for controlling and operating our holographic 3-D display system: (a) user selection from user interface, (b) programmed tasks on DMD, and (c) programmed tasks on the two-axis scanning mirror device.

The GUI in Fig. 2 shows the available CGH folders from a Listbox [see Fig. 2]. From this listbox, users select the folders representing the 3-D objects to be displayed. The selected CGHs are then listed in another panel [see Fig. 2]. This CGH selection sequence is indicated by the section of the flowchart shown in Fig. 3. In the meantime, the information obtained from the selected CGH folder is processed by the system control program working in the background that controls the DMD as well as the scanning mirrors. Users can control the loading of CGHs and select the display mode by clicking on the buttons, as indicated by Fig. 2. The interface also contains system indicators to show the current status of the devices as well as to warn users of any errors that have occurred [see Fig. 2].

The DMD control program performs tasks such as initialization of the DMD and memory allocation for the CGH frames [see Fig. 3]. Timing parameters and synchronization modes are controlled from the program. Synchronization modes determine whether the DMD launches CGH frames independently without synchronizing with the scanning mirror device for single object projection (master mode) or in sync with the scanning mirror device for tiling of reconstructed subobjects (slave mode). The program also contains subprograms to communicate with the scanning mirror device and load the necessary scan jobs. The scan job is a job file containing the scanning sequence of the mirrors as well as other timing parameters [see Fig. 3].

4.

Noninterruptible Tiling Sequence of Reconstructed Subobjects

In our system, we were able to display fully tiled 3-D objects without apparent flickering effect at a refresh rate of $25\mathrm{fps}$ . This refresh rate corresponds to a period of $40\mathrm{ms}$ for a whole tiled frame. Conventional tiling sequences that could be used to tile the reconstructed subobjects are raster and boustrophedral tiling sequences. As an example, 48 subholograms computed for 48 subobjects predivided from a single 3-D object are tiled following the sequence indicated by the number within the tiles shown in Fig. 4. The dotted lines show the large jump steps taken by the respective tiling method. Figures 4 and 4 illustrate the raster and boustrophedral tiling sequences, respectively.

Fig. 4

Tiling of reconstructed subobjects using the (a) raster tiling sequence, (b) boustrophedral tiling sequence, and (c) noninterruptible tiling sequence.

Both the raster and boustrophedral tiling sequences are not continuous due to the large jumps caused by retrace and/or flyback steps. For the raster tiling sequence, this discontinuity is evident in the retrace step when tiling of subobjects reaches the end of every row or during the flyback step on reaching the end of the frame. The discontinuity in the boustrohpedral tiling sequence is present during the flyback step when tiling of subobjects has reached the end of the frame. Since the large jumps require a longer time to move and stabilize the scanning mirrors as compared to the continuous movement between adjacent tiles, the preceding two tiling sequences are not efficient.

In our system, we have adopted a new noninterruptible tiling sequence without any retrace and/or flyback steps. As shown in Fig. 4, no large jumps are necessary throughout the tiling process of a whole frame. During the tiling process, both $X$ and $Y$ axis scanning mirrors move in a “stop-and-go” manner, i.e., after the scanning mirrors project each reconstructed subobject onto its corresponding tile position in space, the mirrors are then moved to the next position and are stabilized before projecting subsequent reconstructed subobjects. This whole process is controlled by the LEC—1 controller card. To implement the noninterruptible tiling sequence of 48 subobject tiles (6 $\text{rows}\times 8$ columns), as shown in Fig. 4, the card outputs $X$ and $Y$ axis voltage signals to the electro servo drivers. These drivers are responsible for actuating the $X$ and $Y$ axis scanning mirrors according to the timing diagram shown in Fig. 5. The flat regions of the $X$ and $Y$ axis voltage signals correspond to the stabilization of the $X$ and $Y$ axis scanning mirrors to project a reconstructed subobject from the DMD to a specific column and row position in space. The DMD projects only a single subhologram when the controller card issues a $5\text{-}\mathrm{V}$ logic high-synchronization signal to the DMD device.

Fig. 5

Timing diagram for the noninterruptible tiling of 48 reconstructed subobject tiles (6 $\text{rows}\times 8$ columns) using the $X$ and $Y$ axis scanning mirror device.

The settling time of the mirrors is dependent on the mirror jump speed and the distance between two tile positions. When an input step voltage is being sent to the servo drivers, it will result in the actuation of the mirrors. The higher the input voltage, the larger will be the scanning mirror deflection. The input voltage to the servo driver was varied for the full range of the servo driver $(\pm 10\mathrm{V})$ and the corresponding scanning mirror position response was studied to establish the relationship between the change in input step voltage and settling time, as shown in Fig. 6. The settling time of the scanning mirror was obtained with an oscilloscope by measuring the time taken for the scanning mirror position to actuate from initial position upon application of the input step voltage to 99% of the final stabilized position of the scanning mirror. The position of the scanning mirror is indicated by the voltage signals tapped from position detectors within the servo drivers, which are read from the oscilloscope. The experimental results from Fig. 6 show that as the change of input voltage $\left(\Delta V\right)$ increases, the settling time also increases. This poses a limitation on how fast the tiling can be done as the scanning mirror does not jump to a certain tile location instantaneously.

Fig. 6

(a) Dependence of settling time on the change of input voltage $\left(\Delta V\right)$ and (b) comparison of the total time taken to tile the subobjects with different tile numbers among different tiling sequences.

Large input voltages are required to achieve the large jumps from row to row (raster tiling sequence) and to perform the large jump from end of the frame back to the first tile location (raster and boustrophedral tiling sequences). These large input voltage values cause a longer settling time for the mirrors and the accumulation of these large jumps inevitably causes the tiling to be slow.

By considering the settling time from tile to tile, a comparison of the total time taken to tile all the subobjects with different tile numbers, as listed in Table 1, using raster, boustrophedral, and the noninterruptible tiling sequence is shown in Fig. 6. We assume that a tile size of $0.25\times 0.3\mathrm{in.}$ is used, and the distance between the mirrors and the tiling screen is $25\mathrm{cm}$ . According to the scanning mirror device specifications, $1\mathrm{V}$ of input voltage corresponds to $2\mathrm{deg}$ of angular mirror deflection. This relationship was used to calculate the settling time. Table 2 further compares the difference in total time between the noninterruptible and raster as well as boustrophedral tiling sequences. We can see from Fig. 6 and Table 2 that the noninterruptible tiling sequence has the minimum required time for tiling the whole frame.

Table 1

Diagonal dimension of tiled screen with different tile numbers.

Table 2

Comparison of total time differences between the noninterruptible and raster as well as boustrophedral tiling sequences.

Furthermore, to achieve the large jump steps, large currents are required to drive the servo to actuate the mechanical mirrors. This leads to high heat dissipation. Some servo systems have protection circuitry that will limit the amount of current flowing through the servo coil when it reaches its thermal budget. In time, the mirrors will inevitably slow down when large jumps are required. This will affect the reliability and repeatability of the scanning mirror positioning. Therefore, in this aspect our noninterruptible tiling sequence shows an advantage over the raster or boustrophedral tiling sequence.

5.

Implementation of Shutterless System Design Using the Black Frame Method

During the transition from one DMD frame to the next, a “sweeping effect” is observed if the laser is not blocked, as shown in Fig. 7. Conventionally, high-speed shutters are used to block the laser when the scanning mirror actuates from one tile location to another.

Fig. 7

(a) Sweeping effect observed between two reconstructed subobjects and (b) removal of sweeping effect with the black frame insertion method.

Mechanical shutters have some disadvantages as they have a limited switching bandwidth. Actuating the mechanical shutters too fast will result in mechanical failure. Slowing down the mechanical shutter to prevent mechanical failure will adversely affect the tiling rate. On the other hand, optical shutters are two state devices that can transmit and block a light path in the on and off state at high speed using liquid crystal devices. These components can reach a switching speed higher than $50\mathrm{kHz}$ . However, the disadvantages of using optical shutters are that it could not reach 100% transmission in the on state. Using optical or mechanical shutters requires additional space in the display system as well as extra driver circuitry and programming to synchronize the device with the DMD frames.

Modulatable lasers are a better alternative compared to the optical and mechanical shutters. This is because the laser transmission is controlled by the laser itself instead of using additional hardware. As a result, the system footprint remains the same. However, similar to the optical and mechanical shutters, this will incur extra circuitry to drive the modulatable laser to synchronize with the DMD frames.

The method implemented in our system takes advantage of the DMD itself to transmit and block light to the scanning mirrors. During the transition from one frame to the other, a black frame is launched onto the DMD. This prevents the laser illuminating the DMD surface from being reflected to the scanning mirrors. During the time period when the black frame is launched, the scanning mirrors will move from one tile to the next tile location, which will not cause the sweeping effect. As the DMD has a high frame rate of up to $6\mathrm{kHz}$ , high-speed tiling is achievable without the use of any extra shutters or modulated lasers that require extra circuitry for control signal synchronization.

Subobjects reconstructed from the DMD that will eventually be tiled using the scanning mirror device to form the whole 3-D object had to be time sequentially launched and synchronized with the scanning mirrors. The timing diagram to synchronize the scanning mirror device with DMD and black frame insertion is shown in Fig. 8. The BF is launched only when the mirror is moving from one location to the next. When the mirror stabilizes, the controller card sends a signal to the DMD to launch the subhologram. Throughout the process, the laser source is constantly on. Figure 7 shows the result after implementing the BF insertion method between two reconstructed subobjects 1 and 2.

Fig. 8

Timing diagram for insertion of black frames (BFs) between subobjects $\mathrm{SH}n$ , where $n$ refers to subobject sequence.

To test this idea further, the BF insertion method was implemented to tile 120 subholograms (12 $\text{rows}\times 10$ columns). Figure 9 shows the tiling of 120 zero orders of laser spots reflected from DMD with noninterruptible tiling sequence with and without black frame insertion. Due to distortions from the telecentric $f$ -theta scanning lens, the rows and columns are not aligned perfectly straight.

Fig. 9

Tiling of 120 (12 $\text{rows}\times 10$ columns) zero orders from DMD with noninterruptible tiling sequence (a) with and (b) without BF insertion.

6.

Tiling of 24 Reconstructed Subobjects Using a Scanning Mirror Device

As a proof of concept to display a tiled 3-D object from reconstructed subobjects with high pixel counts using a two-axis scanning mirror device, we successfully displayed a holographic 3-D teapot from our display system [see Fig. 10]. The 3-D teapot in Fig. 10 was obtained by tiling 24 reconstructed subobjects using the noninterruptible tiling sequence. These subholograms were computed from subobjects obtained by predividing a computer-generated model of a 3-D teapot, as shown in Fig. 10. We also implemented the BF insertion method between subobjects to remove the sweeping effect.

Fig. 10

(a) $50\text{-}\mathrm{Mpixel}$ display of holographic 3-D teapot and (b) computer-generated model of teapot predivided into 24 subobjects.

With each subhologram having a pixel count of $1920\times 1080\text{pixels}$ , the teapot in Fig. 10 has a total pixel count of about $50\mathrm{Mpixels}$ . As compared to a reconstructed 3-D teapot obtained from a single DMD frame, the image size has increased proportionally with the number of subobjects tiled. Currently, our display system has a maximum image size of approximately $70\times 30\mathrm{mm}$ with a small viewing angle of about $5\mathrm{deg}$ . We are still investigating to further improve our system performance in terms of these two parameters.

Note also that the maximum number of subobjects that could be tiled is not limited to 24, as Fig. 9 proves that the system could tile 120 subobjects, which could produce a display of $240\mathrm{Mpixels}$ . As the scanning mirrors have a minimum response and stabilization time, the maximum number of subholograms that could be tiled within a certain time, e.g., $40\mathrm{ms}$ , is limited. Limitations from the scanning mirrors and the DMD also limit the maximum achievable refresh rate of the fully tiled reconstructed subobjects. The $25\mathrm{fps}$ refresh rate of our current system can be further increased by using a higher tiling frequency of scanners and higher DMD refresh rates.

Another issue related to our approach is the quality of the tiled 3-D object. The quality is highly dependent on the alignment and attributes of each reconstructed subobject. As a well-defined alignment of the subobjects is difficult, overlapping of the tiles occurs. This overlapping degrades the uniformity of the tiled 3-D object, as can be seen by certain parts of the 3-D teapot spotting higher intensity due to the summation of intensity at the overlapping region. Furthermore, optical distortion (barrel distortion, as can be seen from Fig. 9) in the tiled reconstructed subobjects due to the use of telecentric lens also affected the quality of the 3-D object. We are currently investigating these issues to overcome the problems encountered and results will be reported in the future.

Nevertheless, this system has been able to produce a higher pixel count compared to a single DMD frame. By using a more advanced scanning mirror device together with the approach demonstrated in this paper, the number of subholograms that could be tiled can be increased and scaled to realize a gigapixel-level display in the future.

7.

Conclusion

We developed a new approach to achieve a $50\text{-}\mathrm{Mpixel}$ display with a maximum image size of $70\times 30\mathrm{mm}$ of a full parallax 3-D object by tiling 24 reconstructed subobjects computed from a predivided 3-D object. A $240\text{-}\mathrm{Mpixel}$ -scale display is achievable with our current system as it has the potential to tile up to 120 subobjects. The tiling is done by using a two-axis scanning mirror device with a noninterruptible tiling sequence. A BF insertion method is adopted to remove image sweeping effect, which enables the implementation of a shutterless system design for effective tiling of subobjects. Despite modest results, our approach provides a scalable solution for future gigapixel-level displays.