Development of free-viewpoint image synthesis system using time varying projection and spacetime stereo

Tatsuro Mori; Keita Takahashi; Toshiaki Fujii

doi:10.1117/12.2038843

6 March 2014 Development of free-viewpoint image synthesis system using time varying projection and spacetime stereo

Tatsuro Mori, Keita Takahashi, Toshiaki Fujii

Proceedings Volume 9011, Stereoscopic Displays and Applications XXV; 90111X (2014) https://doi.org/10.1117/12.2038843
Event: IS&T/SPIE Electronic Imaging, 2014, San Francisco, California, United States

Abstract

The goal of our research is to develop a real-time free-viewpoint image synthesis system for dynamic scenes using multi-view video cameras. To this end, depth estimation that is efficient and suitable for dynamic scenes is indispensable. A promising solution for this is view-dependent depth estimation where per-pixel depth maps are estimated directly for the target views to synthesize. Such view dependent methods were successfully adopted in previous works, but their depth estimation quality was limited especially for textureless objects, resulting in low quality virtual views. This limitation comes from a fact that their depth estimation depended only on passive approaches such as traditional stereo triangulation. To tackle this problem, we considered to use active methods in addition to the passive stereo triangulation. Inspired by the success of recent commercial depth cameras, we developed a customized active illumination using a DLP projector. The projector casts spacially incoherrent patterns to the scene and makes textureless regions identifiable from the cameras, so that stereo triangulation among the multi-view cameras can be greatly improved. Moreover, making the illuminations time-varying, we can stabilize depth estimation more by using spatiotemporal matching across multi-view cameras based on the concept of spacetime stereo method, and also remove the artificial patterns from the synthesized virtual views by averaging successive time frames. Our system consisting of 16 video cameras synchronized with the DLP projector runs in real-time (about 10 fps) thanks to our sophisticated GPGPU implementation.

Citation Download Citation

Tatsuro Mori, Keita Takahashi, and Toshiaki Fujii "Development of free-viewpoint image synthesis system using time varying projection and spacetime stereo", Proc. SPIE 9011, Stereoscopic Displays and Applications XXV, 90111X (6 March 2014); https://doi.org/10.1117/12.2038843

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available