Real-time 3D Video

Conceptual Overview

The real-time system is built on the concept of 3D video fragments. Differential update schemes are used to exploit the spatio-temporal coherence of the video streams of multiple cameras. These updates are issued by operators such as inserts and deletes, which account for changes in the input video images. We also introduce a novel concept for camera control which dynamically selects the set of relevant cameras for reconstruction, and adapts to the processing load and rendering platform.

We acquire images from multiple calibrated video cameras. The images are processed to segment foreground from background. By means of dynamic camera control we determine a set of active cameras from which we generate 3D point samples as well as a set of supporting cameras delivering additional data to improve the 3D reconstruction. Using inter-frame prediction in image space we generate a stream of differential operators which dynamically update point sample attributes including position or color. We thus avoid to recompute the full 3D representation in each frame. The dynamic point samples are rendered by an efficient point splatting scheme and are composited with a virtual scene. In a final stage we apply deferred operations like visual effects by running them directly on the graphics hardware.

  Home Info

Back to top
© SW, 2004
Last Update: 05.01.2004