Object-space Compression

We developed a system capable of recording, processing, and playing 3D video from multiple points of view. An off-line processing stage converts the previously recorded 2D sequences into a time-varying three-dimensional hierarchical point-based data structure and stores this data to disk. A typical sequence - where the average number of point samples per frame is of 45k - is encoded at less than 7 megabit per second at a frame rate of 8.5 frames per second. The 3D Video Player decodes and renders 3D videos from hard-disk in real-time, providing the well-known interaction features like variable-speed forward and reverse, and slow motion. 3D video playback can be enhanced with novel 3D video effects such as freeze-and-rotate and arbitrary scaling. The player builds upon point-based rendering techniques and is thus capable of rendering high-quality images in real-time.

Currently we use PRk-trees as hierarchical point data structure. Since we do not yet exploit inter-frame coherence in object-space, every 3D video frame is represented by an independent PRk-tree. After 3D reconstruction, we encode the connectivity of the PRk-tree and the point attributes, e.g. color values and surface normals, into separate streams. The breadth-first traversal of the tree enables progressively decodable streams. From the connectivity information of the PRk-tree, the 3D point positions can be approximated during decoding. In most cases however, an additional refinement of the point positions is necessary and we therefore also encode error vectors of the approximated and the original point positions.