Mixed-Order Compositing for 3D Paintings
I. Baran, J. Schmid , T. Siegrist , M. Gross, R. SumnerProceedings of ACM SIGGRAPH Asia (Hong Kong, China, December 12-15, 2011), ACM Transactions on Graphics, vol. 30, no. 6, pp. 132:1--132:6
Abstract
We present a method for rendering 3D paintings by compositing brush strokes embedded in space. The challenge in compositing 3D brush strokes is reconciling conflicts between their $z$-order in 3D and the order in which the strokes were painted, while maintaining temporal and spatial coherence. Our algorithm smoothly transitions between compositing closer strokes over those farther away and compositing strokes painted later over those painted earlier. It is efficient, running in O(nlogn) time, and simple to implement. We demonstrate its effectiveness on a variety of 3D paintings.Overview
Traditional 2D painting techniques and digital 2D painting systems provide an artist with a lot of expressive freedom in the creation of a painting, allowing a wide range of styles. Translating this freedom into 3D so that paintings can be viewed from multiple angles has been a research challenge for many years. One approach to achieve this goal is to embed individual brush strokes in 3D in a parametric form, for example with a stroke path and a brush stamp texture that is to be rendered in 2D. In this kind of approach, scenes are rendered by projecting each brush stroke onto the current view plane and rasterizing it as one or more fragments for every pixel that the stroke overlaps. This rendering method exposes a technical dilemma about the order in which the fragments should be composited. In the 2D painting metaphor, when the artist places a new paint stroke, it obscures all previous paint strokes that it overlaps. Such behavior is achieved by compositing in stroke order. From a 3D point of view, however, strokes that are closer to the viewer should obscure those that are farther away, which amounts to compositing in depth order. Compositing purely in stroke order negates much of the benefit of 3D painting, as the sense of tangible objects is lost when the view is changed. Compositing purely in depth order, on the other hand, leads to Z-fighting, precluding the artist from painting over existing strokes, and thus ignores an important part of the 2D painting metaphor.
The inventors of the 3D painting system Deep Canvas articulated the desire for mixed-order compositing: fragments that, in the artist's mind, belong on the same surface should be composited in stroke order, while those that belong on different surfaces should be composited in depth order. Unfortunately, assigning each fragment to a specific surface is often impossible: the stroke that generated the fragment may span several surfaces, may self-occlude, or may not even conform to a surface at all. The guideline Deep Canvas adopts is therefore to choose the appropriate ordering based on a depth tolerance d: fragments whose depths are within d of each other are assumed to lie on the same surface and composited in stroke order, but fragments that are farther apart are composited in depth order.
In this work, we formalize this idea for the first time. We state the requirements for compositing order and temporal coherence as four properties, which a mixed-order compositing function must satisfy. We discuss the ways in which existing solutions fail to satisfy one or more of these properties and describe the resulting artifacts. We design a function that does satisfy all four properties and show how to compute it efficiently.
