Smart Displays

Daniel Cotting, Markus Gross

Many users of tele-immersion systems or CAVE™-like environments have dreamed of having more compelling displays in their own offices. Usually, computer and home entertainment screens are fixed in size, position and perspective. Due to these limitations, users are required to adapt to given setups, instead of the display systems continuously accommodating the users’ needs. Designing smart display systems, which can be activated, redirected and interacted with anytime and anywhere, is one of the core objectives of our work. The two key ingredients of such pervasive display technologies are projectors for image generation and cameras for calibration, 3D scene acquisition, tracking, and interaction with humans or sensor-enabled objects. In practice, the seamless and robust combination of multiple projection devices and cameras poses numerous technical difficulties and introduces many interesting challenges. Techniques to project screens at arbitrary locations need to be developed, and the systems need to recalibrate and recover the display geometry continuously in case an undistorted view is desired. Furthermore, novel forms of multimodal interaction beyond mouse and keyboard need to be devised in order to use the new display opportunities to full capacity. Ideally, this has to be achieved without encumbrances of tracking devices on heads and hands. Although steady progress in research has been made to counter the restrictions of traditional computer displays, most attempts have not focused on flexibility and elegance of the systems.

Relying on a novel technique of per-pixel light control exploiting the micro-mirror modulation pattern used by Digital Light Processing™ (DLP™) projectors, our research aims at creating a complete, intuitive framework for realizing smart displays with the help of scalable projections. Simple optical I/O-bricks containing cameras and projectors are combined in a flexible way to achieve increasingly immersive, interactive environments, where dynamic changes are continuously accounted for by a recalibration procedure based on imperceptible per-pixel light control. Using encumbrance-free tracking and user-modeling, we are developing intuitive interaction metaphors with standard applications, replacing traditional screens, keyboards and mice in an appealing and effective way.

• D. Cotting, M. Gross, Interactive Visual Workspaces with Dynamic Foveal Areas and Adaptive Composite Interfaces, Proceedings of Eurographics (Prague, Czech Republic, September 3-7, 2007), Computer Graphics Forum, vol. 26, no. 3, pp. 685-694
  [Abstract] [PDF] [Video]
• D. Cotting, M. Waschbüsch, M. Duller, M. Gross, WinSGL: Synchronizing Displays in Parallel Graphics using Cost-Effective Software Genlocking, Parallel Computing, Elsevier, vol. 33, no. 6, 2007, pp. 420-437
  [Abstract] [PDF] [Video]
• D. Cotting, M. Gross, Interactive Environment-Aware Display Bubbles, Proceedings of The ACM Symposium on User Interface Software and Technology (UIST) (Montreux, Switzerland, October 15-18, 2006), pp. 245-254
  [Abstract] [PDF] [Video]
• M. Waschbüsch, D. Cotting, M. Duller, M. Gross, WinSGL: Software Genlocking for Cost-Effective Display Synchronization under Microsoft Windows, Proceedings of the 6th Eurographics Symposium on Parallel Graphics and Visualization, 2006, pp. 111-118
  [Abstract] [PDF] [Video]
• D. Cotting, R. Ziegler, M. Gross, H. Fuchs, Adaptive Instant Displays: Continuously Calibrated Projections Using Per-Pixel Light Control, Proceedings of Eurographics '05, pp. 705-714 (Dublin, Ireland, Aug 29th - Sep 2nd, 2005)
  [Abstract] [PDF] [Video]
• D. Cotting, M. Naef, M. Gross, H. Fuchs, Imperceptible Patterns for Reliable Acquisition of Mixed Reality Environments, Proceedings of The International Workshop on Image Analysis for Multimedia Interactive Services 2005
  [Abstract] [PDF] [Video]
• D. Cotting, M. Naef, M. Gross, H. Fuchs, Embedding Imperceptible Patterns into Projected Images for Simultaneous Acquisition and Display, Proceedings of the International Symposium on Mixed and Augmented Reality 2004, p.100-109 (ISMAR 2004, November 2-5, Washington DC, USA, 2004)
  [Abstract] [PDF] [Video]