Paper Reading #2: MirageTable: Freehand Interaction on a Projected Augmented Reality Tabletop

Paper Reading #2: MirageTable: Freehand Interaction on a Projected Augmented Reality Tabletop

Intro: 

• Title - MirageTable: Freehand Interaction on a Projected Augmented Reality Tabletop
• Reference Information - 
  
  

  ACM Classification: H.5.2 [Information Interfaces and Presentation]: User Interfaces - Graphical user interfaces

  
  Session: Curves & Mirages: Gestures & Interaction with Nonplanar Surfaces
  

  May 5–10, 2012, Austin, Texas, USA
  
  Microsoft Research - Redmond, Washington
  VIMMI / Inesc-ID IST / Technical University of Lisbon - Lisbon, Portugal
  
  
• Author Bios - Hrvoje Benko,  Ricardo Jota, and Andrew D. Wilson

Hrvoje Benko

Hrvoje Benko - He is a researcher in Natural Interaction Research group at Microsoft Research. His research is on novel surface computing technologies and their impact on human-computer interaction. Prior to working at Microsoft, he received his PhD at Columbia University, working on augmented reality projects that combine immersive experiences with interactive tabletops. His projects explore the space where the flat digital world of surface computing meets the curved, physical, 3D space we live in.

Andrew D. Wilson - Also works at Microsoft Research. He has an impressive 55 publications (1,034 citations) from 1995-2012, so is the veteran of the team.  

Ricardo Jota - Works at Microsoft Research, but also at VIMMI / Inesc-ID IST / Technical University of Lisbon in Lisbon, Portugal. He has 10 publications from 2003-2012.

Summary: 

MirageTable is a curved projection-based augmented reality system, which digitizes any object on the surface, presenting correct perspective views accounting for real objects and supporting freehand physics-based interactions. (YouTube link). It focuses on the input of users, instead of the typical augmented reality setups these days that are output oriented. The user can interact with objects without having to wear any additional trackers, gloves, or gear. By simply placing an object on the screen, you can copy it and have a digital copy to interact with. The user's head is also tracked so that the 3D perspective is always correct. Freehand interaction with these virtual objects is much the same way as interaction with their real world object counterparts.

Related Work Not Referenced in the Paper:

1. Projector-Based Location Discovery and Tracking -Johnny Chung Lee 
2.  Spatial Augmented Reality - Oliver Bimber, Ramesh Raskar 
3.  Marker tracking and HMD calibration for a video-based augmented reality conferencing system - Kato, H. Fac. of Inf. Sci., Hiroshima City Univ. Billinghurst, M.
4. The Visual Computing of Projector-Camera Systems - 
5. Oliver Bimber, 
6. Daisuke Iwai
7. , 
8. Gordon Wetzstein, 
9. Anselm Grundhöfer
10. Making one object look like another: controlling appearance using a projector-camera system - Peri, H.;  Nayar, S.K.;  Belhumeur, P.N. 
11.  iLamps: geometrically aware and self-configuring projectors - Ramesh Raskar, Jeroen van Baar, Paul Beardsley, Thomas Willwacher, Srinivas Rao, and Clifton Forlines 
12.  Embedding imperceptible patterns into projected images for simultaneous acquisition and display - Cotting, D.;Naef, M. ;  Gross, M. ;  Fuchs, H. 
13.  A Multi-Camera Method for 3D Digitization of Dynamic, Real-World Events - Peter Rander 
14.  Real-Time 3D Model Acquisition - Szymon Rusinkiewicz, Olaf Hall-Holt, Marc Levoy
15. Telepresence: Integrating Shared Task and Person Spaces - William A. S. Buxton

The work these authors did has been demonstrated independently in the past (i.e. the office of the future, the perceptive workbench, depth sensing video cameras), but MirageTable demonstrates how integrating them together creates a compelling augmented reality experience. MirageTable uses these other works to create 3 new advances to the area of study. 

1. system design and implementation
2. three prototype applications (i.e. 3D video conferencing)
3. user study on 3D perception and image quality

So, I would not say this work is novel, but it is quite an advancement on previous works in the similar area. The authors did do a good job of pointing out their related work as well, as they reference most similar work in their areas of projection based augmented reality solutions and the use of depth sensing cameras for input.

Evaluation:

The authors' main evaluations of this system were done with projective texturing quality. The first evaluation was of the effect of projection surface on image quality. This evaluation involved measuring the perceived distance of a virtual object when it was projected on varying surfaces -- both color and depth. This was quantified using the Root Mean Square difference between the image on a white background (base) to the image on the varying backgrounds (quantitative, objective). The second evaluation was of effect of projection surface on depth perception. This study was done by having users rate the depth of a sphere floating above the table of various surfaces and colors. This gave a quantitative but subjective evaluation of the project. The users also ranked their most preferred surface to work on for another quantitative subjective evaluation. These evaluations mainly focused on the projective texturing quality and so did not test the whole system of its usability, but their evaluation was a very important part of research in the area of work.

Discussion: 

I think the work is a very important step in the advancement of projected augmented reality systems. It does a good job of combining and advancing previous works in the area. I think it does a good job of evaluating the system in the areas that the authors were unsure about. The system as a whole was not tested for user experience and usability, but the area of evaluation was crucial for the system to work. I do not think this was a novel idea in that it is based on combining previous works, but it is a very great advancement. The practical use is hard to see at this point in most of its functions, but the 3D video conferencing is a very impressive idea. The idea of sharing virtual objects on a virtual desk is quite influential, and I see this being something this project creates.