RegisterGet new password
Welcome to ISMAR 2013!
header
Home

2013 ISMAR All Papers

Viewing and Rendering
Session : 
You Look Great Today
Date & Time : October 02 10:30 am - 12:00 pm
Location : H2-02 Hetzel Building Main Lecture Theatre
Chair : Steven Feiner
Papers : 
TVGC: View Management of Projected Labels on Non-Planar and Textured Surfaces
Authors:Daisuke Iwai, Tatsunori Yabiki, Kosuke Sato
Abstract :
This paper presents a new label layout technique for projection-based augmented reality (AR) that determines the placement of each label directly projected onto an associated physical object with a surface that is normally inappropriate for projection (i.e., non-planar and textured). Central to our technique is a new legibility estimation method that evaluates how easily people can read projected characters from arbitrary viewpoints. The estimation method relies on the results of a psychophysical study that we conducted to investigate the legibility of projected characters on various types of surfaces that deform their shapes, decrease their contrasts, or cast shadows on them. Our technique computes a label layout by minimizing the energy function using a genetic algorithm (GA). The terms in the function quantitatively evaluate different aspects of the layout quality. Conventional label layout solvers evaluate anchor regions and leader lines. In addition to these evaluations, we design our energy function to deal with the following unique factors, which are inherent in projection-based AR applications: the estimated legibility value and the disconnection of the projected leader line. The results of our subjective experiment showed that the proposed technique could significantly improve the projected label layout.
Approximated User-Perspective Rendering in Tablet-based Augmented Reality
Authors:Makoto Tomioka, Sei Ikeda, Kosuke Sato
Abstract :
This study addresses the problem of geometric consistency between displayed images and real scenes in augmented reality using a video see-through hand-held display or tablet. To solve this problem, we present approximated user-perspective images rendered by homography transformation of camera images. Homography approximation has major advantages not only in terms of computational costs, but also in the quality of image rendering. However, it can lead to an inconsistency between the real image and virtual objects. This study also introduces a variety of rendering methods for virtual objects and discusses the differences between them. We implemented two prototypes and designed three types of user studies on matching tasks between real scenes and displayed images. We have confirmed that the proposed method works in real time on an off-the-shelf tablet. Our pilot tests show the potential to improve users's visibility, even in real environments, by using our method.
Adaptive Ghosted Views for Augmented Reality
Authors:Denis Kalkofen, Eduardo Veas, Stefanie Zollmann, Markus Steinberger, Dieter Schmalstieg
Abstract :
In Augmented Reality (AR), ghosted views allow a viewer to explore hidden structure within the real-world environment. A body of previous work has explored which features are suitable to support the structural interplay between occluding and occluded elements. However, the dynamics of AR environments pose serious challenges to the presentation of ghosted views. While a model of the real world may help determine distinctive structural features, changes in appearance or illumination detriment the composition of occluding and occluded structure. In this paper, we present an approach that considers the information value of the scene before and after generating the ghosted view. Hereby, a contrast adjustment of preserved occluding features is calculated, which adaptively varies their visual saliency within the ghosted view visualization. This allows us to not only preserve important features, but to also support their prominence after revealing occluded structure, thus achieving a positive effect on the perception of ghosted views.
Diminished Reality using Appearance and 3D Geometry of Internet Photo Collections
Authors:Zhuwen Li, Yuxi Wang, Jiaming Guo, Loong-Fah Cheong, Steven Zhou
Abstract :
This paper presents a new system level framework for Diminished Reality, leveraging for the first time both the appearance and 3D information provided by large photo collections on the Internet. Recent computer vision techniques have made it possible to automatically reconstruct 3-D structure-from-motion points from large and unordered photo collections. Using these point clouds and a prior provided by GPS, reasonably accurate 6 degree of freedom camera poses can be obtained, thus allowing localization. Once the camera (and hence the user) is correctly localized, photos depicting scenes visible from the user's viewpoint can be used to remove unwanted objects indicated by the user in the video sequences. Existing methods based on texture synthesis bring undesirable artifacts and video inconsistency when the background is heterogeneous; the task is rendered even harder for these methods when the background contains complex structures. On the other hand, methods based on plane warping fail when the background has arbitrary shape. Unlike these methods, our algorithm copes with these problems by making use of internet photos, registering them in 3D space and obtaining the 3D scene structure in an offline process. We carefully design the various components during the online phase so as to meet both speed and quality requirements of the task. Experiments on real data collected demonstrate the superiority of our system.
Computational Augmented Reality Eyeglasses
Authors:Andrew Maimone, Henry Fuchs
Abstract :
In this paper we discuss the design of an optical see-through head-worn display supporting a wide field of view, selective occlusion, and multiple simultaneous focal depths that can be constructed in a compact eyeglasses-like form factor. Building on recent developments in multilayer desktop 3D displays, our approach requires no reflective, refractive, or diffractive components, but instead relies on a set of optimized patterns to produce a focused image when displayed on a stack of spatial light modulators positioned closer than the eye accommodation distance. We extend existing multilayer display ray constraint and optimization formulations while also purposing the spatial light modulators both as a display and as a selective occlusion mask. We verify the design on an experimental prototype and discuss challenges to building a practical display.
Augmented Interfaces
Session : 
Augmented Interfaces
Date & Time : October 03 09:30 am - 10:30 am
Location : H2-02 Hetzel Building Main Lecture Theatre
Chair : Jay Bolter
Papers : 
Archiving the `Fabric of Digital Life"
Authors:Isabel Pedersen, Jeremiah Baarbé
Abstract :
This paper describes the process for establishing the `Fabric of Digital Life' archive, which is dedicated to the study of wearable media and augmented reality inventions. Fabric of Digital Life features the development of an open repository online research archive using CollectiveAccess software that tracks, catalogues, and in some cases, stores artifacts that imply future invention. The collection of this corpus will enable the analysis of inventors's writings and discourses, news articles, images, videos, documents, commercial ventures, artifacts, and events that instantiate the discourse of emerging inventions relating to reality-shifting, digital life, and digital culture. Inventions do not emerge from the hands of solo inventors, they emerge within a vast context of overlapping texts that communicate the motives of everyday people as much as they do the motives of inventors.
Kaidan: An Outdoor AR Puzzle Adventure Game
Authors:Yong Li, Dongdong Weng, Haiyun Zhou, Jianying Hao, Lu Zhao
Abstract :
Puzzle adventure is a widely popular genre of entertainment games. This paper presents a wide-field outdoor puzzle game system, Kaidan, which is based on augmented reality (AR) technology. In this system, users can move freely throughout a college campus, and are prompted by mobile device instructions to locate clues, solve puzzles and ultimately complete the game. The puzzle clues in the system are delivered through AR technology, GPS position navigation and sensor-based interactive technology, and are also associated with the environment of real scene, which together enhance the level of immersion of the game. The paper also describes a user study, based on actual feedback data from users. Finally, the augmented reality technology enhancing users's experience of the outdoor puzzle game is analyzed in detail.
User-centered Perspectives for Automotive Augmented Reality
Authors:Victor Ng-Thow-Hing, Karlin Bark, Lee Beckwith, Cuong Tran, Rishabh Bhandari, Srinath Sridhar
Abstract :
Augmented reality (AR) in automobiles has the potential to significantly alter the driver's user experience. Prototypes developed in academia and industry demonstrate a range of applications from advanced driver assist systems to location-based information services. A user-centered process for creating and evaluating designs for AR displays in automobiles helps to explore what collaborative role AR should serve between the technologies of the automobile and the driver. In particular, we consider the nature of this role along three important perspectives: understanding human perception, understanding distraction and understanding human behavior. We argue that AR applications should focus solely on tasks that involve the immediate local driving environment and not secondary task spaces to minimize driver distraction. Consistent depth cues should be supported by the technology to aid proper distance judgement. Driving aids supporting situation awareness should be designed with knowledge of current and future states of road users, while focusing on specific problems. Designs must also take into account behavioral phenomena such as risk compensation, inattentional blindness and an over-reliance on augmented technology in driving decisions.
Interactive Modeling
Session : 
Shaping The World
Date & Time : October 03 01:30 pm - 03:00 pm
Location : H2-02 Hetzel Building Main Lecture Theatre
Chair : Gregory Welch
Papers : 
Interactive Syntactic Modeling With a Single-Point Laser Range Finder and Camera
Authors:Thanh Nguyen, Raphael Grasset, Dieter Schmalstieg, Gerhard Reitmayr
Abstract :
In-situ 3D Modeling becomes increasingly prominent in current Augmented Reality research, particularly for mobile scenarios. However, real-time performance and qualitative modeling remain highly challenging. In this work, we propose a new interactive 3D modeling approach for indoor environments, combining an assistive user interface and constrained reconstruction with a device consisting of a single-point laser range finder and a camera. Using our system, a user pans around capturing a panorama of the environment, while simultaneously measuring the distance to a single point per frame. An automatic detection process estimates planes from these sparse 3D measurements. The user can highlight specific geometric features in the environment, such as 2- or 3-way corners, with simple gestures, adding more 3D points to the estimation. The segmented planes are refined using a constrained optimization, enforcing orthogonality and parallel constraints as well as minimizing the number of planes used in the reconstruction. Finally a volumetric space-carving approach determines the geometry of the environment. Our reconstruction approach can output highly accurate models built only from simple, clean geometry. To examine the quantitative performance of our approach, we run evaluations on both synthetic and real data.
Real-time Modeling and Tracking Manual Workflows from First-Person Vision
Authors:Nils Petersen, Alain Pagani, Didier Stricker
Abstract :
Recognizing previously observed actions in video sequences can lead to Augmented Reality manuals that (1) automatically follow the progress of the user and (2) can be created from video examples of the workflow. Modeling is challenging, as the environment is susceptible to change drastically due to user interaction and camera motion may not provide sufficient translation to robustly estimate geometry. We propose a piecewise homographic transform that projects the given video material onto a series of distinct planar subsets of the scene. These subsets are selected by segmenting the largest image region that is consistent with a homographic model and contains a given region of interest. We are then able to model the state of the environment and user actions using simple 2D region descriptors. The model elegantly handles estimation errors due to incomplete observation and is robust towards occlusions, e.g. due to the user's hands. We demonstrate the effectiveness of our approach quantitatively and compare it to the current state of the art. Further, we show how we apply the approach to visualize automatically assessed correctness criteria during run-time.
Scanning and Tracking Dynamic Objects with Commodity Depth Cameras
Authors:Mingsong Dou, Henry Fuchs, Jan-Michael Frahm
Abstract :
The 3D data collected using state-of-art algorithms often suffers from various problems, such as incompletion and inaccuracy. Using temporal information has been proven effective for improving the reconstruction quality; for example, KinectFusion shows significant improvements for static scenes. In this work, we present a system that uses commodity depth and color cameras, such as Microsoft Kinects, to fuse the 3D data captured over time for dynamic objects to build a complete and accurate model, and then tracks the model to match later observations. The key ingredients of our system include a nonrigid matching algorithm that aligns 3D observations of dynamic objects by using both geometry and texture measurements, and a volumetric fusion algorithm that fuses noisy 3D data. We demonstrate that the quality of the model improves dramatically by fusing a sequence of noisy and incomplete depth data of human and that by deforming this fused model to later observations, noise-and-hole-free 3D models are generated for the human moving freely.
Simultaneous 3D Tracking and Reconstruction on a Mobile Phone
Authors:Victor Adrian Prisacariu, Olaf Kahler, David Murray, Ian Reid
Abstract :
A novel framework for joint monocular 3D tracking and reconstruction is described that can handle untextured objects, occlusions, motion blur, changing background and imperfect lighting, and that can run at frame rate on a mobile phone. The method runs in parallel (i) level set based pose estimation and (ii) continuous max flow based shape optimisation. By avoiding a global computation of distance transforms typically used in level set methods, tracking rates here exceed 100Hz and 20Hz on a desktop and mobile phone, respectively, without needing a GPU. Tracking ambiguities are reduced by augmenting orientation information from the phone's inertial sensor. Reconstruction involves probabilistic integration of the 2D image statistics from keyframes into a 3D volume. Per-voxel posteriors are used instead of the standard likelihoods, giving increased accuracy and robustness. Shape coherency and compactness is then imposed using a total variational approach solved using globally optimal continuous max flow.
MonoFusion: Real-time 3D Reconstruction of Small Scenes with a Single Web Camera
Authors:Vivek Pradeep, Christoph Rhemann, Shahram Izadi, Michael Bleyer, Steven Bathiche, Christopher Zach
Abstract :
MonoFusion allows a user to build dense 3D reconstructions of their environment in real-time, utilizing only a single, off-the-shelf web-camera as the input sensor. The camera could be one already available in tablet or a phone, or a peripheral web camera. No additional input hardware is required. This removes the need for power intensive active sensors that do not work robustly in natural outdoor lighting. Using the input stream of the camera we first estimate the 6DoF pose of the camera using a sparse tracking method. These poses are then used for efficient dense wide-baseline stereo matching between the input frame and a key frame (extracted previously). The resulting dense depth maps are directly fused into a voxel-based implicit model (using a computationally inexpensive method) and surfaces are extracted per frame. The system is able to recover from tracking failures as well as filter out geometrically inconsistent noise from the 3D reconstruction. Furthermore, compared to existing approaches, our system does not require maintaining a compute and memory intensive cost volume and avoids using expensive global optimization methods for fusion. This paper details the algorithmic components that make up our system and a GPU implementation of our approach. Qualitative results demonstrate high quality reconstructions even visually comparable to active depth sensor-based systems such as KinectFusion.
User Study and Performance
Session : 
Make It Easy
Date & Time : October 03 11:00 am - 12:30 pm
Location : H2-02 Hetzel Building Main Lecture Theatre
Chair : Andreas Duenser
Papers : 
Effects of an In-Car Augmented Reality System on Improving Safety of Younger and Older Drivers
Authors:Wai-Tat Fu, John Gasper, Seong-Whan Kim
Abstract :
We designed and tested the effects of an in-car augmented reality system (ARS) on younger and older drivers, with and without a secondary distraction task. When potential danger is detected, the ARS alerts the driver by progressively indicating the time to collision to the lead vehicle as well as merging vehicles from side lanes by an AR display that overlaps with the lead or merging vehicles. We tested the ARS with younger (18-30) and older (65-75) drivers in a high-fidelity driving simulator. Results showed that the ARS could significantly reduce collisions caused by hazard events such as sudden slowing of the lead vehicle or merging of vehicles from sides lanes. Consistent with previous results, older drivers, despite age-related decline in cognitive and motor abilities, could leverage their driving experience to avoid forward collisions with the lead vehicle as much as younger drivers. However, older drivers were poorer in avoiding collisions caused by sudden merging events than younger drivers. The ARS was found to be most useful in helping older adults to avoid collision caused by sudden hazard events, especially with the presence of a distraction task. The ARS was also more effective for older than younger drivers to encourage a safe driving distance with the lead vehicle. Interestingly, there seemed to be differential effects of the ARS on the general driving behavior of younger and older drivers. While older drivers in general became more careful and safer in how they drive with the ARS, younger drivers seemed to rely on the ARS to alert them to potential hazard events without adopting safer driving behavior.
Through the Looking Glass: Pretend Play for Children with Autism
Authors:Zhen Bai, Alan Blackwell, George Coulouris
Abstract :
Lack of spontaneous pretend play is an early diagnostic indicator of autism spectrum conditions (ASC) along with impaired communication and social interaction. In a previous ISMAR poster [2] we proposed an Augmented Reality (AR) system to encourage pretend play, based on an analogy between imaginative interpretation of physical objects (pretense) and the superimposition of virtual content on the physical world in AR. This paper reports an empirical experiment evaluating that proposal, involving children between the ages of 4 and 7 who have been diagnosed with ASC. Results find significantly more pretend play, and higher engagement, using the AR system by comparison to a non-augmented condition. We also discuss usability issues and design implications for AR systems that aim to support children with ASC and other pervasive developmental disorders.
Improving Procedural Task Performance with Augmented Reality Annotations
Authors:Michael Marner, Andrew Irlitti, Bruce Thomas
Abstract :
This paper presents results of a study measuring user performance in a procedural task using Spatial Augmented Reality (SAR). The task required participants to press sequences of buttons on two control panel designs in the correct order. Instructions for the task were shown either on a computer monitor, or projected directly onto the control panels. This work was motivated by discrepancies between the expectations from AR proponents and experimental findings. AR is often promoted as a way of improving user performance and understanding. With notable exceptions however, experimental results do not confirm these expectations. Reasons cited for results include limitations of current display technologies and misregistration caused by tracking and calibration errors. Our experiment utilizes SAR to remove these effects. Our results show that augmented annotations lead to significantly faster task completion speed, fewer errors, and reduced head movement, when compared to monitor based instructions. Subjectively, our results show augmented annotations are preferred by users.
Going With the Flow: Modifying Self-Motion Perception with Computer-Mediated Optic Flow
Authors:Gerd Bruder, Phil Wieland, Benjamin Bolte, Markus Lappe, Frank Steinicke
Abstract :
One major benefit of wearable computers is that users can naturally move and explore computer-mediated realities. However, researchers often observe that users' space and motion perception severely differ in such environments compared to the real world, an effect that is often attributed to slight discrepancies in sensory cues, for instance, caused by tracking inaccuracy or system latency. This is particularly true for virtual reality (VR), but such conflicts are also inherent to augmented reality (AR) technologies. Although, head-worn displays will become more and more available soon, the effects on motion perception have rarely been studied, and techniques to modify self-motion in AR environments have not been leveraged so far. In this paper we introduce the concept of computer-mediated optic flow, and analyze its effects on self-motion perception in AR environments. First, we introduce different techniques to modify optic flow patterns and velocity. We present a psychophysical experiment which reveals differences in self-motion perception with a video see-through head-worn display compared to the real-world viewing condition. We show that computer-mediated optic flow has the potential to make a user perceive self-motion as faster or slower than it actually is, and we discuss its potential for future AR setups.
Mobile Interactive Hologram Verification
Authors:Andreas Hartl, Jens Grubert, Dieter Schmalstieg, Gerhard Reitmayr
Abstract :
Verification of paper documents is an important part of checking a person's s identity, authorization for access or simply establishing a trusted currency. Many documents such as passports or paper bills include holograms or other view-dependent elements that are difficult to forge and therefore are used to verify the genuineness of that document. View-dependent elements change their appearance based both on viewing direction and dominant light sources, thus it requires special knowledge and training to accurately distinguish original elements from forgeries. We present an interactive application for mobile devices that integrates the recognition of the documents with the interactive verification of view-dependent elements. The system recognizes and tracks the paper document, provides user guidance for view alignment and presents a stored image of the element's s appearance depending on the current view of the document also recording user decisions. We describe how to model and capture the underlying spatially varying BRDF representation of view-dependent elements. Furthermore, we evaluate this approach within a user study and demonstrate that such a setup captures images that are recognizable and that can be correctly verified.
New places, new views
Session : 
New places, new views
Date & Time : October 04 09:30 am - 10:30 am
Location : H2-02 Hetzel Building Main Lecture Theatre
Chair : Maria Engberg
Papers : 
Views, Alignment and Incongruity in Indirect Augmented Reality
Authors:Gunnar Liestøl, Andrew Morrison
Abstract :
Alignment between the real and the virtual has been a defining quality of mixed and augmented reality. With the emergence of Indirect Augmented Reality the problem of alignment is no longer primarily concerned with the relationship between a live video feed and a 3D graphics layer at the level of the screen, but with the relationship between the visual information on the display and the real world perspective outside the display of the device. Experiments show that users easily connect the perspective into the 3D virtual environment on the full screen with their parallel perspective in the real world. It also turns out that although congruence and alignment between the two perspectives is fundamental to the user experience, in certain contexts it may be transcended. This paper describes and discusses applications of Indirect Augmented Reality where we explore how the discrepancy between the virtual and the real perspectives in a variety of ways can be used to improve the user experience. We call these features views. The views will be exemplified with several Indirect Augmented Reality applications: reconstructions of Augustus' Forum and the Republican Forum in Rome and a preconstruction of the planned National Museum in Oslo. The applications have been tested with users on location, and their feedback and evaluation is included in the discussion. Finally, we relate the experiential value of the views to some epistemological and pedagogical perspectives.
AntarcticAR: An Outdoor AR Experience of a Virtual Tour to Antarctica
Authors:Gun Lee, Andreas Duenser, Alaeddin Nassani, Mark Billinghurst
Abstract :
In this paper we introduce AntarcticAR, a mobile outdoor Augmented Reality (AR) application that provides a virtual tour of Antarctica. Outdoor AR interfaces have often been used as a navigational tool, providing information related to the real world environment where the user is located in. In contrast, this research explores using outdoor AR to provide a virtual tour that has little relationship with the user's s real environment. The AntarcticAR application allows people to virtually visit places of interest in Antarctica or follow historic expeditions to the South Pole. The paper describes the design and implementation of the application, and reports on a user study with members of the public. Users felt that the application provided a compelling virtual tour, and that the outdoor AR technology especially enriched the experience.
The Journey of a White Rhinoceros: Sculpture Augmentation for Gallery Exhibition
Authors:Taegyu Kim, Jay Jang, Joonsuk Park, Jun Park
Abstract :
Rhinoceros are huge and strong but now almost extinct. `White Rhinoceros' is a sculpture that was exhibited in 1993 and now placed in storage. Twenty years after `White Rhinoceros' was first displayed, a team of animation artists and engineers revived this outdated sculpture endowing new meanings, stories, and technologies. `The Journey of a White Rhinoceros', using Augmented Reality technologies, delivered messages to the gallery visitors on the greed and blindness of human being, the ecosystem of the earth, and imminent crises of natural disasters. Two projectors projected videos and animated special effects on the sides of the White Rhinoceros, and one movable large display was employed to show the augmented views on the rhinoceros as a background. `The Journey of a White Rhinoceros' was displayed for four days at a gallery attracting interests of many visitors, especially of young generations.
Tracking and Registration
Session : 
Track Me If You Can
Date & Time : October 04 04:00 pm - 05:15 pm
Location : H2-02 Hetzel Building Main Lecture Theatre
Chair : Tobias Hoellerer
Papers : 
Image-guided Simulation of Heterogeneous Tissue Deformation For Augmented Reality during Hepatic Surgery
Authors:Nazim Haouchine, Jeremie Dequidt, Igor Peterlik, Erwan Kerrien, Marie-Odile Berger, Stephane Cotin
Abstract :
This paper presents a method for real-time augmentation of vascular network and tumors during minimally invasive liver surgery. Internal structures computed from pre-operative CT scans can be overlaid onto the laparoscopic view for surgery guidance. Compared to state-of-the-art methods, our method uses a real-time biomechanical model to compute a volumetric displacement field from partial three-dimensional liver surface motion. This permits to properly handle the motion of internal structures even in the case of anisotropic or heterogeneous tissues, as it is the case for the liver and many anatomical structures. Real-time augmentation results are presented on in vivo and phantom data and illustrate the benefits of such an approach for minimally invasive surgery.
Augmented Reality Binoculars
Authors:Taragay Oskiper, Mikhail Sizintsev, Vlad Branzoi, Supun Samarasekera, Rakesh Kumar
Abstract :
In this paper we present an augmented reality binocular system to allow long range high precision augmentation of live telescopic imagery with aerial and terrain based synthetic objects, vehicles, people and effects. The inserted objects must appear stable in the display and must not jitter and drift as the user pans around and examines the scene with the binoculars. The design of the system is based on using two different cameras with wide field of view, and narrow field of view lenses enclosed in a binocular shaped shell. Using the wide field of view gives us context and enables us to recover the 3D location and orientation of the binoculars much more robustly, whereas the narrow field of view is used for the actual augmentation as well as to increase precision in tracking. We present our navigation algorithm that uses the two cameras in combination with an IMU and GPS in an Extended Kalman Filter (EKF) and provides jitter free, robust and real-time pose estimation for precise augmentation. We have demonstrated successful use of our system as part of a live simulated training system for observer training, in which fixed and rotary wing aircrafts, ground vehicles, and weapon effects are combined with real world scenes.
Handling Pure Camera Rotation in Keyframe-Based SLAM
Authors:Christian Pirchheim, Gerhard Reitmayr, Dieter Schmalstieg
Abstract :
Handling degenerate rotation-only camera motion is a challenge for keyframe-based simultaneous localization and mapping with six degrees of freedom. Existing systems usually filter corresponding keyframe candidates, resulting in mapping starvation and tracking failure. We propose to employ these otherwise discarded keyframes to build up local panoramic maps registered in the 3D map. Thus, the system is able to maintain tracking during rotational camera motions. Additionally, we seek to actively associate panoramic and 3D map data for improved 3D mapping through the triangulation of more new 3D map features. We demonstrate the efficacy of our approach in several evaluations that show how the combined system handles rotation only camera motion while creating larger and denser maps compared to a standard SLAM system.
Robust Monocular SLAM in Dynamic Environments
Authors:Wei Tan, Haomin Liu, Zilong Dong, Guofeng Zhang, Hujun Bao
Abstract :
We present a novel real-time monocular SLAM system which can robustly work in dynamic environments. Different to the traditional methods, our system allows parts of the scene to be dynamic or the whole scene to gradually change. The key contribution is that we propose a novel online keyframe representation and updating method to adaptively model the dynamic environments, where the appearance or structure changes can be effectively detected and handled. We reliably detect the changed features by projecting them from the keyframes to current frame for appearance and structure comparison. The appearance change due to occlusions also can be reliably detected and handled. The keyframes with large changed areas will be replaced by newly selected frames. In addition, we propose a novel prior-based adaptive RANSAC algorithm (PARSAC) to efficiently remove outliers even when the inlier ratio is rather low, so that the camera pose can be reliably estimated even in very challenging situations. Experimental results demonstrate that the proposed system can robustly work in dynamic environments and outperforms the state-of-the-art SLAM systems (e.g. PTAM).
Photometric AR
Session : 
Mehr Licht!
Date & Time : October 04 08:30 am - 09:30 am
Location : H2-02 Hetzel Building Main Lecture Theatre
Chair : Dieter Schmalstieg
Papers : 
Delta Light Propagation Volumes for Mixed Reality
Author:Tobias Alexander Franke
Abstract :
Indirect illumination is an important visual cue which has traditionally been neglected in mixed reality applications. We present Delta Light Propagation Volumes, a novel volumetric relighting method for real-time mixed reality applications which allows to simulate the effect of first bounce indirect illumination of synthetic objects onto a real geometry and vice versa. Inspired by Radiance Transfer Fields, we modify Light Propagation Volumes in such a way as to propagate the change in illumination caused by the introduction of a synthetic object into a real scene. This method combines real and virtual light in one representation, provides improved temporal coherence for indirect light compared to previous solutions and implicitly includes smooth shadows.
Differential Irradiance Caching for Fast High-Quality Light Transport Between Virtual and Real Worlds
Authors:Peter Kán, Hannes Kaufmann
Abstract :
Fast and realistic synthesis of real videos with computer generated content has been a challenging problem in computer graphics. It involves computationally expensive light transport calculations. We present a novel and efficient algorithm for diffuse light transport calculation between virtual and real worlds called Differential Irradiance Caching. Our algorithm produces a high-quality result while preserving interactivity and allowing dynamic geometry, materials, lighting, and camera movement. The problem of expensive differential irradiance evaluation is solved by exploiting the spatial coherence in indirect illumination using irradiance caching. We enable multiple bounces of global illumination by using Monte Carlo integration in GPU ray-tracing to evaluate differential irradiance at irradiance cache records in one pass. The combination of ray-tracing and rasterization is used in an extended irradiance cache splatting algorithm to provide a fast GPU-based solution of indirect illumination. Limited information stored in the irradiance splat buffer causes errors for pixels on edges in case of depth of field rendering. We propose a solution to this problem using a reprojection technique to access the irradiance splat buffer. A novel cache miss detection technique is introduced which allows for a linear irradiance cache data structure. We demonstrate the integration of differential irradiance caching into a rendering framework for Mixed Reality applications capable of simulating complex global illumination effects.
3D High Dynamic Range Dense Visual SLAM and Its Application to Real-time Object Re-lighting
Authors:Maxime Meilland, Christian Barat, Andrew Comport
Abstract :
Acquiring High Dynamic Range (HDR) light-fields from several images with different exposures (sensor integration periods) has been widely considered for static camera positions. In this paper a new approach is proposed that enables 3D HDR environment maps to be acquired directly from a dynamic set of images in real-time. In particular a method will be proposed to use an RGB-D camera as a dynamic light-field sensor, based on a dense real-time 3D tracking and mapping approach, that avoids the need for a light-probe or the observation of reflective surfaces. The 6dof pose and dense scene structure will be estimated simultaneously with the observed dynamic range so as to compute the radiance map of the scene and fuse a stream of low dynamic range images (LDR) into an HDR image. This will then be used to create an arbitrary number of virtual omni-directional light-probes that will be placed at the positions where virtual augmented objects will be rendered. In addition, a solution is provided for the problem of automatic shutter variations in visual SLAM. Augmented reality results are provided which demonstrate real-time 3D HDR mapping, virtual light-probe synthesis and light source detection for rendering reflective objects with shadows seamlessly with the real video stream in real-time.
Initialization and Calibration
Session : 
Starting Over
Date & Time : October 04 11:00 am - 12:30 pm
Location : H2-02 Hetzel Building Main Lecture Theatre
Chair : Gudrun Klinker
Papers : 
Behaviour Aware Sensor Fusion: Inferring Registration of Coordinate Systems from User Behaviour
Authors:Anthony Steed, Simon Julier
Abstract :
Within mobile mixed reality experiences, we would like to engage the user's s head and hands for interaction. However, this requires the use of multiple tracking systems. These must be aligned, both as part of initial system setup and to counteract inter-tracking system drift that can accumulate over time. Traditional approaches to alignment use obtrusive procedures that introduce explicit constraints between the different tracking systems. These can be highly disruptive for the user's s experience. In this paper, we propose another type of information which can be exploited to effect alignment: the behaviour of the user. The crucial insight is that user behaviours such as selection through pointing introduce implicit constraints between tracking systems. These constraints can be used as the user continually interacts with the system to infer alignment without the need for disruptive procedures. We call this concept behaviour aware sensor fusion. We introduce two different interaction techniques the redirected pointing technique and the yaw fix technique to illustrate this concept. Pilot experiments show that behaviour aware sensor fusion can increase ease of use and speed of interaction in exemplar mixed-reality interaction tasks.
Single-Shot Extrinsic Calibration of a Generically Configured RGB-D Camera Rig from Scene Constraints
Authors:Jiaolong Yang, Yuchao Dai, Hongdong Li, Henry Gardner, Yunde Jia
Abstract :
With the increasing use of commodity RGB-D cameras for computer vision, robotics, mixed and augmented reality and other areas, it is of significant practical interest to calibrate the relative pose between a depth (D) camera and an RGB camera in these types of setups. In this paper, we propose a new single-shot, correspondence-free method to extrinsically calibrate a generically configured RGB-D camera rig. We formulate the extrinsic calibration problem as one of geometric 2D-3D registration which exploits scene constraints to achieve single-shot extrinsic calibration. Our method first reconstructs sparse point clouds from a single-view 2D image. These sparse point clouds are then registered with dense point clouds from the depth camera. Finally, we directly optimize the warping quality by evaluating scene constraints in 3D point clouds. Our single-shot extrinsic calibration method does not require correspondences across multiple color images or across different modalities and it is more flexible than existing methods. The scene constraints can be very simple and we demonstrate that a scene containing three sheets of paper is sufficient to obtain reliable calibration and with a lower geometric error than existing methods.
A Camera-Based Calibration for Automotive Augmented Reality Head-Up-Displays
Authors:Folker Wientapper, Harald Wuest, Pavel Rojtberg, Dieter Fellner
Abstract :
Using Head-up-Displays (HUD) for Augmented Reality requires to have an accurate internal model of the image generation process, so that 3D content can be visualized perspectively correct from the viewpoint of the user. We present a generic and cost-effective camera-based calibration for an automotive HUD which uses the windshield as a combiner. Our proposed calibration model encompasses the view-independent spatial geometry, i.e. the exact location, orientation and scaling of the virtual plane, and a view-dependent image warping transformation for correcting the distortions caused by the optics and the irregularly curved windshield. View-dependency is achieved by extending the classical polynomial distortion model for cameras and projectors to a generic five-variate mapping with the head position of the viewer as additional input. The calibration involves the capturing of an image sequence from varying viewpoints, while displaying a known target pattern on the HUD. The accurate registration of the camera path is retrieved with state-of-the-art vision-based tracking. As all necessary data is acquired directly from the images, no external tracking equipment needs to be installed. After calibration, the HUD can be used together with a head-tracker to form a head-coupled display which ensures a perspectively correct rendering of any 3D object in vehicle coordinates from a large range of possible viewpoints. We evaluate the accuracy of our model quantitatively and qualitatively.
User-Friendly SLAM Initialization
Authors:Alessandro Mulloni, Mahesh Ramachandran, Daniel Wagner, Gerhard Reitmayr, Raphael Grasset, Serafin Diaz
Abstract :
The development of new Simultaneous Localization and Mapping (SLAM) techniques is quickly advancing in research communities and rapidly transitioning into commercial products. Creating accurate and high-quality SLAM maps relies on a robust initialization process. However, the robustness and usability of SLAM initialization for end-users has often been disregarded. This paper presents and evaluates a novel tracking system for 6DOF pose tracking between a single keyframe and the current camera frame, without any prior scene knowledge. Our system is particularly suitable for SLAM initialization, since it allows 6DOF pose tracking in the intermediate frames before a wide-enough baseline between two keyframes has formed. We investigate how our tracking system can be used to interactively guide users in performing an optimal motion for SLAM initialization. However, our findings from a pilot study indicate that the need for such motion can be completely hidden from the user and outsourced to our tracking system. Results from a second user study show that letting our tracking system create a SLAM map as soon as possible is a viable and usable solution. Our work provides important insight for SLAM systems, showing how our novel tracking system can be integrated with a user interface to support fast, robust and user-friendly SLAM initialization.
Real-Time RGB-D Camera Relocalization
Authors:Ben Glocker, Shahram Izadi, Jamie Shotton, Antonio Criminisi
Abstract :
We introduce an efficient camera relocalization approach which can be easily integrated into real-time 3D reconstruction methods, such as KinectFusion. Our approach makes use of compact encoding of whole image frames which enables both online harvesting of keyframes in tracking mode, and fast retrieval of pose proposals when tracking is lost. The encoding scheme is based on randomized ferns and simple binary feature tests. Each fern generates a small block code, and the concatenation of codes yields a compact representation of each camera frame. Based on those representations we introduce an efficient frame dissimilarity measure which is defined via the block-wise hamming distance (BlockHD). We illustrate how BlockHDs between a query frame and a large set of keyframes can be simultaneously evaluated by traversing the nodes of the ferns and counting image co-occurrences in corresponding code tables. In tracking mode, this mechanism allows us to consider every frame/pose pair as a potential keyframe. A new keyframe is added only if it is sufficiently dissimilar from all previously stored keyframes. For tracking recovery, camera poses are retrieved that correspond to the keyframes with smallest BlockHDs. The pose proposals are then used to reinitialize the tracking algorithm. Harvesting of keyframes and pose retrieval are computationally efficient with only small impact on the run-time performance of the 3D reconstruction. Integrating our relocalization method into KinectFusion allows seamless continuation of mapping even when tracking is frequently lost. Additionally, we demonstrate how marker-free augmented reality, in particular, can benefit from this integration by enabling a smoother and continuous AR experience.