Sensing-by-Overlaying: A Practical Implementation of a Multiplayer Mixed-Reality Gaming System by Integrating a Dense Point Cloud and a Real-Time Camera

Abstract

This paper presents a mixed-reality gaming system architecture with a two-phase spatial information processing model. This model assumes that the physical world consists of mainly stable elements such as buildings, and wide-area physical structures are captured in advance by using a 3D laser range scanner. The captured structure data has exactly the same structure and topology as the physical world. Our system recognizes the physical world at runtime by overlaying this virtual structure data on a runtime monocular camera image at exactly the same position. By adopting this model, it is straightforward to implement image-based lighting for photometric integrity and occlusion handling for geometric integrity, because the overlaid virtual structure provides precise distance metrics and depth information. This model also resolves conflicts involving multiple sensing results obtained by multiple users. This paper shows the results of empirical evaluation experiments using the implemented prototype system.

Info

DOI: 10.1109/ISM.2016.0136

Citation: .bib format

Keywords: system implementation, mixed reality, augmented reality, gaming system, case study