Atlas: Better augmented reality with occlusion by using slam and learned depth prediction

Abstract

Augmented Reality has seen a lot of growth in recent years, thanks to cheaper hardware and better software. However it still suffers from the major issue of the lack of Spatial Awareness. This is the root of many limitations of the technology, such as the inability for a real world object to occlude a digital one, or limited interactions between the environment and the AR experience. Such base problems should ideally be solved in a way that is accessible to all, without restrictions of expense or accessibility. The Author proposes a software library that will use regular color images such as those supplied from common monocular RGB cameras to perform spatial mapping of the environment around the user. While this kind of “Visual SLAM” algorithms already exist, most suffer from the inability to sense depth without camera motion, a while some researchers have sought to address that, the author presents a new method in which a Conditional Generative Adversarial Network is used to predict the depth of a given image, and build a 3d map over time. Reinforcement learning techniques will be used to help the CGAN improve its prediction accuracy over time. Please note for brevity: The library being developed will be called “Atlas”