Reconstructing an environment’s 3D models is traditionally a computer vision problem, crucial for virtual reality (VR) applications and mobile robots that have to estimate the pose of the camera that moves with them. Well-known vision methods, such as structure from motion (SfM), and robotics methods, such as visual simultaneous localization and mapping (SLAM), while effective in static environments are still challenging in dynamic environments.

This survey illustrates the state of the art of vision and robotics methods for real-time rendering in real-world environments containing dynamic objects. It proposes a taxonomy of the available approaches divided into three main themes: building static maps by rejecting dynamic features (robust visual SLAM), extracting moving objects while ignoring the static background (dynamic object segmentation and 3D tracking), and simultaneously handling the static and dynamic components of the world (joint motion segmentation and reconstruction). It also critically discusses the advantages and disadvantages of the many illustrated approaches, which rely on methods spanning from geometry to statistics to machine learning.

The authors nicely organize about 200 references, using figures with flow diagrams and summarizing via tables the existing approaches. The paper can serve as an introduction for researchers new to the field, as well as a practical guide to specific approaches for application-oriented developers.