MAP-Adapt: Motion-Aware Projection for Adaptive Scene Reconstruction

Jianhao Zheng¹ · Dániel Béla Baráth² · Marc Pollefeys^{2, 3} · Iro Armeni¹

European Conference on Computer Vision (ECCV) 2024

¹Stanford University · ²ETH Zurich · ³Microsoft

Abstract

Creating 3D semantic reconstructions of environments is fundamental to many applications, especially when related to autonomous agent operation (e.g., goal-oriented navigation or object interaction and manipulation). Commonly, 3D semantic reconstruction systems capture the entire scene in the same level of detail. However, certain tasks (e.g., object interaction) require a fine-grained and high-resolution map, particularly if the objects to interact are of small size or intricate geometry. In recent practice, this leads to the entire map being in the same high-quality resolution, which results in increased computational and storage costs. To address this challenge, we propose MAP-ADAPT, a real-time method for quality-adaptive semantic 3D reconstruction using RGBD frames. MAP-ADAPT is the first adaptive semantic 3D mapping algorithm that, unlike prior work, generates directly a single map with regions of different quality based on both the semantic information and the geometric complexity of the scene. Leveraging a semantic SLAM pipeline for pose and semantic estimation, we achieve comparable or superior results to state-of-the-art methods on synthetic and real-world data, while significantly reducing storage and computation requirements.

Getting Started

TBA, we are currently cleaning the code.

Contact

If you have any question, please contact Jianhao Zheng ([email protected]).

Ackownledgement

Our implementation is heavily based on Voxblox. We thank the authors for open sourcing their code. If you use the code that is based on their contribution, please cite them as well.

Citation

If you find our code and paper useful, please cite

@inproceedings{zheng2024map,
  title={Map-adapt: real-time quality-adaptive semantic 3D maps},
  author={Zheng, Jianhao and Barath, Daniel and Pollefeys, Marc and Armeni, Iro},
  booktitle={European Conference on Computer Vision},
  pages={220--237},
  year={2024},
  organization={Springer}
}