RAMEN: Real-time Asynchronous Multi-agent Neural Implicit Mapping

Hongrui Zhao, Boris Ivanovic, Negar Mehr,

           

Installation

(1) Install basic python packages

First clone the repositroy and create the conda environment.

git clone https://github.com/labicon/RAMEN.git
cd RAMEN
# create conda env
conda create -n RAMEN python=3.10
conda activate RAMEN

Now install pytorch. RAMEN has been tested on CUDA 11.8 with RTX4090 & RTX2060M on Ubuntu 22/Linux mint 22.

pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu118

Install other python packages for mesh generation, visualization, and configuration files.

pip install PyMCubes==0.1.6 open3d==0.19.0 trimesh==4.5.3 opencv-python==4.11.0.86 matplotlib==3.10.0 pyyaml==6.0.2

If you are facing issues with qt, such as Could not load the Qt platform plugin "xcb", try pip install opencv-python-headless==4.11.0.86.

(2) Build tinycudann from source

Make sure your CMake is 3.21 or higher, and your gcc is 11 or higher.
Now we will build tinycudann from source. It is going to take a while. As long as you are seeing Building wheel for tinycudann (setup.py) ... , it's all good.

pip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch

(3) Build pytorch3d from source

First install pytorch3d's dependecies

pip install -U iopath

Now we will build pytorch3d from source. It is going to take a while. As long as you are seeing Building wheel for pytorch3d (setup.py) ... , it's all good.

 pip install "git+https://github.com/facebookresearch/pytorch3d.git"

Dataset

(1) Replica

Download the sequences of the Replica Dataset generated by the authors of iMAP into ./data/Replica folder.

bash scripts/download_replica.sh # Released by authors of NICE-SLAM

To just download office 1 scene for a quick testing, here is the google drive link for downloading: https://drive.google.com/drive/folders/1211QzrvfZGgiNk7H0t-NhpuUz6vv2MNN?usp=sharing.

(2) ScanNet

Please follow the procedure on ScanNet website, and extract color & depth frames from the .sens file using the code.

Config files

Config yaml files for different scenes are located at /configs folder.
Some config parameters of your interest:

• data.exp_name: results will be saved at path data.output + data.exp_name.
• multi_agents.complete_graph: if set to True, the agent communication graph will be a complete graph (every agent can talk to every agent).
• multi_agents.edges_list: if complete_graph is set to False, this will define the comuunication edges between agents.
• multi_agents.edges_for_dropout: A list of [node i, node j, possibility of dropping out] to define the communication dropout rate. Empty list means no communication dropout.
• multi_agents.distributed_algorithm: To run RAMEN, set it to AUQ_CADMM. Other baseline methods are CADMM (this is DiNNO in our paper), MACIM, DSGD, DSGT.
• multi_agents.fix_decoder: need to set to True to run RAMEN.

(1) Single agent mapping

To run centralized/single agent mapping, do the following:

• num_agents: 1
• edges_for_dropout: []
• distributed_algorithm: 'CADMM'.

Run

Let's run office 1 from replica dataset

python main.py --config configs/Replica/office1.yaml

Visualization and evaluation

(1) Visualization

python visualizer.py --config ./configs/Replica/office1.yaml 

(2) Evaluation

Check out analysis.ipynb.

Acknowledgement

Our code is partially based on Co-SLAM. We thank the authors for making these codes publicly available.

Citation

@inproceedings{Zhao2025RSS
  title={RAMEN: Real-time Asynchronous Multi-agent Neural Implicit Mapping},
  author={Zhao, Hongrui and Ivanovic, Boris and Mehr, Negar},
  booktitle={Robotics: Science and Systems (RSS)},
  year={2025}
}