Skip to content
/ roft Public

Real-time Optical Flow-aided 6D Object Pose and Velocity Tracking

License

Notifications You must be signed in to change notification settings

hsp-iit/roft

Repository files navigation

ROFT

ROFT: Real-Time Optical Flow-Aided 6D Object Pose and Velocity Tracking

IEEE Robotics and Automation Letters, vol. 7, no. 1, pp. 159-166, Jan. 2022

DOI CI badge

Reproducing the experiments

We support running the experiments on the Fast-YCB dataset via the provided Docker image.

If you want to install the repository manually, please refer to the recipe contained in the Dockerfile.

  1. Pull the docker image:

    docker pull ghcr.io/hsp-iit/roft:latest
  2. Launch the container:

    docker run -it --rm --user user --env="DISPLAY" --net=host --device /dev/dri/ ghcr.io/hsp-iit/roft:latest

    If an NVIDIA GPU is adopted, please use instead:

    docker run -it --rm --user user --env="DISPLAY" --net=host -e NVIDIA_DRIVER_CAPABILITIES=all -v /tmp/.X11-unix:/tmp/.X11-unix --gpus all --runtime=nvidia ghcr.io/hsp-iit/roft:latest
  3. Update and build the project:

    cd /home/user/roft
    git pull
    cd build
    make install
  4. Download and extract the accompanying data (Fast-YCB dataset and pre-evaluated results) and the YCB-Video model set:

    cd /home/user/roft
    bash tools/download/download_results.sh
    bash tools/download/download_fastycb.sh
    bash tools/download/download_ycb_models.sh
  5. Initialize the datasets:

    cd /home/user/roft
    bash test/init.sh
  6. Run the experiments (optional):

    cd /home/user/roft
    bash test/run_paper_experiments

    The accompanying data contains the pre-evaluated results. If desired, the results can be re-evaluated using the above command.

  7. Run the evaluation:

    cd /home/user/roft
    bash evaluation/run_paper_evaluation
  8. Visualize the results: The results on the Fast-YCB dataset (Table I, II, IV and Figure 3) can be found in /home/user/roft/evaluation_output:

    • tableI.pdf
    • tableII.pdf
    • tableIV.pdf
    • Fig3_*.png

    The docker image provides evince and eog in order open pdf and png files, respectively.

In order to run part of the provided software it could be required to temporarily execute xhost + in a console outside of Docker in order to allow the container accessing the X server facilities. The command can be run even after the container has been already launched.

Support for reproducing the experiments on the HO-3D dataset will be added in the near future.

Instructions on how to use the ROFT library in external C++ projects and how to execute ROFT on custom datasets will be added in the near future.

Citing ROFT

If you find the ROFT code useful, please consider citing the associated publication:

@ARTICLE{9568706,
author={Piga, Nicola A. and Onyshchuk, Yuriy and Pasquale, Giulia and Pattacini, Ugo and Natale, Lorenzo},
journal={IEEE Robotics and Automation Letters},
title={ROFT: Real-Time Optical Flow-Aided 6D Object Pose and Velocity Tracking},
year={2022},
volume={7},
number={1},
pages={159-166},
doi={10.1109/LRA.2021.3119379}
}

and/or the repository itself by pressing on the Cite this respository button in the About section.

The pre-evaluated results on the Fast-YCB dataset are stored within the IIT Dataverse and identified by the following DOI:10.48557/9FJW42.

Generate optical flow frames for a custom dataset

If a custom dataset, using the same format as Fast-YCB, is available, the optical flow frames can be generated by:

  1. Enabling the option BUILD_NVOF when building using CMake
  2. Using the ROFT-of-dumper executable (synopsis available via CLI)

Note: a system with a optical flow-enabled NVIDIA GPU is required. Moreover, OpenCV should be compiled with the cudaoptflow contribution module.

Maintainer

This repository is maintained by:

@xenvre