DINOSAVi

Self-supervised learning of Video Object Segmentation using DINOSAUR and SAVi

Currently, results are being consolidated and a paper is in progress. For now, enjoy the source code.

Presentation Slides: https://1drv.ms/p/s!AgE9E4ZerfvahsdfaYUe8caynzF0iw

Models: https://1drv.ms/f/s!AgE9E4Zerfvahsd8TTjyDmHvikwY6g?e=o2n7EY

Note: https://hydra.cc/ is used to manage configuration, CLI and experiment running. See https://hydra.cc/docs/advanced/override_grammar/basic/ for CLI override grammar. Jump to Codebase Notes for more info.

Installation

# (Optional) Use conda for virtual environment instead. Poetry creates venv by default.
conda create -n dinosavi python=3.10
poetry install

Training

python -m dinosavi mode=train hparam={hparam_file_name} exp_name={experiment_name}

Evaluation

python -m dinosavi mode=eval resume={.ckpt_to_load} exp_name={results_name} device={cpu_or_cuda}

Codebase Notes

• Hydra (built on OmegaConf) is used for configuration management, CLI and experiment running.
  • For CLI options, you can temporarily refer to dinosavi/cfg/main.yaml and dinosavi/cfg/mode for now. Proper CLI documentation is in progress.
  • Hyperparameter config files are stored in dinosavi/cfg/hparam.
    • OmegaConf Variable Interpolation is heavily used to link hyperparameter values into relevant config files.
  • Many parts of the code (i.e., models, datasets, trainers) are implemented as config files using Hydra's Instantiate API.
• We use Black for formatting, isort for import sorting, and Google-style docstrings.
• Anything to do with Contrastive Random Walk (CRW) is leftover legacy code from earlier experiments.