Nikos Athanasiou | Alpár Cseke | Markos Diomataris | Michael J. Black | Gül Varol
| Paper Video | Dataset Exploration |
|---|---|
 |
 |
-
🗞️[Nov. 7th '24] Updated and released our video on youtube, and added a preview for our data exploration website.
-
🗞️[Oct. 31th '24] Release of a script to visualize motions and to save the groundtruth motions. Look at our website to see the visualized pairs of the dataset. For a quick workaround to visualize one or more motions give a try to:
python visualize_sample.py --path experiments/tmed/3way_steps_300_motionfix_noise_last/ld_txt-2.0_ld_mot-2.0/000136.npy.If you want to save the source and the target motions while saving the samples you can do so by setting the
save_gtflag, i.e., to generate samples do:python motionfix_evaluate.py folder=experiments/tmed/ guidance_scale_text_n_motion=2.0 guidance_scale_motion=2.0 data=motionfix save_gt=true. -
🗞️[Oct. 30th '24] Release of a script to visualize motions (a demo will follow-up until December). For a quick workaround to visualize see here.
-
🗞️[Sep. 20th '24] Release of the dataset and the exploration website.
-
🗞️[Aug. 30th '24] First release of the code.
You can try extracting the dataset from AMASS, given our annotations which can be downloaded from here. Check and our explore our data in our exploration webpage. First, check our License. Then, you can download our data already processed and ready to use from this link.
MotionFix has been implemented and tested on Ubuntu 20.04 with python >= 3.10.
Clone the repo:
git clone https://github.com/athn-nik/motionfix.gitAfter it do this to install DistillBERT:
cd deps/
git lfs install
git clone https://huggingface.co/openai/clip-vit-large-patch14
cd ..Install the requirements using virtualenv :
# pip
source scripts/install.shYou can do something equivalent with conda as well.
After that get the data and models needed.
# pip
source scripts/download_data.shTo get a checkpoint of our model and run inference, you can download it from this link. Then, you can use this checkpoint to extract samples and compute metrics as described below.
python motionfix_evaluate.py folder=/path/to/exp/ guidance_scale_text_n_motion=2.0 guidance_scale_motion=2.0 data=motionfixThis will generate samples from the diffusion model for the MotionFix test set. By default the /path/to/exp should be experiments/tmed, but your path should be adjusted
according to where you have your experiments. The samples are generated in a path that looks like 3way_steps_300_motionfix_noise_last/ld_txt-2.0_ld_mot-2.0. The first part indicateds
the parameters of sampling (number of steps, dataset name, initialization of diffusion, checkpoint which we sample from) and the second part the diffusion values for the two conditions.
python compute_metrics.py folder=/path/to/exp/samples/npysThat /path/to/exp/samples/npys should be something like experiments/tmed/3way_steps_300_motionfix_noise_last/ld_txt-2.0_ld_mot-2.0, if you followed the previous step successfully.
Metrics will be printed in stdout.
Then create a directory named data and put the babel data and the processed amass data in.
You should end up with a data folder with the structure like this (tree dirname):
data
|---- motionfix
| ├── motionfix.pth.tar
| ├── motionfix_val.pth.tar
| ├── motionfix_test.pth.tar
|---- body_models
| ├── smplh
| ├──SMPLH_MALE.npz
| ├──...
| ├── smpl
| ├──SMPLH_MALE.pkl
| ├──...
eval-deps
|---- config.json
|---- last_weights
│ ├── motion_decoder.pt
│ ├── motion_encoder.pt
│ └── text_encoder.pt
├──-- logs
│ ├── checkpoints
│ │ ├── last.ckpt
│ ├── hparams.yaml
│ └── metrics.csv
└── stats
└── humanml3d
└── amass_feats
├── mean.pt
└── std.pt
Be careful not to push any data! Then you should softlink inside this repo. To softlink your data, do:
ln -s /path/to/data
You can do the same for your experiments:
ln -s /path/to/logs experiments
This the best way to have a consistent structure and namings, while keeping the data stored in a different location, which does not interfere with the code (IMO).
To start training after activating your environment. Do:
python train.py experiment=... logger=... run_id=... exp_dir=...logger=wandbor none.experimentname of the experiment folder e.g.,ablation-data.run_idsubname of the experiment e.g.,10percent_dataexp_dirthe folder where you want the experiments to be saved (defaults is./experiments).
Such a training will create a folder exp_dir/experiment/run_id where all the experiments logs are saved.
Explore configs/train.yaml to change some basic things like where you want
your output stored, which data you want to choose if you want to do a small
experiment on a subset of the data etc.
You can check the train.yaml for the main configuration
and this file will point you to the rest of the configs (eg. model refers to a config found in
the folder configs/model etc.).
@inproceedings{athanasiou2024motionfix,
title = {{MotionFix}: Text-Driven 3D Human Motion Editing},
author = {Athanasiou, Nikos and Ceske, Alpar and Diomataris, Markos and Black, Michael J. and Varol, G{\"u}l},
booktitle = {SIGGRAPH Asia 2024 Conference Papers},
year = {2024}
}This code is available for non-commercial scientific research purposes as defined in the LICENSE file. By downloading and using this code you agree to its terms. For MotionFix dataset check its license. Third-party datasets and software are subject to their respective licenses.
This code repository was implemented by Nikos Athanasiou. It incorporates a version of TMR for evaluation.
Give a ⭐ if you like it.