- freeglut (
sudo apt-get install freeglut3-dev
) - (optional) EGL used for headless rendering (
apt install libgl1-mesa-dri libegl1-mesa libgbm1
)
export PYOPENGL_PLATFORM=egl
before running these scripts, also set egl=True
in scripts/render_single.py
, otherwise, set egl=False
and unset PYOPENGL_PLATFORM
.
mesh.ray.intersects_any
, uninstall and reinstall pyembree
and trimesh
, more details in issue #62.
Please refer to THuman2.0-Dataset to download the original scans into data/thuman2/scans
, and its SMPL-X(male) fits into data/thuman2/fits
. Then generate all.txt
by ls > ../all.txt
under data/thuman2/scans
, which contains all the subject names (0000~0525).
PaMIR only support SMPL instead of SMPL-X, please download THuman2.0's SMPL fits (.obj, .pkl) and put them at ./data/thuman2.0/smpl
wget https://download.is.tue.mpg.de/icon/smpl.zip --no-check-certificate -O ./data/thuman2/smpl.zip
unzip ./data/thuman2/smpl.zip -d ./data/thuman2/
rm ./data/thuman2/smpl.zip
👀 ./sample_data
contains one example of THuman2.0 which shows the data folder structure.
conda activate icon
bash scripts/render_batch.sh debug all
bash scripts/vis_batch.sh debug all
Then you will get the rendered samples & visibility results under debug/
1. Rendering phrase: RGB images, normal images, calibration array. If you need the depth maps, just set depth=True
at render_single.py:L56
conda activate icon
bash scripts/render_batch.sh gen all
You could check the rendering status from log/render/thuman2-{num_views}-{size}-{part}.txt
2. Visibility phrase: SMPL-X based visibility computation
bash scripts/vis_batch.sh gen all
You could check the visibility computing status from log/vis/thuman2-{num_views}-{part}.txt
✅ NOW, you have all the synthetic dataset under data/thuman2_{num_views}views
, which will be used for training.
➡️ NEXT, please jump to Training Instruction for more details.
![]() |
![]() |
![]() |
![]() |
![]() |
---|---|---|---|---|
RGB Image | Normal(Front) | Normal(Back) | Normal(SMPL-X, Front) | Normal(SMPL-X, Back) |
![]() |
![]() |
![]() |
![]() |
![]() |
Visibility | Depth(Front) | Depth(Back) | Depth(SMPL-X, Front) | Depth(SMPL-X, Back) |
If you use this dataset for your research, please consider citing:
@InProceedings{tao2021function4d,
title={Function4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensors},
author={Yu, Tao and Zheng, Zerong and Guo, Kaiwen and Liu, Pengpeng and Dai, Qionghai and Liu, Yebin},
booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR2021)},
month={June},
year={2021},
}
This PyTorch Dataloader
benefits a lot from MonoPortDataset, so please consider citing:
@inproceedings{li2020monoport,
title={Monocular Real-Time Volumetric Performance Capture},
author={Li, Ruilong and Xiu, Yuliang and Saito, Shunsuke and Huang, Zeng and Olszewski, Kyle and Li, Hao},
booktitle={European Conference on Computer Vision},
pages={49--67},
year={2020},
organization={Springer}
}
@incollection{li2020monoportRTL,
title={Volumetric human teleportation},
author={Li, Ruilong and Olszewski, Kyle and Xiu, Yuliang and Saito, Shunsuke and Huang, Zeng and Li, Hao},
booktitle={ACM SIGGRAPH 2020 Real-Time Live},
pages={1--1},
year={2020}
}