SV3D update README (Stability-AI#305)

* Makes init changes for SV3D * Small fixes : cond_aug * Fixes SV3D checkpoint, fixes rembg * Black formatting * Adds streamlit demo, fixes simple sample script * Removes SV3D video_decoder, keeps SV3D image_decoder * Updates README * Minor updates * Remove GSO script * Updates REAME, fixes names --------- Co-authored-by: Vikram Voleti <[email protected]>
LinearFalcon · Jul 6, 2024 · 66ea5b4 · 66ea5b4
1 parent 00ddc63
commit 66ea5b4
Show file tree

Hide file tree

Showing 3 changed files with 12 additions and 2 deletions.
diff --git a/README.md b/README.md
@@ -12,8 +12,18 @@
     - We extend the streamlit demo `scripts/demo/video_sampling.py` and the standalone python script `scripts/sampling/simple_video_sample.py` for inference of both models.
     - Please check our [project page](https://sv3d.github.io), [tech report](https://sv3d.github.io/static/paper.pdf) and [video summary](https://youtu.be/Zqw4-1LcfWg) for more details.
 
+<<<<<<< HEAD
+To run SV3D_u on a single image:
+- Download `sv3d_u.safetensors` from https://huggingface.co/stabilityai/sv3d to `checkpoints/sv3d_u.safetensors`
+- Run `python scripts/sampling/simple_video_sample.py --input_path <path/to/image.png> --version sv3d_u`
+
+Additionally for SV3D_p,
+- Specify sequences of 21 elevations and 21 azimuths (in degrees) to `elevations_deg` ([-90, 90]), and `azimuths_deg` [0, 360] in sorted order from 0 to 360. For example:
+`python scripts/sampling/simple_video_sample.py --input_path <path/to/image.png> --version sv3d_p --elevations_deg [<list of 21 elevations in degrees>] --azimuths_deg [<list of 21 azimuths in degrees>]`
+=======
 To run SV3D on a single image:
 `python scripts/sampling/simple_video_sample.py --input_path <path/to/image.png> --version sv3d_p`
+>>>>>>> main
 
 To run SVD or SV3D on a streamlit server:
 `streamlit run scripts/demo/video_sampling.py`

diff --git a/scripts/sampling/configs/sv3d_p.yaml b/scripts/sampling/configs/sv3d_p.yaml
@@ -3,7 +3,7 @@ model:
   params:
     scale_factor: 0.18215
     disable_first_stage_autocast: True
-    ckpt_path: checkpoints/sv3d_p_image_decoder.safetensors
+    ckpt_path: checkpoints/sv3d_p.safetensors
 
     denoiser_config:
       target: sgm.modules.diffusionmodules.denoiser.Denoiser

diff --git a/scripts/sampling/configs/sv3d_u.yaml b/scripts/sampling/configs/sv3d_u.yaml
@@ -3,7 +3,7 @@ model:
   params:
     scale_factor: 0.18215
     disable_first_stage_autocast: True
-    ckpt_path: checkpoints/sv3d_u_image_decoder.safetensors
+    ckpt_path: checkpoints/sv3d_u.safetensors
 
     denoiser_config:
       target: sgm.modules.diffusionmodules.denoiser.Denoiser