Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Deep Blending results, full evaluation script, and per-scene results #388

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

paulbzm
Copy link

@paulbzm paulbzm commented Sep 5, 2024

This adds a new evaluation script for the Deep Blending dataset, a new evaluation script to eval all datasets with different numbers of Gaussians and write the per scene results in csvs for the 3dgs.zip survey.
To this end the 2 existing evaluation scripts are modified to accept arguments.

The PSNR results on the Deep Blending dataset (dark brown) seem strange as they decrease with more Gaussians, but the SSIM and LPIPS results seem plausible. Any ideas on how to explain this or if it could be a bug related to the data loading are welcome @jefequien.

Capture Capture2 Capture3

The dataset can be downloaded with the updated download script in #385.

When skipping through the renderings, compress_0022.png of scene playroom (1M Gaussians) was the only really bad results that i noticed

compress_0022

However this is probably only an outlier due to lightening because it looks very different in compress_0018.png in a slightly different angle.

compress_0018

So maybe it is caused by the data itself that this approach leads to decreasing PSNR results with more Gaussians..

@liruilong940607
Copy link
Collaborator

Thanks for the efforts!

I never played with this dataset, but from the images you posted it seems like at least for this scene the exposure changes need to be compensated. The simple_trainer.py has an option app_opt to optimize for an appearance embedding for it but that is by default not being compressed I believed. The Bilateral Grid PR #291 is also another nice solution for this exposure issue which I think is more compatible with the compression.

BTW, if I would to guess what is happening for the really bad image, I think there is a huge transparent Gaussian living in the scene, which overfits the exposure in the training images but mess up the rendering from this viewpoint.

@jefequien
Copy link
Contributor

@paulbzm Issue #252 reports poor performance on the playroom scene under default mcmc settings. One of the original author recommends, per the paper, setting opacity_reg to 0.001.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants