Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A confusion comparing code with paper #25

Open
SuX97 opened this issue Apr 18, 2024 · 0 comments
Open

A confusion comparing code with paper #25

SuX97 opened this issue Apr 18, 2024 · 0 comments

Comments

@SuX97
Copy link

SuX97 commented Apr 18, 2024

Hi, authors:

Great work! While, I'm a bit confused about the description and code, in paper A.5:
image
x_t is called noisy image. However, in code,

        if self.cfg.recon_loss:
            # reconstruct x0
            latents_recon = self.model.predict_start_from_noise(
                latents_noisy, t, noise_pred
            )
            # x0-reconstruction loss from Sec 3.2 and Appendix
            loss = (
                0.5
                * F.mse_loss(latents, latents_recon.detach(), reduction="sum")
                / latents.shape[0]
            )
            grad = torch.autograd.grad(loss, latents, retain_graph=True)[0]

x_0, x_t is actually latent after vae and noisy latent, if correct.

There do exist methods that apply loss on image-space such as HiFA, and ReconFusion, which may be confusing.

Please clearify that I'm understanding it right, Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant