Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Have some doubts about the calculation of evaluation indicators. #99

Open
swing148 opened this issue Feb 7, 2023 · 0 comments
Open

Comments

@swing148
Copy link

swing148 commented Feb 7, 2023

Hello, your paper is really good!
But for fid50k_ full.There are some problems with the evaluation index of full.
"we report the FID between 50k generated and all real images" in your paper.
However, Pokemon has only more than 800 data sets, and the number of generated images is greater than the number of real images. Is this comparison really accurate? Although the initial literature did not mention this issue[1]. However, this document mentions that generating 50k images will lack diversity[2].
We try to study this problem in full dataset(<1000 and >800), half dataset(500) and 100 dataset vs same number generated images,We find that FID decreases as the number of images increases. Of course, generated 50k images vs full dataset(<1000 and >800)is least.The evaluation method we use is [3].
[1]Heusel M , Ramsauer H , Unterthiner T , et al. GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium[J]. 2017.
[2]Konstantin Shmelkov, Cordelia Schmid, Karteek Alahari.How good is my GAN[C].ECCV,2018,218-234.
[3]https://github.com/mSEitzer/pytorch-fid

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant