-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Anime4K_Upscale_GAN_x2_M seem to be producing some minor artifacts that are not present in the other GAN variants #146
Comments
Thanks for the in-depth comparison pictures! I think that it's simply by chance the M shader didn't converge correctly during training. I will try to fix this before the official release of shaders. |
In fact I think I can see very tiny (1 pixel) artifacts in the L version too. There might be broader convergence issues from the adversarial training. I will have to investigate. Solving this correctly might take more time as there's no papers that tackle training tiny SRGANs. For now I will try re-training both networks to see if the issue repeats itself. |
By my test, upscale GAN x2-UL could produce the best quality while the least artifacts, but it was removed. @bloc97 Could we get another choice to replace GAN x2-UL later? |
That's because GAN x2-UL was trained with small amounts of denoising, I will see if we should support 6 GAN+Denoise upscalers, or let the GAN-Restore handle denoising. |
The optimal case would be to delegate all degradations to RestoreGAN shaders, and let the SRGAN work with pristine images. The current problem is that Restore shaders sometimes introduce ringing, blurs out details or do not restore lines enough. This happens because the restore shader has no idea of the whole image and only works on small patches of the image. It cannot know if the entire image is blurred or whether the blur was intentional and only for a part of the image. This can cause the restore shader to try removing noise or blur where there is no degradation and cause a loss of detail/ringing. A lazy method was used for the v4.0 restore shaders, by training one shader (non-soft version) on some degradation and another (soft version) on other types of degradation. By implementing global feature-wise transformations we can hope the new RestoreGAN shaders will not denoise, deblur or add ringing to images that have no degradation. |
I went back to check the old GAN x2-UL shader and it does seem to be working great and producing less artifacts for most cases. But it also has some edge cases where it was losing a lot of the minor details on grainy textures (not as much as the CNN shaders but still). Would be great if there was a way to reduce the artifacts while minimizing the loss of details ^^ |
That's basically what we're trying to do with feature-wise transformations. If the shader can actually know how much compression is in an entire image, it can learn to not remove details when overall noise is low. Compression and degradation is almost always applied to the entire image, while artistic degradation is usually applied to parts of an image. The important thing is to distinguish between the two by giving the neural network a way to look at the entire image without slowing it down too much. |
Upon further examination of the image you provided, it is a very good example of having all kinds of artistic degradation + unintended noise from compression. There are four things:
If you try restore shaders with this image it will fail considerably, as expected. The goal is for the new RestoreGAN shaders to take in account all four cases, so that it can make a better restoration overall. |
If the RestoreGANs will be successful at restoring an image without distorting it, the SRGAN shaders will not need to denoise the image. This makes both training and using the shaders much easier, and we would not need a second set of SRGAN+Denoise shaders: Pristine images from pixiv: Use SRGAN only. |
Not really sure if this is a bug or not, but earlier I was playing with the new GAN shaders and noticed one thing. The X2_M variant of the GAN shader seem to be producing some artifacts here and there, which are not present in any other GAN shaders. To demonstrate, here are some images.
And album of the full images for reference: https://imgur.com/a/qV66BA3
All tested with the latest windows build of MPV on default settings on a 1080p display(I don't have 4k :-x ), running on RTX2060 GPU.
I might be nitpicking but feels odd to me that only one variant of the GAN shaders would produce such artifacts.
And good work on the new shaders. These seem to be much better at keeping grainy textures intact compared to the CNN ones.
Edit: My input.conf:
The text was updated successfully, but these errors were encountered: