-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
remove ImageFiltering
dependency
#10
Comments
Hello @johnnychen94, I want to work on this issue. If you could help me with some pointers, maybe I can implement this 😄 . |
Thanks for the interest @LakshyaKhatri I haven't done enough research yet, but basically what's needed to be implemented is a simplified version of Given that Zygote is FFTW compatible, we could provide a fftw-based convolution operation with the famous convolution theorem here. Once that's done, we could then replace the usage of Some of the codes might need to be modified accordingly to be differentiable, though. However, note that in other implementations(e.g., https://github.com/Po-Hsun-Su/pytorch-ssim) , usually the SSIM is simplified and optimized for 4D Tensor. |
Making JuliaImages zygote-friendly is a giant project. TBH, I don't know yet what Zygote-compatible means to JuliaImages. Since Images are typically represented as I'm wondering if I can get some advice from @avik-pal and @timholy about this. |
Perhaps the most important step is allowing dual numbers as the eltype of colortypes. Then autodiff will Just Work, right? |
I think allowing dual numbers to work will allow only ForwardDiff / ReverseDiff to work. Zygote in theory (Here is a small example using Colors.jl.) should work for Making the entirety of JuliaImages Zygote friendly definitely is a big project but, making the gradients work for SSIM should not be very difficult. I can see 2 possible solutions:
|
I think removing ImageFiltering dependency might be impractical if we ever want to extend this library. Thus I'm closing this for now. The effort should be put into making |
There're two reasons for this:
ImageFiltering
is heavy; currently the only place it's used isimfilter
inSSIM
.SSIM
compatible toZygote
, we might need to use a simplified version ofimfilter
built only upon FFTW.The text was updated successfully, but these errors were encountered: