Formalise AD integration status, rewrite AD page correspondingly #595
Conversation
penelopeysm
changed the title
penelopeysm
force-pushed
the
py/ad
branch
2 times, most recently
from
23469c6 to
e594515
Compare
|
Preview the changes: https://turinglang.org/docs/pr-previews/595 |
| **Tier 3** is the same as Tier 2, but in addition to that, we formally also take responsibility for ensuring that the backend works with Turing models. | ||
| If you submit an issue about using Turing with a Tier 3 library, we will actively try to make it work. | ||
| Realistically, this is only possible for AD backends that are actively maintained by somebody on the Turing team, such as Mooncake. |
There was a problem hiding this comment.
What are the limits to how far we're willing to take this? Per our discussion yesterday, if someone does something really non-differentiable (e.g. a custom ccall), we're not going to try and add support for their proposal.
Maybe "we will actively try to make it work" could be extended to say how we'll try to make it work? e.g. if someone encounters a bug, we'll fix it, but if they're doing something unusual we might suggest a more standard way to go about it that avoids the problem they're seeing entirely.
| Firstly, you could broaden the type of the container: | ||
|
|
||
| ```{julia} | ||
| @model function forwarddiff_working1() | ||
| x = Real[0.0, 1.0] | ||
| a ~ Normal() | ||
| x[1] = a | ||
| b ~ MvNormal(x, I) | ||
| end | ||
| sample(forwarddiff_working1(), NUTS(; adtype=AutoForwardDiff()), 10) | ||
| ``` | ||
|
|
||
| Or, you can pass a type as a parameter to the model: | ||
|
|
||
| ```{julia} | ||
| @model function forwarddiff_working2(::Type{T}=Float64) where T | ||
| x = T[0.0, 1.0] | ||
| a ~ Normal() | ||
| x[1] = a | ||
| b ~ MvNormal(x, I) | ||
| end | ||
| sample(forwarddiff_working2(), NUTS(; adtype=AutoForwardDiff()), 10) | ||
| ``` |
There was a problem hiding this comment.
Would it be helpful for users to make it clear that the second option here is highly preferable to the first in general, and that the first should only be used if the second doesn't work for some reason?
There was a problem hiding this comment.
Yes, definitely, and a link to https://discourse.julialang.org/t/vector-real-vector-float64-methoderror/25926/5 might also be helpful (it helped me back in the day)
Co-authored-by: Will Tebbutt <[email protected]>
This is an initial attempt to put down in words what @willtebbutt and I think our approach to integrating AD backends should be going forward.