Distributions with latent variables

[yebai]

As of now, all distributions from the Distributions.jl package follow a simple convention, i.e.

x ~ distr(..)

the variable x contains all R.Vs of distr. In other words, distr doesn't have any latent variables.

We inherit the same convention from Distributions.jl for Turing.jl's tilde pipeline. For the majority of Turing use cases, this is flexible enough. However, there are cases in which latent variables inside distr are desirable, such as Hidden Markov Models (see #595), State Space Models (see, here), or submodels (see here). Thus, it is helpful to consider how to support distributions with latent variables in a more first-class and intuitive way. Here are some APIs that we could to better work with distributions containing latent variables

• has_latent(distr) returns true if distr has latent variables and false otherwise.
• distr_of_latent(distr) returns the distribution of the latent variables
• distr_of_returned(distr; marginal::Bool) returns the distribution of the latent variables, conditioning on the latent variables if marginal equals false; otherwise, it returns the marginal distribution of returned variables if marginal equals true (requires special support of distr, i.e. hand-written marginal distribution of returned variables).

Some practical considerations:

• For distribution objects returned by distr_of_latent / distr_of_returned, we are likely only able to support rand or logpdf functions instead of both.
• We won't be able to condition when lacking logpdf.