Document how to use Pathfinder to initialize PPLs

[sethaxen]

Originally posted by @joshualeond in #10 (comment):

Exciting to see this work being done in Julia! Do you think it would make sense to include an example of how you could use the results of this package to initialize MCMC in Turing/Soss/Stan/etc?

[sethaxen]

This is a great idea! We can at least initialize from the random points Pathfinder returns. It's even more useful if we can initialize the metric.

Ideally we would also refine the metric using the adaptation scheme explained in https://arxiv.org/abs/1905.11916, but that will come with AdvancedHMC integration either here or in AdvancedHMC. See TuringLang/AdvancedHMC.jl#282

[joshualeond]

Thanks! I hadn't heard of the metric used in Stan's HMC before. I don't see the Euclidean metric referenced in the Pathfinder paper though. Is it mentioned under a different name or is this something that hasn't been done before in Stan?

[sethaxen]

The metric is sometimes called the mass matrix, and its use in Stan is explained at https://mc-stan.org/docs/2_28/reference-manual/hmc-algorithm-parameters.html#euclidean-metric . Typically the inverse of the metric is adapted by setting it to the sample covariance or a diagonal estimate of it, so sometimes people just talk about estimating the sample covariance. This is what they're talking about in the discussion of the Pathfinder paper:

The Bales et al paper is the one I linked to above. So you can see how Pathfinder gives a different approach for setting the covariance instead of estimating it from samples, which one could then use to initialize the metric adaptation.