Tempered optimisation

[SamuelBrand1]

For problems with a complicated log-posterior landscape I've found doing a sequence of tempered optimisation quite a successful approach.

This looks pretty doable in Turing because DynamicPPL has a MiniBatchContext, which is essentially an equivalent approach, .e.g the log-likelihood contribution to the log-posterior density is scaled by some value $s \in [0,1]$.

The snag in code seems to be passing this context to estimate_mode which at the moment is hard coded to be either the DefaultContext or the LikelihoodContext which passes through to OptimizationContext for (i think) just checking.

Is there any chance of loosening this up to allow MiniBatchContext based evaluation of the mode?

[SamuelBrand1]

PS obviously I can (and will) hand roll this, e.g. via LogDensityFunction, but that replicates most of the existing code in Turing for mode estimation.

[SamuelBrand1]

By the way, I've realised that you can do this trivially by using contextualise

e.g.

using DynamicPPL, Turing

ctx = DynamicPPL.MiniBatchContext(DynamicPPL.DefaultContext(), tau)
opt_result = maximum_a_posteriori(DynamicPPL.contextualize(model, ctx))

Leads to optimisation with the loglike term scaled by tau. I wasn't sure this would work given the call out to OptimizationContext under the hood, but it seems to work (I've checked it on some non-trivial examples and it get the right answer for a range of tau values).