Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Model comparsion for model fit with optimisation #162

Open
jeffreypullin opened this issue Jul 12, 2021 · 0 comments
Open

Model comparsion for model fit with optimisation #162

jeffreypullin opened this issue Jul 12, 2021 · 0 comments

Comments

@jeffreypullin
Copy link
Owner

Currently we implement loo() and (as of today) waic() to calculate model comparison metrics on the models fit using rater. Both functions, however, only support models fit using MCMC. This is because the underlying implementations in the {loo} package only supports MCMC-fit models. Ideally, we would also provide some way to compare models fit using optimization.

In the models fit using optimization we do have access to the log-likelihood evaluated at the MAP parameter values - but not the log-likelihood evaluated at the MLE. Unfortunately, I can't find any information criterion style metrics which work for models optimizing the log-posterior so I'm not really sure how to approach this issue.

@dvukcevic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant