Subgroup analysis

[wlandau]

Suggested by @chstock. I picture a model with subgroup and subgroup/treatment interaction and subgroup/treatment/time interaction with the following summaries:

1. Tables like in Postprocessing functionality #8 for each subgroup.
2. Marginal posteriors of each subgroup minus a reference subgroup.

We would also want a test of treatment/subgroup interaction as a whole, c.f. openpharma/mmrm#164.

I think this issue deserves to be addressed eventually, but the base case and informative priors are more important.

[kkmann]

+1 and +1 to adressing it later.

[kkmann]

-> tern.mmrm does this for mmrm https://github.com/insightsengineering/tern.mmrm

[wlandau]

Interesting. Looking at the README, I see:

Subgroup specific refitting of the MMRM model and resulting forest plot.

Internally at my company for the Bayesian case, so far we fit a single model to all the data with subgroup-specific fixed effects, then show a bunch of posterior summaries of contrasts. We don't use a Bayesian equivalent of openpharma/mmrm#164, and I am not sure if such a method exists, but maybe it would be good to eventually find out.

[wlandau]

I plan to start implementing subgroup models soon in order to support internal work.

Statistically, it seems clear enough how to add a subgroup factor to the model and then estimate meaningful subgroup-specific marginal means. From my perspective, the new part is figuring out how to measure the impact of the subgroup as a whole (as opposed to any individual subgroup level).

In the frequentist case, this sort of thing is a straightforward model comparison problem: run one model with a subgroup, run a nested model without the subgroup, and compare them with an F test. For Bayesian models, this could easily lead us to Bayes factors. I'm not really a fan of Bayes factors because they are difficult to interpret. And as far as the computation goes, bridge sampling is a good as it gets (https://cran.r-project.org/web/packages/bridgesampling/vignettes/bridgesampling_tutorial.pdf) and I have seen it diverge and produce inconsistent results a lot when computing marginal likelihoods for MMRM-like models.

If we really want something like a Bayes factor, the posterior odds might be more feasible using mixture importance sampling: https://arxiv.org/abs/2209.09190. At the very least, this would require 4 model fits instead of 2, and 2 of those runs would need to have an extra target += ... line somewhere in the model {} block for mixture IS. I think it would be more reliable than bridge sampling, but it seems like a lot of extra work for both developers and users given what we are trying to do.

Alternatively, we could straightforwardly list and compare the DIC and/or WAIC of the subgroup model vs the reduced model. These quantities would have serious problems if used directly for model averaging, but maybe the metrics themselves are enough.

Visually, we could compute the overall marginals of each model (by treatment and visit, averaging over subgroup for both models) and then compare them using brms.mmrm::brm_plot_compare(). That might give us an intuitive picture of how much variability is explained by the subgroup.

That covers the standard techniques I am aware of. So far, I am in favor of DIC/WAIC and visualization. Am I missing anything? Ideally, for a package like brms.mmrm, it would be great to land with one or more tried and true techniques that represent a consensus across our industry.

[wlandau]

Current branch for subgroup: https://github.com/openpharma/brms.mmrm/tree/18. A tentative roadmap is below, and it will depend on team consensus. For each function below, either the function itself needs to handle the subgroup, or brms.mmrm needs a new subgroup-oriented counterpart. Today I implemented subgroups in the brm_data() function and the underlying "brm_data" class.

• subgroup field in the "brm_data" S3 class
• brm_data() with subgroup
• brm_simulate_outline() with subgroup
• brm_simulate_prior() should use prior as NULL or brms::get_prior() a separate function for the prior. Implemented as brm_prior_simple(). brms.mmrm will likely extend the brms_prior_*() family of functions as we get into informative prior MMRMs.
• In preparation for subgroup, simplify argument names in brm_formula(), e.g. drop the effect_ and interaction_ argument prefixes using graceful deprecation.
• brm_formula() with subgroup: add specific terms for specific subgroup interactions.
• brm_marginal_data() for subgroup
• brm_marginal_draws() for subgroup
• brm_marginal_draws_average() for subgroup
• brm_marginal_probabilities() for subgroup
• brm_marginal_summaries() for subgroup
• brm_plot_draws() for subgroup
• brm_plot_compare() for subgroup
• DIC and WAIC of each model brms::waic() and brms::loo() are already available. We just need to document them.
• Re-render all existing fixed vignettes.
• Write a new vignette for subgroup analysis.
• SBC to validate the subgroup model.

[wlandau]

I think we have everything in place for subgroup so far, unless there are ideas for model comparison techniques other than brms::waic() and brms::loo(). I will get this merged before Wednesday's Bayesian MMRM sub-team meeting, then work on updating the simulation-based calibration workflow to use subgroup.

[wlandau]

It would be great to discuss in tomorrow's meeting what else we might want as far as the subgroup analysis. The functionality is in https://openpharma.github.io/brms.mmrm/articles/subgroup.html, and the validation of the subgroup model part of the "complex scenario" in https://openpharma.github.io/brms.mmrm/articles/sbc.html.

[wlandau]

Wrapped up in #79