ahead of its time?

[bitowaqr]

Hi,

I love the idea of an HTA in R manifesto.

I think you cover some really important points and it would be really useful if people adopt these good practices. However, I am slightly worried that this is moving a bit too fast. At the moment, 99% of all HTA submission are probably still in MS Excel. For many modellers, moving to R will be a big step. It is not only about learning a (new) programming language, but the entire workflow is different (working with repositories, packages, etc.). I am thus not sure it would help with the transition, if this manifesto immediately leads to setting very specific standards – which are much higher than what people were used to in Excel.

I would also think that specific standards (for data, input/output formats etc) should emerge from a larger community of R HTA modellers – I don't know if there is enough experience with R models to really know what is most useful to NICE and other stakeholders.

Instead of trying to push for a specific submission style, at this point, it might be more helpful to promote general principles and showcase some best practices – is there even one (simple!) model that adheres to all the points made in the manifesto that could be shown as an example?

Again, I really like the idea of a manifesto, but I worry that it is a bit too early to go into the details of submission standards. Not sure if this is helpful, but these were my general reflections.

Many thanks,
Paul

[giabaio]

Hi @bitowaqr and thanks for this (also: apologies if nobody has bothered responding to your very interesting and helpful point! I meant to do it straightaway, but then you know how it is...).

So: I don't disagree in principle and I think you make very valid points, like I said. I guess there are a couple of counters, though, which maybe of relevance.

1. On the point of the vast majority of models being in Excel. That's incontrovertible (sadly), but also (and I think we did discuss at the last R-HTA workshop in Oct 2020), there's a unique opportunity to perhaps direct and guide the various jurisdictions that are now establishing themselves (eg in many low-middle income countries). Also, I think that perhaps there's a bigger (possibly hidden) interest within our "reference industry" (by which I mainly mean consultancy) to increasingly make the transition. Or at least there is a recognition that being able to work in Excel only is unlikely to be a sustainable model. So I'd say it's a good time to keep banging on.
2. Re the point on standards, I think that we don't need to be absolutely prescriptive here. Also, in my view standards are meant to be shared and agreed upon by as wide as possible a proportion of the relevant community. But by nature, they are not immutable --- just think of what tidyverse has been/is doing to the R community... We should be very clear that what we'd be suggesting is meant to be increasingly understood and contributed to by as many modellers as possible, which will inevitably lead to some possibly re-adjustment.
3. Re general principle, also agreed. But then again, I think having a list of "requirements" or styles that people should, more or less, abide by, I think, may increase our chances of "imposing our will" --- I think if you have very clear guidelines, non-experienced modellers may rely on them (that they may do it uncritically, it does bother me!) and be willing to apply them, because there's an element of "expert opinion" at play...

Hope these make sense and happy to continue the discussion!