Uncertainty quantification (UQ)

[ufsf]

What would you like to report?

Is it feasible to perform uncertainty quantification for eqV2 models, for instance, through ensemble methods? Many thanks!

[rayg1234]

Hello, it is possible to add UQ for our models but this is outside of the scope of anything we've released so far. Can you give a bit more detail on your use case and some more context? Thanks!

[ufsf]

Thank you so much for your response! I think it would be better to release several different models for eqV2_31M_omat_mp_salex.pt, for example. Five models should be enough to assess the UQ based on ensemble learning by calculating the variance for energy and atomic force.

I think UQ is very important because we need to make sure eqV2 models are reliable for new structures, without DFT calculations.

MatterSim has this functionality (Fig. S14 in https://arxiv.org/pdf/2405.04967) but they haven't released other models.

[rayg1234]

Hi, we also believe UQ is very important, but we haven't decided if ensemble of models is the best path forward. Currently we don't have the bandwidth to retrain/release an ensemble set of models for the past releases but this is being discussed and considered internally for future releases

If you would like to help and contribute to UQ or retrain ensembles given all the data/code is available, you are certainly welcome to!

[mshuaibii]

We had an intern explore this for some of our models that may be helpful to look at - https://pubs.acs.org/doi/full/10.1021/acs.jpcc.4c04972.