Add esmvaltool compare command to run esmvaltool/utils/testing/regression/compare.py

[alistairsellar]

Is your feature request related to a problem? Please describe.
As discussed in ESMValGroup/ESMValTool#3437, it would be useful for recipe test workflow (and more generally for users) to have an esmvaltool compare command that uses the functions in esmvaltool/utils/testing/regression/compare.py to compare results from two recipe runs.

Would you be able to help out?
Yes, I would make this change.

[alistairsellar]

Having scoped the code, I propose that the command (at least in first implementation) compares a single "current" run with the reference, in contrast to compare.py which allows for a list of "current" runs to each be compared to the reference. This approach would allow the command to call the existing compare() function without changes to ESMValTool, whereas a list would require refactoring of main(), or duplication of some of its code into Core. Users who need the more advanced multi-run comparison in a single call can run compare.py itself.

Tagging @bouweandela for interest and opportunity to disagree with proposal.

[bouweandela]

Nice idea! You may want to consider if this will be the only recipe development and testing utility that you want to integrate, or if there might be more in the future. In the latter case, it might be nicer to group them under e.g. an esmvaltool development command, to avoid that we end up with 25 esmvaltool something commands.

[alistairsellar]

Good point @bouweandela, will have a think about that.

[alistairsellar]

Nice idea! You may want to consider if this will be the only recipe development and testing utility that you want to integrate, or if there might be more in the future. In the latter case, it might be nicer to group them under e.g. an esmvaltool development command, to avoid that we end up with 25 esmvaltool something commands.

Thanks @bouweandela, have thought about this and discussed with @ehogan. We can't think of any other development commands, apart from potentially esmvaltool config validate (discussed in ESMValGroup/ESMValTool#3392) which would be in the existing config command group.

So I'll get back to my draft PR for an esmvaltool compare command.