Here is an overview of my updates to make it easier to follow the code changes:

• created a droughts folder for recipes/diagnostics and a common documentation page

spei.R

• rewrote most of the diag_spei.R or diag_spi.R code, to be more flexible and work with SPEI v1.8
• split pet calculation into a seperate diag to make it easier to use pet from different sources
• save an additional metadata.yml (similar to preprocessor output) to make it easier to further process
• spi can be run as special case of spei

collect_droughts.py

• based on droughtindex/collect_drought_obs_multi.py (and other collect_*.py)
• updated to use ancestor metadata instead of searching output folders
• cleaned up the code and made it more readable
• added reference_dataset option, to set it explicitly rather than deciding based on meta keys
• aiming to support models, obs and different indices with one diagnostic

utils.py

• aiming for a module of reusable, documented, public functions
• merged general utility functions from collect_droughts_func.py and some required by diffmap.py
• paper specific plots/provenance functions are moved to collect_drought.py

TODOs:

• archive old recipe and diags. update notes in docs (recipe_spei.rst, recipe_martin18.rst, collect_droughts.py). Waiting for Extend broken recipe policy #3913