-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add functionality for dH_abs in methods choice #81
base: main
Are you sure you want to change the base?
Conversation
Thanks for the addition @nezzag ! Let me know when you're done with any edits and I can review. |
Hey @nezzag - I realized that I have to (for some reason) approve the test suite, and it looks like there are failures. You can run these locally using pytest (let me know if you have any issues). |
Looks like everything passes but the linters, @nezzag - great work! Let me know if you have any questions there. |
Hi Matt, In brief, what I have done is add a new column to the methods choice DataFrame, which is dH_abs, the absolute difference between m and h. This is then compared to the "scale" of the data, which is calculated using the function In the methods choice, I've just added one decision gate. Which is if dH is large (>0.5) but dH_abs is small (<10% of the parent variable). This means that the historic data is <20% of the parent variable, and the modelled data to be harmonised is within 10% of this historic data. So this is where
Here aneris now still uses the convergence approaches, rather than defaulting to a constant ratio. BUT, it decides whether to use ratio_converge_2080 or offset_converge_2080, depending on whether the offset would be positive or negative. If the offset is negative we use ratio, to avoid introducing negative values (basically because I'm moving to use aneris more for energy variable harmonisation where negatives don't make sense, although I guess that's the case for most gases also). If the offset is positive we use offset, because otherwise the ratio could be very large (e.g. 20-30x), and this can result in very odd harmonised pathways. So the code should behave pretty much as before, except in the case where the data to be harmonised is ~small, and so the absolute difference is small, but the relative difference is large. Let me know what you think! |
Hey @nezzag - thanks again. I am about to dive into this and noticed there were no tests looking at the new values you've added. Would you mind adding at least one test for this? I think the easiest way (and most useful for future use cases) is to add a test for the |
This adds an option to determine methods choice not based on dH (a relative measure of the difference between historical and modelled data), but on dH_abs, an absolute measure.
This is useful in cases where the component being harmonised is a small variable (e.g. harmonising bio-liquids demand in industry, or a minor sub-species of gas). Here dH can be large, but dH_abs is still small, and hence the 'difference' between modelled and historic can still be seen as small enough to enforce convergence.
Otherwise a
constant_ratio
approach can be enforced, even though dH_abs is small.Code assesses what counts as a "small" absolute value by looking at the parent variable (which is the next level up in the IAMC hierarchy in this variable, e.g. Final Energy|Industry|Gases parent variable is Final Energy|Industry). dH_abs variables which are under half of the parent variable values are seen as small enough to still enforce convergence.