Skip to content

Commit

Permalink
linting
Browse files Browse the repository at this point in the history
  • Loading branch information
@konstantinstadler
konstantinstadler committed Jul 16, 2024
1 parent 75cb378 commit f5e7aff
Show file tree
Hide file tree
Showing 4 changed files with 207 additions and 160 deletions.
212 changes: 121 additions & 91 deletions doc/source/notebooks/convert.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,24 +16,24 @@
# # Convert and Characterize MRIO satellite accounts and results

# %% [markdown]
# Here we discuss the possibilities for converting MRIO satellite accounts (Extensions)
# Here we discuss the possibilities for converting MRIO satellite accounts (Extensions)
# and results.
# The term *convert* is used very broadly here, it includes the following tasks:
#
# - renaming the index names of results/extensions
# - adjusting the numerical values of the data,
# - adjusting the numerical values of the data,
# e.g. for unit conversion or characterisation
# - finding and extracting data based on indicies across a table or an mrio(-extension).
# This can be system based on name and potentially constrained by sector/region
# This can be system based on name and potentially constrained by sector/region
# or any other specification.
# - Aggregation/Summation of satellite accounts
# - Characterization of stressors to impact categories
#
# We will cover each of these points in the examples below.
# We will start with applying the conversion to a single table
# We will cover each of these points in the examples below.
# We will start with applying the conversion to a single table
# and then cover the conversion of a full MRIO extension.
#
# For the connected topic of *Aggregation of MRIOs*
# For the connected topic of *Aggregation of MRIOs*
# see the [Aggregation](./aggregation_examples.ipynb) page.

# %% [markdown]
Expand All @@ -44,15 +44,15 @@
# the indices of the source data to the indices of the target data.

# %% [markdown]
# This tables requires headers (columns) corresponding to the column headers
# This tables requires headers (columns) corresponding to the column headers
# of the source data as well as bridge columns which specify the new target index.
# The later are indicated by "NewIndex__OldIndex" - **the important part are
# the two underscore in the column name**. Another column named "factor" specifies
# the multiplication factor for the conversion.
# The later are indicated by "NewIndex__OldIndex" - **the important part are
# the two underscore in the column name**. Another column named "factor" specifies
# the multiplication factor for the conversion.
# Finally, additional columns can be used to indicate units and other information.

# %% [markdown]
# All mapping occurs on the index of the original data.
# All mapping occurs on the index of the original data.
# Thus the data to be converted needs to be in long matrix format, at least for the index
# levels which are considered in the conversion.
# TODO: In case conversion happens on MRIO Extensions this conversion happens automatically.
Expand All @@ -69,59 +69,73 @@

# %%
import pandas as pd

import pymrio

ghg_result = pd.DataFrame(
columns=["Region1", "Region2", "Region3"],
index=pd.MultiIndex.from_tuples(
[
("Carbon Dioxide", "Air"),
("Methane", "air"),
]
),
data=[[5, 6, 7], [0.5, 0.6, 0.7]],
columns=["Region1", "Region2", "Region3"],
index=pd.MultiIndex.from_tuples(
[
("Carbon Dioxide", "Air"),
("Methane", "air"),
]
),
data=[[5, 6, 7], [0.5, 0.6, 0.7]],
)
ghg_result.index.names = ["stressor", "compartment"]
ghg_result.columns.names = ["region"]

# %% [markdown]
# Our first task here is to rename to the chemical names of the stressors
# Our first task here is to rename to the chemical names of the stressors
# and fix the compartment spelling.

# %%
# %%
ghg_map = pd.DataFrame(
columns=["stressor", "compartment", "chem_stressor__stressor", "compartment__compartment", "factor"],
data=[["Carbon Dioxide", "[A|a]ir", "CO2", "Air", 1.0],
["Methane", "[A|a]ir", "CH4", "Air", 1.0]
],
columns=[
"stressor",
"compartment",
"chem_stressor__stressor",
"compartment__compartment",
"factor",
],
data=[
["Carbon Dioxide", "[A|a]ir", "CO2", "Air", 1.0],
["Methane", "[A|a]ir", "CH4", "Air", 1.0],
],
)
ghg_map

# %%
# %%
ghg_new = pymrio.convert(ghg_result, ghg_map)
ghg_new

# %% [markdown]
# Explanation: The column headers indicates that the stressor index level
# should be renamed from "stressor" to "chem_stressor" and the compartment index level
# should stay the same (NewName__OldName). The factor column is not used in this case.
# All renaming columns consider regular expressions,
# All renaming columns consider regular expressions,
# so that the spelling of the compartment can be fixed in one go.

# %% [markdown]
# For simple rename (and aggregation cases, see below) we can omit the factor column.
# Thus we obtain the same result with the following mapping table:

# %%
# %%
ghg_map_wo_factor = pd.DataFrame(
columns=["stressor", "compartment", "chem_stressor__stressor", "compartment__compartment"],
data=[["Carbon Dioxide", "[A|a]ir", "CO2", "Air"],
["Methane", "[A|a]ir", "CH4", "Air"]
],
columns=[
"stressor",
"compartment",
"chem_stressor__stressor",
"compartment__compartment",
],
data=[
["Carbon Dioxide", "[A|a]ir", "CO2", "Air"],
["Methane", "[A|a]ir", "CH4", "Air"],
],
)
ghg_map_wo_factor

# %%
# %%
ghg_new_wo_factor = pymrio.convert(ghg_result, ghg_map_wo_factor)
ghg_new_wo_factor

Expand All @@ -136,14 +150,14 @@

# %%
ghg_result_ton = pd.DataFrame(
columns=["Region1", "Region2", "Region3"],
index=pd.MultiIndex.from_tuples(
[
("Carbon Dioxide", "Air"),
("Methane", "air"),
]
),
data=[[5, 6, 7], [0.5, 0.6, 0.7]],
columns=["Region1", "Region2", "Region3"],
index=pd.MultiIndex.from_tuples(
[
("Carbon Dioxide", "Air"),
("Methane", "air"),
]
),
data=[[5, 6, 7], [0.5, 0.6, 0.7]],
)
ghg_result_ton.index.names = ["stressor", "compartment"]
ghg_result_ton.columns.names = ["region"]
Expand All @@ -154,10 +168,17 @@


ghg_map_to_kg = pd.DataFrame(
columns=["stressor", "compartment", "chem_stressor__stressor", "compartment__compartment", "factor"],
data=[["Carbon Dioxide", "[A|a]ir", "CO2", "Air", 1000],
["Methane", "[A|a]ir", "CH4", "Air", 1000]
],
columns=[
"stressor",
"compartment",
"chem_stressor__stressor",
"compartment__compartment",
"factor",
],
data=[
["Carbon Dioxide", "[A|a]ir", "CO2", "Air", 1000],
["Methane", "[A|a]ir", "CH4", "Air", 1000],
],
)
ghg_map_to_kg

Expand All @@ -169,7 +190,6 @@
# TODO: unit conversion extensions



# %% [markdown]
# ## Characterization

Expand All @@ -184,29 +204,30 @@
# An simple example is a conversion/aggregation based on GWP100 characterization factors.
# Here, we continue with the unit converted and cleanup dataframe from above:

# %%
# %%
ghg_new_kg


# %% [markdown]
# We define a general purpose characterization map for GHG emissions
# (based on
# We define a general purpose characterization map for GHG emissions
# (based on
# [AR6 GWP100 and GWP20 factors](https://www.ipcc.ch/report/ar6/wg1/downloads/report/IPCC_AR6_WGI_Chapter07.pdf)
# ,with some simplifications):

# %%
GWP_characterization = pd.DataFrame(
columns=["chem_stressor", "GWP__chem_stressor", "factor"],
data=[["CO2", "GWP100", 1],
["CH4", "GWP100", 29],
["NHx", "GWP100", 273],
["CO2", "GWP20", 1],
["CH4", "GWP20", 80],
["NHx", "GWP20", 273],
["CO2", "GWP500", 1],
["CH4", "GWP500", 8],
["NHx", "GWP500", 130],
],
columns=["chem_stressor", "GWP__chem_stressor", "factor"],
data=[
["CO2", "GWP100", 1],
["CH4", "GWP100", 29],
["NHx", "GWP100", 273],
["CO2", "GWP20", 1],
["CH4", "GWP20", 80],
["NHx", "GWP20", 273],
["CO2", "GWP500", 1],
["CH4", "GWP500", 8],
["NHx", "GWP500", 130],
],
)
GWP_characterization

Expand All @@ -216,21 +237,23 @@


# %% [markdown]
# As we can see, GWP_characterization can include factors for stressors not actually
# As we can see, GWP_characterization can include factors for stressors not actually
# present in the data.
# These are silently ignored in the conversion process.
# We also did not specify the compartment and assumed the same factors apply
# We also did not specify the compartment and assumed the same factors apply
# independent of the compartment (we could pass through the compartment to
# the new result table via passing drop_not_bridge=False to the convert function).

# %%
GWP_result_with_comp = pymrio.convert(ghg_new_kg, GWP_characterization, drop_not_bridged=False)
GWP_result_with_comp = pymrio.convert(
ghg_new_kg, GWP_characterization, drop_not_bridged=False
)
GWP_result_with_comp

# %% [markdown]
# All stressors mapped to the same "impact" are first converted via the
# All stressors mapped to the same "impact" are first converted via the
# value given in the factor column
# and then summed up (the aggregation function can be changed
# and then summed up (the aggregation function can be changed
# via the `agg_func` parameter).

# %% [markdown]
Expand All @@ -242,43 +265,51 @@
# (The same principle applies to sector specific factors.)
# For that, we assume some land use results for different regions:

# %%
# %%
land_use_result = pd.DataFrame(
columns=["Region1", "Region2", "Region3"],
index=["Wheat", "Maize", "Rice", "Pasture", "Forest extensive", "Forest intensive",],
data=[[3, 10, 1],
[5, 20, 3],
[0, 12, 34],
[12, 34, 9],
[32, 27, 11],
[43, 17, 24],
],
columns=["Region1", "Region2", "Region3"],
index=[
"Wheat",
"Maize",
"Rice",
"Pasture",
"Forest extensive",
"Forest intensive",
],
data=[
[3, 10, 1],
[5, 20, 3],
[0, 12, 34],
[12, 34, 9],
[32, 27, 11],
[43, 17, 24],
],
)
land_use_result.index.names = ["stressor"]
land_use_result.columns.names = ["region"]
land_use_result

# %% [markdown]
# Now we setup a pseudo characterization table for converting the land use data into
# biodiversity impacts. We assume, that the characterization factors vary based on
# biodiversity impacts. We assume, that the characterization factors vary based on
# land use type and region.

# %% [markdown]
landuse_characterization = pd.DataFrame(
columns=["stressor", "BioDiv__stressor", "region", "factor"],
data=[
["Wheat|Maize", "BioImpact", "Region1", 3],
["Wheat", "BioImpact", "Region[2,3]", 4],
["Maize", "BioImpact", "Region[2,3]", 7],
["Rice", "BioImpact", "Region1", 12],
["Rice", "BioImpact", "Region2", 12],
["Rice", "BioImpact", "Region3", 12],
["Pasture", "BioImpact", "Region[1,2,3]", 12],
["Forest.*", "BioImpact", "Region1", 2],
["Forest.*", "BioImpact", "Region2", 3],
["Forest ext.*", "BioImpact", "Region3", 1],
["Forest int.*", "BioImpact", "Region3", 3],
],
columns=["stressor", "BioDiv__stressor", "region", "factor"],
data=[
["Wheat|Maize", "BioImpact", "Region1", 3],
["Wheat", "BioImpact", "Region[2,3]", 4],
["Maize", "BioImpact", "Region[2,3]", 7],
["Rice", "BioImpact", "Region1", 12],
["Rice", "BioImpact", "Region2", 12],
["Rice", "BioImpact", "Region3", 12],
["Pasture", "BioImpact", "Region[1,2,3]", 12],
["Forest.*", "BioImpact", "Region1", 2],
["Forest.*", "BioImpact", "Region2", 3],
["Forest ext.*", "BioImpact", "Region3", 1],
["Forest int.*", "BioImpact", "Region3", 3],
],
)
landuse_characterization

Expand All @@ -292,7 +323,6 @@
# CONT: start working on convert for extensions/mrio method



# %% [markdown]
# Irrespectively of the table or the mrio system, the convert function always follows the same pattern.
# It requires a bridge table, which contains the mapping of the indicies of the source data to the indicies of the target data.
Expand Down
2 changes: 1 addition & 1 deletion pymrio/__init__.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -69,10 +69,10 @@
from pymrio.tools.ioutil import (
build_agg_matrix,
build_agg_vec,
convert,
convert_to_long,
index_contains,
index_fullmatch,
index_match,
convert,
)
from pymrio.version import __version__
Loading

0 comments on commit f5e7aff

Please sign in to comment.