refactor: update values and logic for representative transformer designs

[danielolsen]

Pull Request doc

Purpose

Update the logic used to generate transformers and assign parameter values for the HIFLD grid. These parameters are generated by examining real data sets to identify representative values for different voltage combinations (thank you @danlivengood and @YifanLi86 for helping to identify these values).

What the code is doing

• We add two new data files to contain values which are defined based on voltage pairs (transformer_params.csv) and high-side voltages (transformer_impedance_ratios.csv).
• In load.py, we add a helper function to load these files and return a joined dataframe.
• In transmission.py:
  • We refactor the create_transformers function so that it no longer returns just bus pairs, but also adds parameter values. The other big change is that we also take the lines dataframe, which already has capacities assigned, and uses this information to determine the highest-capacity line connected to each transformer. This capacity is used to estimate how many parallel transformers to add to be able to meet this capacity. This is a deviation from the previous logic, which generated a single transformer, keeping the impedance constant based on the previous hard-coded values, and modifying only the transformer capacity to meet the lines capacity, using an assumption about the thermal capacity of connected lines based on their voltage alone.
  • A lower-level helper function estimate_transformers is used to support the refactored create_transformers function. estimate_transformers returns a single Series for each voltage pair describing the parameters of each transformer and the number required, while create_transformers used this information to create N parallel transformers, each matching the given parameters.
  • Based on the new behavior of create_transformers, we make some changes to the existing add_impedance_and_rating function (renaming it to add_lines_impedances_ratings now that it only deals with transmission lines, not transformers) and to build_transmission, so that we have the right parameters in the order that we need them.
• We remove the now-obsolete functions estimate_branch_impedance and estimate_branch_rating.
• In test_transmission.py we update the values expected from the refactored functions, and remove the checks on the now-obsolete functions.

Testing

Unit testing.

Usage Example/Visuals

Usage of build_transmission is unchanged, only internal logic is refactored.

Time estimate

30-60 minutes. This is a draft PR for now, since it relies on #276, but all logic should be complete and ready for review. EDIT: #276 is merged, so this is ready to go.

[danielolsen]

#276 is merged, so this is now ready for a full review.

[rouille]

Do we need to create entries in the Attribution file for the two CSV files?

[danielolsen]

Do we need to create entries in the Attribution file for the two CSV files?

They are self-created values, made by examining a few different data sources, so I don't think so. @BainanXia had the same question about transmission lines: #276 (comment).

[danielolsen]

Thinking a bit more about the logic for determining how many transformers to add, I wonder if our current logic is adding too many. If we have a situation where there are a couple of higher-voltage lines and one lower-voltage line connected to a substation, adding enough transformer capacity to support the highest capacity line perhaps doesn't make sense, if this transformer capacity ends up being much higher than the lower-voltage line. This could also effectively reduce the impedance between the high-voltage side and the low-voltage side, which could lead to unrealistic low-voltage line congestion/violations.

If we trust the transmission line capacities that are being added, perhaps we only need enough transformer capacity to support the smaller of either: the highest-capacity high-voltage line, or the sum of line capacities on the lower-voltage side. Just speculating here, but I think it would be rare to end up with a situation where the total line capacity on the low-voltage side is so large that it requires transformers capable of handling more power than any one high-voltage line.

E.g. one of the current worst congested lines in the HIFLD Western grid is a 66 kV line connected to a substation with 4x 230 kV lines and one other 66 kV line. The 2x 66 kV lines could be fully served by 1x 69/230 kV transformer (each 66 kV line has a capacity of about 82 MW, and each representative transformer is rated for 190 MW), but based on the current logic we add enough transformers to handle the full capacity of one of the 230 kV lines (552 MW), which equals 3x transformers and 1/3 of the effective impedance between the 230 and 66 kV parts of the network, which is liable to create more local congestion issues.

EDIT: this approach seems to only sort of work. There won't be any transformers which are congested due to hitting their power limits before the connected lines, but this doesn't solve the problem of meeting demand which exists at the low-voltage bus itself. We don't know the demand at this stage, because the demand doesn't get added until after the entire transmission network is built, and the plant data is built, such that we don't add demand at plant substations.

This could be a use case for the code in #279: we can go through our current grid-building process (i.e. within create_grid: build_transmission(); build_plant(); assign_demand_to_buses();), and then run the code in #279 to identify any bottlenecks, and if they're transformer-caused, then we add the requisite number of additional transformers in parallel to be able to serve the demand. The reduction in the number of transformers added with this approach is pretty dramatic though: 10,237 fewer transformers (from a starting number of 28,977, or a reduction of 35%). We don't yet have the real bus-level demands to be able to identify real bottlenecks (see #293), but the Pd values assigned by assign_demand_to_buses will hopefully be close enough to make this approach generally work. If the existing const.demand_per_person value is an under-estimation, we can increase it (since the absolute values of Pd aren't ever used anywhere, just the ratios within a zone).

[danielolsen]

The last four commits implement the main idea discussed in #286 (comment). Briefly: instead of just looking at the capacity to support the highest-capacity-line (probably on the high-voltage side), we also look at the required capacity by all of the lines on the low-voltage side, and we add enough transformer capacity to meet the smaller of those two. There are a few select locations where this results in inadequate capacity where there's demand at the low-voltage bus, but that's difficult or impossible to solve a priori at grid-build time (since not all of these bottlenecks are on the radial part of the grid that #279 analyzes), so we add one more data file of assumptions for which substations need additional transformer capacity. These assumptions are identified pretty quickly in ERCOT with only an iteration or two of power flow solves, compared to the line upgrades which currently require many iterations and are therefore the main effort required to getting to a feasible grid (although maybe having more reasonable impedances between the higher- and lower-voltage parts of the grid will help mitigate this).

The numbers:

• Transformers to support the highest-capacity line power: 28,977
• Transformers once we limit to the sum of capacities on the low-voltage side: 18,740
• Additional transformers added via explicit assumptions: 8
• Other adjustments required to regain ERCOT feasibility using this new transformer logic:
  • 1 additional transmission line
  • 1 more line voltage assumption
  • 5 more line design assumptions
  • 2 more transformers automatically created due to new lines/voltage assumptions.

[BainanXia]

Probably I need to refresh my brain on the current logic of adding transformers: taking the example above, one substation is connected by 6 lines, 2x 66kV (82MW each) and 4x 230kV (552MW each). In this case, how many buses are there in this substation? Should it be 2 given there are 2 different voltage levels among the lines connected to it? If so, there should be only one transformer added, i.e. 66kV/230kV (forgot about the capacity calculation, did we choose the max(82, 552) before?)

Another question not related to transformers, how do we figure out that new transmission line needs to be added during feasibility tweaking? In the adjustments required to regain ERCOT feasibility, I understand the last 3 bullets but a little surprise on the first one.

[danielolsen]

Probably I need to refresh my brain on the current logic of adding transformers: taking the example above, one substation is connected by 6 lines, 2x 66kV (82MW each) and 4x 230kV (552MW each). In this case, how many buses are there in this substation? Should it be 2 given there are 2 different voltage levels among the lines connected to it? If so, there should be only one transformer added, i.e. 66kV/230kV (forgot about the capacity calculation, did we choose the max(82, 552) before?)

You are correct that there are 2 buses since there are 2 voltage levels.

• Before this PR: every transformer's power rating is a multiple of 800 MW where the multiple is determined based on the thermal rating of the default tower/conductor design for the higher voltage.
• At the start of this PR: we introduce representative transformer designs (x, r, rateA) for each voltage pair. We first determine the real power rating for each line using line geometries, then we pick the highest-capacity line connected to either end of the bus pair and add enough of the representative transformers to meet that level of power transfer.
• Now: to determine the maximum amount of power that we need to support between bus pairs, instead of just looking at the highest-voltage capacity line, we look at the smaller of a) the highest capacity line or b) the sum of powers of all lower-voltage lines.

Another question not related to transformers, how do we figure out that new transmission line needs to be added during feasibility tweaking? In the adjustments required to regain ERCOT feasibility, I understand the last 3 bullets but a little surprise on the first one.

This line wasn't strictly necessary, but when looking at real maps I noticed that there should definitely be a line here, but wasn't. So it was manually noticing a data gap, rather than inferring a data gap or data mistake based on power flow results.

[BainanXia]

@danielolsen Very clear. Thanks for the detailed explanation. Now I remember everything:)

[danielolsen]

@danielolsen Very clear. Thanks for the detailed explanation. Now I remember everything:)

I had to go back and look at the before/after code to remind myself. I also realized that we hadn't deleted the old parameters which are now unused, so I just did that too.

[BainanXia]

Thanks for the test which helps a lot to verify the logic.