Fix topological errors within 380 and 220 kV level with respect to ENTSO-E map

[ulfmueller]

When abstracting the 380 and 220 kV grid some sub grids are occurring which obviously are stating errors mainly due to osm data issues. See this issue for more information on these locations.

Apart from developing and using subgrid connection algorithms we want to use the ENTSO-E grid map in order to fix these errors by changing openstreetmap.org

@ClaraBuettner : I think that could be a fun job for you. What do you think? Do you need any more details? Just contact me when you have time for this.

[ClaraBuettner]

Just to be sure, I should try to connect them by subgrid connection algorithms and if that doesn't work change openstreetmap.org?
I think I can start next week, probably I ask for more details at that time.

[ulfmueller]

the subrid connection algorithm is already implemented by @lukasol ... Your task would be to change osm by looking at the ENTSO-E map. That way the algorithm would not be needed not that much and we get more realistic data and at the same time give that effort back to osm...
yeah just ask for more detail when you are ready...

[ClaraBuettner]

Okay, sorry for misunderstanding. Up to now, I have never worked with osm.
I wanted to start looking for the errors in osm. But I couldn't find an osm-layer for electricity which divides in different voltage levels. Can you help me with this?

[ulfmueller]

The key voltage defines the voltage of a cable in osm. If you take for example this power line from Lübeck to Siems at has a voltage of 110 kV. On the same route there is a 220 kV (ENTSO-E grid map) missing which you could add to openstreetmap. Actually it seems that there is not a line missing (see also satallite pictures like at bing or google) but there is 2 lines with each 6 cables (2 systems). It total there are 4 systems of 110kV but I would think that at least one the systems is a 220kV. If it is like that. It would be an relatively easy change. You would not have to add a new line and think about where to but it (including the poles) but only have to change the voltage level of the particular system from 110000 to 220000...

[ulfmueller]

if your about to have some trips to places it like that in Schleswig-Holstein you could also use and check out this app

[ClaraBuettner]

I just added a 220kV from Lübeck to Siems, hopefully that was right.
Looking for the other busses, it seems that some are transformers (bus_id 25562 and 25564), I couldn't find unconnected busses or lines there. Maybe there is a problem with them?

[ClaraBuettner]

Fix topological errors within 380 and 220 kV.pptx

I've collected everything I've found in a PPP. In some cases I could not see anything causing the error.

[ulfmueller]

Cool, thx! Could you also upload it as a pdf?

[ClaraBuettner]

Fix topological errors within 380 and 220 kV.pdf

[ClaraBuettner]

I would like to fix the errors but, apart from the Lübeck Siems connection, we have not discoussed how I should do this...

[ClaraBuettner]

I just took a closer look at the transformer station in Uchtelfangen. It seems that some lines got the wrong voltage level. Two 220 kV lines are missing in the DB, one of them is as well 110 kV. But in the substation, both are connected via an 110kV line. Satelite pictures show that the line is connecteted to the 220kV busbar, in osm it is also connected to the 110kV busbar, I can't see that on the pictures.
So I guess there is a mistake in osm.

[ulfmueller]

I am sorry I am not sure about the 'take-home-message'. Can you fix what is wrong in osm? Change 110kV to 220kV(did I understand it more-or-less right?)? At the same time do you think you can fix it in the post-processing being part of the data processing (as we discussed it)?

[ulfmueller]

@lukasol I just mention you here so you are informed. Clara and I went through that presentation (see pdf above) which tried to cover the cases which popped up in the context of the ehv clustering.
We decided to introduce a fixing-script in the dp (powerflow part after reading the osmTGmod data) in that way, that it seems right. Only in the cases where it is very sure we would fix it in osm (unfortunately that is the case very seldomly).

[ClaraBuettner]

Yes, I would change the voltage level and delete a line I can't see on satelite pictures.
In addition, I add the line to the script I am writing as we discussed it.

[ClaraBuettner]

I just changed some voltage levels from lines in Uchtelfangen in osm respecting bing and entso-e.

[ClaraBuettner]

I added the script to the feature branch. Up to now there are still some problems.
For the errors in Weiher and Heinitz, the lines will just be added if they havn't exist before.
I tried to do the same for Lübeck, but the missing line has the same geom as an existing line, just with another voltage level. But the voltage level is not defined in the line-table, so I couln't find a possibility to get that running.
Maybe I find another solution, so the file is only temporary.

[ClaraBuettner]

I added electrical parameters to the lines, I took the one for transmission lines, as listed on GitHub.
If it was possible, I chose the number of cables from entso-e, otherwise I implemented 2 systems.
Hopefully thats all right.
Unfortunately, I couldn't find a solution for the line in Lübeck.

[ClaraBuettner]

I was just looking at the problem again. I'm afraid I had to notice that the script I wrote won't work. It deletes buses which are needed for components like generators. In some cases, the power is very small and it can be reconnected to a bus with the same geom, but not always.
I tried to figure out a new method but in some cases it is really difficult to find a good way.
In addition it needs a very long time to test, because I need a new busmap every time I delete or add a bus.

[ClaraBuettner]

Here are some new information about the subnetworks:
The ones in Lübeck, Weiher and Heinitz can be fixed by adding a line, which was already discussed before. For the others I got more information and other solutions.

In Berlin, there are 380kV lines and buses which shouldn't be there in respect to entso-e. Two buses (Umspannwerk Lichtenberg and Umspannwerk Biesdorf Nord) are used to connect a small solar generator and/or loads. In the osm both are tagged as 110kV stations, and there are 110kV buses in our data. That's why I suggest to reconnect the generator and the loads to the 110kV buses and delete the 380kV buses and lines.

The industrial park in Höchst has a subnetwork due to a 220kV line inside. The osm is nor sure, if it's 220kV or 110kV. Looking at the website of the park, it says they only have high voltage and not extra high voltage. In addition there are mostly 110kV buses with the same geom like the 220kV buses. So I would change the voltage of the line to 110kV by reconnecting it to the 110kV buses and delete the 220kV buses.

The subnetwork in Plattling is caused by a line, which looks different in osm. The substation is divided into two parts, one transforming from 380kV to 110kV and one only with 110kV. But in our data, both have an 380kV bus and are connected via a line which doesn't exists in osm and satellite pictures. There is a quite similar line, connecting the substation to the nearest 380kV line, which is missing in our data. I suggest to delete the line between the substation parts and add the missing line.

The subnetwork in Weisweiler is just one 380kV bus. In our data, it's just connected via a transformer, but in the osm, there are connections to the next 380kV line as well. So I would add a line, like in the osm, that connects the 380kV bus to the 380kV line. All buses are already there, I just need to connect them.

When you agree with the suggestions, I would test it local in the next days.

[ulfmueller]

nice @ClaraBuettner . Your suggestions sound reasonable. We just have to be very careful if we change voltage levels of buses. Especially when we change 380/220 kV buses to 110kV buses.
If buses are changed/deleted/added it influences the whole generator/load assignment procedure. So for a clean process these topology adjustments would have to be included in the data processing with care and at early stages. That is probably nothing you could do or wait for in your remaining bachelor thesis time.
So you need a post-processing method - which at least works for your thesis work.
Please check whether the buses you change have loads/generator attached. If they have, the changes are more critical.
changing/deleting/adding lines is zero critical.
How can you proceed? Anyhow you can do as you suggested and test it locally and see if your changes have the desired effect. You can definitely use this for your thesis. It would be nice if you could document your changes in detail also with some screenshots etc. (because it is hard to understand these things without something graphic). And try to check whether the changes you make could have an effect on our assignment procedure and therefore try to estimate the error which is produced by that. Maybe we can meet on this some time e.g on the second half of next week (so this knowledge does not get lost and we check that you do not commit any major error)...

[ClaraBuettner]

For my thesis, I would reconnect the one-ports to existing buses, so it can be used as a post-processing. Most often, there are already 110kV buses when I need them, there is just one 220kV bus I need to change to 110kV.
I think it's just in the industrial park in Höchst where I need to change from 220kV to 110kV. When this is not possible, I could sum the geoms of the connected lines, then I don't have to do so.
I started documenting while writing the script. It's not ready but the problem with Höchst is hopefully
easier to understand..

subnetworks.pdf

[ulfmueller]

Can you check if the 380kV buses which you want to delete have loads and generators connected?

[ClaraBuettner]

I already did it, they are listed in the table as generators, loads and storages

[ClaraBuettner]

I finished the documentation and am currently testing most of the suggested options. Just the one in Pleiting is missing up to now. I need to connect the substation to an existing line, but there is no bus at the position I have to do that. So I'm wondering if pypsa needs lines with the same buses to know that they are connected, or if it's enough that they touch. I suggest the first option, but that would mean I have to split up an existing line. That's why I'm not sure what to do.

subnetworks.pdf

[ulfmueller]

Like you suggest it is not enough if they geographically touch. I think you would have to delete the existing line which you want to split up and then built a new bus and three new lines...

The buses with generators/loads/storages attached are crucial. Can you check how much installed power these units have? It would be good to attach them to the most adequate bus around with respect to our assignment methodology. Maybe @IlkaCu can help you with that.

By the way, are you working with the newest data set (v0.3.0)? The amount of storages in Berlin seem a bit weird to me...

[ClaraBuettner]

It worked! I created a new busmap overnight and could run the eHV-clustring with just one sub-network, which is the one in Plattling I havn't adjusted yet (as you can read above). Looking at the geometries with qGis, everything worked fine. So I just need to find a method for the last one.
I wrote a SQL-script which works like a post processing at the moment, so I can work with it. To use it in the dp, I can easily delete the lines which reconnect one-ports, which won't be needed then.

[ClaraBuettner]

Ok, then I will adjust this like you said.
I will take a look at the installed power by now, in some cases I already did so. But the problem won't be there when is't early integrated in the dp, is it?
Yes, I work with the new data. The reason for the high number of storages (and generators as well) is, that the different scenarios have different ids for generators and storages, so in one scenario there is mostly just one third. Does that make it better?

[ClaraBuettner]

For the generators and storages in Berlin, I somehow made a mistake in the documentation. There is only one 63kW solar generator and two storages with p_nom = 0 in each scenario.
I Höchst, the number of generators per scenario is either 6 (Status Quo, NEP 2035) or 3 (ego 100), and there are two p_nom = 0 storages in each scenario.
The maximum power of a genetrator is 96.5MW, which seems to be to high to connect it in 110kV.
The other sub_networks were fixed without changing buses.
I also updated the documentation.

subnetworks.pdf

[IlkaCu]

@ClaraBuettner: If you like, we can meet this week and discuss how your changes interact with the DP and how we can bring both together.

[ClaraBuettner]

That would be great! Can you tell me when you have time for that?

[ulfmueller]

good and thx @IlkaCu! the storages are extendable storages and therefore do not have p_nom values. They are optimization variables - so you do not have to care about them in this case.

[IlkaCu]

Tomorrow 15:30?

[ClaraBuettner]

Yes, that's fine.

[IlkaCu]

@ClaraBuettner: Is there anything which needs to be added to the branch feature/fix_eHV_subnetworks before merging it back into the dev branch?
If not, I would ask you to create a pull request for this branch.

[ClaraBuettner]

Some lines won't be needed anymore when it's included earlier.
Up to now, it is included in utilities as a post-processing. We talked about adding it right after the osmTGmod to pypsa script. Is that still right?
And I did a small adjustment, too. I will upload it like that soon.