Cell.write and Layout.write are very slow on Windows if there are gigantic polygons in the cell

[sebastian-goeldi]

Without further ado, here's a small test case

import klayout.db as kdb

ly = kdb.Layout()
test_cell = ly.create_cell("HUGERING")

p_outer = kdb.DPoint(100, 0)
p_hole = kdb.DPoint(96, 0)

poly = kdb.DPolygon(
    [
        kdb.DCplxTrans(1, 360 * angle / 100_000, False, 0, 0) * p_outer
        for angle in range(100_000)
    ]
)
poly.insert_hole(
    [
        kdb.DCplxTrans(1, 360 * angle / 100_000, False, 0, 0) * p_hole
        for angle in range(100_000)
    ]
)

test_cell.shapes(ly.layer(1, 0)).insert(poly)
test_cell.write(
    "test.gds"
)  # super fast on linux (~1.1 sec), super slow on windows (~29 sec)

On Windows this is incredibly slow compared to Linux (tested on arch and debian) - 29+ sec vs 1.1 sec on linux

This also is the case for ly.write.

Interestingly if the polygon is split with the new break_ function first, it's similar speed on both platforms:

polys = poly.break_(4000, 0)
for poly in polys:
     test_cell.shapes(ly.layer(1, 0)).insert(poly)
test_cell.write("test.gds")

I suspect write is getting compiled funnily on Windows and is doing something very inefficient that probably will not matter much if the polygon only needs splitting very few times for the GDS.

Initially discovered here gdsfactory/gdsfactory#3088

[klayoutmatthias]

Thanks for bringing this up.

I guess you are referring to the binaries on PyPI, right? These are made with MSVC while the standalong binary is made with gcc/MINGW.

The problem could also be the STL implementation. The interesting part is that using the break function explicitly solves the problem. Basically, the writer uses a very similar function internally.

Matthias

[sebastian-goeldi]

Yes, correct, this is for the PyPi version (just tested it in the GUI, problem doesn't exist there). Sorry, should have mentioned that (still not used klayout.db being available in the GUI :P).

I assumed the writer would use something very similar (I assume essentially it's with different settings), therefore I tested it and brought it up here.

Hope this makes the bug hunting and squashing a bit easier ;)

Best,
Sebastian

[klayoutmatthias]

I suspect it is more an issue of the merge step that normalizes polygons with holes to simple polygons. This is a safety feature as non-merged polygons are likely to get spoiled by the splitting algorithm.

[klayoutmatthias]

Well ... looks like the std::deque implementation is more efficient on Linux.

I tried to fix it, but the effect is the opposite. Now Linux is also slow :( ... but on the positive side, it is easier to debug on Linux.

To be continued ...

[klayoutmatthias]

std::list::splice works miracles.

Now Linux is 3x faster than before and Windows is comparable. Maybe slightly slower, but I can't check because I don't have identical machines to run comparisons. At least the script now executes in 1.7s in my (old i5) Windows box.