Invalid window decoration with/without compositor

[luigifab]

First I discovered, when compositor is enabled, some Mate windows, use dialog instead of window node. So, now I'm trying to fix my theme with and without compositor, with and without GTK_CSD=1, and I found a problem:

---

When compositor is enabled, when I run: GTK_DEBUG=interactive GTK_CSD=1 GTK_BACKDROP=1 mate-window-properties or GTK_DEBUG=interactive GTK_CSD=1 GTK_BACKDROP=0 mate-window-properties, there are no problems (it's a CSD dialog):

I can enable or disable compositor without any problems, except the black background under the window/dialog, but it's probably easy to fix it via composited-changed signal (csd -> solid-csd):

---

When compositor is enabled or When compositor is disabled, when I run: GTK_DEBUG=interactive GTK_CSD=0 GTK_BACKDROP=1 mate-window-properties and GTK_DEBUG=interactive GTK_CSD=0 GTK_BACKDROP=0 mate-window-properties, there are no problems (it's a marco window):

I can enable or disable compositor without any problems.

---

When compositor is not enabled, when I run: GTK_DEBUG=interactive GTK_CSD=1 GTK_BACKDROP=1 mate-window-properties or GTK_DEBUG=interactive GTK_CSD=1 GTK_BACKDROP=0 mate-window-properties, there are no problems (even if it's NOT a CSD window/dialog):

But when I try to enable the compositor, there is something wrong with decoration node:

[luigifab]

I'm not sure for now, but it looks like this is because that Mate window use the solid-csd class with Marco window.

compositor off
GTK_DEBUG=interactive GTK_CSD=0 transmission-gtk -> window.background, without decoration, Marco window, gtk4
GTK_DEBUG=interactive GTK_CSD=0 bleachbit -> window.background.solid-csd, decoration, CSD window, gtk3
GTK_DEBUG=interactive GTK_CSD=0 geany -> window.background, decoration, Marco window, gtk3
GTK_DEBUG=interactive GTK_CSD=0 engrampa -> window.background, decoration, Marco window, gtk3

ok lol, the about dialog of engrampa has solid-csd class

I suppose the problem come from gtk_window_set_titlebar (GTK_WINDOW (dialog), NULL); of GtkDialog, because after it call gtk_window_enable_csd (window);.

[lah7]

Just to note, for the default classic behaviour (no CSDs), you must unset GTK_CSD (not 0)

• No GTK_CSD = Remove CSDs (gtk3-classic default)
• GTK_CSD=0 = Normal GTK 3 behaviour, application decides. Hopefully like a "passthrough" mode.
• GTK_CSD=1 = Forced CSDs. I'm not sure who'd want to do that. Some programs might end up buggy.

You might already know this. I couldn't tell from your testing since it was either GTK_CSD=0 or GTK_CSD=1

[luigifab]

Oh really? No, first news, I didn't read the readme. With =1, for now, I have a problem only caja desktop.

[lah7]

As a reminder to myself:

• GTK_CSD=1 was designed to enable server side shadows.
• GTK_BACKDROP=1 was your patch to enable backdrop (inactive windows looking different) without CSD. (#56)

For what it's worth, KWin (KDE window manager) also has buggy behaviour if I block compositing, start a non-CSD app (with forced CSD) and then switch on the compositor while it was running.

This was ghex 3.41.1. Using X11. Command:

GTK_CSD=1 GTK_BACKDROP=1 ghex

Started with no compositor	Turned on compositor while running	Started with compositor
Has square corners	Starts glitching out then turns into this	Forced CSDs

... probably because the app was never designed for CSDs? ghex doesn't use CSDs normally:

For comparison, I tried gnome-disks 3.38.2 (a CSD app) both with gtk3 and gtk3-classic and there were no issues when toggling compositing on/off.

Being an obscure problem and GTK_CSD=1 being an unsupported configuration, I don't think it's worth trying to fix. I suspect most users won't have GTK_CSD set (server side by default), but might have GTK_CSD=0 set if they need a GTK 3 experience for specific apps or wish to preserve original GTK 3 CSD behaviour for all apps.

[luigifab]

Alright, found!

I made appearance__toggle_csd.patch. But I think in csd__server-side-shadow.patch, it's better to use: if (g_getenv("GTK_CSD") == FALSE).

Crash test with mate-window-properties, bleachbit, engrampa (& gtkinspector) :

compositor disabled on app start, then -> compositor on -> compositor off
GTK_DEBUG=interactive mate-window-properties
GTK_DEBUG=interactive GTK_CSD= mate-window-properties
GTK_DEBUG=interactive GTK_CSD=0 mate-window-properties
GTK_DEBUG=interactive GTK_CSD=1 mate-window-properties

compositor enabled on app start, then -> compositor off -> compositor on
GTK_DEBUG=interactive mate-window-properties
GTK_DEBUG=interactive GTK_CSD= mate-window-properties
GTK_DEBUG=interactive GTK_CSD=0 mate-window-properties
GTK_DEBUG=interactive GTK_CSD=1 mate-window-properties

No black background/border.