Documentation on how to create/install custom layouts needed

[kermitfrog]

So.. I don't know where else to ask about this...

I have, after a LOT of reading on the internet, managed to somehow get my system to use a custom keyboard layout.

To make this happen, I had to modify/create the following files in /usr/share/X11/xkb/:

• rules/evdev.xml -- to make my layout choosable in KDE system settings
• symbols/pd -- the layout
• symbols/inet -- had to comment out some lines, so they don't overrule my changes for FK13-FK22

This has been working fine for a while now... what bothers me, besides that it took me faaaar too long to figure out how to do it, is that I have to overwrite system files. If I correctly understand something that I read (sorry don't remember the details - probably somewhere on freedesktop.org), xkbcommon also reads stuff from ~/.xkb/ (at least on wayland). So I tried to place my files there as overrides (with a directory structure of rules/symbols), but that didn't do the trick...

So the big question is:

What is the easiest, correct way to define a custom keyboard layout and having it available in KDE system settings (and the GNOME equivalent and so on) without modifying anything outside of '$HOME'?

Is it even possible?

It would also be great if the answer to this were available somewhere on an official xkb-related site - I'm surely not the only one who spent hours looking for a solution.

[whot]

There's a guide here: https://xkbcommon.github.io/libxkbcommon/doc/current/user-configuration.html

And a more long-form post series when all this was added:
https://who-t.blogspot.com/2020/09/user-specific-xkb-configuration-putting.html
This is the last one but it has the backlinks to the whole series.

And, finally, for X11: https://who-t.blogspot.com/2021/02/a-pre-supplied-custom-keyboard-layout.html

[kermitfrog]

Thanks, that first guide is exactly what I was looking for :)

Unfortunately, it still does not work on my system :/. I assume the discoverability problem comes from plasma systemsettings (ack xkbregistry on the source did not find anything - I'll file a bug report later). However I'd expect to be able to load my layout with setxkbmap -layout, which returns with "Error loading new keyboard description" (error code 251).

To be sure, I copied all the config examples from the guide, but no change (even when run inside ~/.xkb).

strace revealed that setxkbmap does not even try to read these files:

[..]
openat(AT_FDCWD, "./rules/evdev-C.lst", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "./rules/evdev.lst", O_RDONLY) = -1 ENOENT (No such file or directory)
[..]
openat(AT_FDCWD, "/usr/share/X11/xkb/rules/evdev-C.lst", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/X11/xkb/rules/evdev.lst", O_RDONLY) = 4
[..]
openat(AT_FDCWD, "/usr/share/X11/xkb/rules/evdev-C", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/X11/xkb/rules/evdev", O_RDONLY) = 4

I installed libxkbcommon-tools, but xkbcli does not seem to offer a way to load the keymap.

Is there any way I can test my keymap from the command line?

[wismill]

Plasma currently does not use xkbcommon registry to discover layouts, so it will not display layouts in ~/.config/xkb.

strace revealed that setxkbmap does not even try to read these files:

setxkbmap is a XOrg tools and is totally independent to xkbcommon. But it will not work anyway: see https://who-t.blogspot.com/2020/09/no-user-specific-xkb-configuration-in-x.html.

[whot]

I installed libxkbcommon-tools, but xkbcli does not seem to offer a way to load the keymap.

Yeah, there are no generic APIs to load a keymap into a wayland compositor, it's specific to the compositor implementation.

Plasma currently does not use xkbcommon registry to discover layouts, so it will not display layouts in ~/.config/xkb.

That should only affect the GUI selection part of the layout right? Once it's set in the config storage it should just work, so you need to find the way to set the config directly rather than going through the GUI. in gnome this would be setting it in gsettings but in plasma... no idea.

[wismill]

Yeah, there are no generic APIs to load a keymap into a wayland compositor, it's specific to the compositor implementation.

@whot Any plan in Wayland?

That should only affect the GUI selection
part of the layout right? Once it's set in the config storage it should just work, so you need to find the way to set the config directly rather than going through the GUI. in gnome this would be setting it in gsettings but in plasma... no idea.

Yes, it works (see: ~/.config/kxkbrc) but then the kbd settings gui is very unstable, so I do not recommend that option unless understanding this. I plan to make Plasma use xkbregistry as soon as my openSUSE receives the Plasma 6.0.2 upgrade.

[whot]

@whot Any plan in Wayland?

"In Wayland" as in "the wayland protocol specification" - no, setting the keymap is a compositor-private implementation detail.

I plan to make Plasma use xkbregistry [...]

🎉

[wismill]

"In Wayland" as in "the wayland protocol specification" - no, setting the keymap is a compositor-private implementation detail.

@whot I mean the protocol, yes. But I do not understand the reasoning: why a request to set a keymap (raw or via RMLVO) cannot be in the protocol?

[whot]

Two reasons:
One is that as a general rule, a display server protocol should have anything input-related being read-only. This includes button mappings, keymaps, LEDs, etc. - these are all things that a Wayland (or any display server) client must not be able to change. In X this was based on trust - we expect our terminal not to change the keyboard layout ever time we open it, but trust in applications is a concept that went out the window in the last few decades...

That's one of the drivers behind flatpaks et al. too - the trust we currently place in applications (e.g. that it doesn't encrypt your home directory on first launch) is no longer necessary once we're sandboxed.

If you think about it, there are no Wayland clients that need to modify the keymap. Those applications that do are configuration tools but which display protocol the compositor uses is irrelevant to them - the only reason Wayland is wanted is because it's convenient. Like in GNOME I have such a configuration tool: gsettings set .... I don't need to care if my current session runs X or Wayland.

The second reason is that these things are can of worms: Wayland only exposes one logical keyboard but that doesn't match the physical setup. A compositor may choose to implement different layouts on different physical keyboard layouts and hide that behind keymap changes on-the-fly. How would you expose this configuration in Wayland? You'd have to expose the whole physical device tree again. But now you're expecting every compositor to do this and handle whatever keyboard configuration you throw at it - which is not going to work and you're back at square 1 with "no tool to work in every wayland compositor".

Also a fun fact: setxkbmap is an X client that reads the XKB configuration on the host the client runs on. But the actual keymap loaded by X is loaded from the host the X server runs on. So in the 1980s style setup where the clients run on a mainframe you can run setxkbmap with a keymap that isn't available on your local X terminal. IIRC there are still warnings in the man pages for this use-case.

Anyway, two anecdotes that should serve as examples for why display-client keyboard configuration isn't necessary useful.

Oh, and last thing: RMLVO is a convenience approach we use but there's no technical reason a compositor cannot allow a specific keymap to be loaded. The Wayland client won't care or know because they only ever get the full key map anyway - with RMLVO information dropped.

[wismill]

Thank you for this insightful answer.

Like in GNOME I have such a configuration tool: gsettings set .... I don't need to care if my current session runs X or Wayland.

I wish there were some standard for such tools, since they all read/set XKB configuration in the end.

[whot]

I wish there were some standard for such tools, since they all read/set XKB configuration in the end.

setxkbmap parses rules files and then uploads the keymap to the server
xkbcomp uploads a compiled keymap based on kccgst to the server
xmodmap modifies the core keymap and is unaware of XKB
xorg.conf.d snippets match on various bits and tell the server (which invokes xkbcomp to compile)
gsettings sets a dconf database which mutter then converts into into the corresponding XKB protocol calls

Even in X there was no standard :)

[wismill]

@kermitfrog Did you get it work? There is also another hack for Plasma.

[kermitfrog]

I didn't expect to start such a lively and informative conversation here :).

@wismill

@kermitfrog Did you get it work? There is also another hack for Plasma.
&
[..] I plan to make Plasma use xkbregistry as soon as my openSUSE receives the Plasma 6.0.2 upgrade.

No, it's not urgent enough to try any new hacks - I'll rather wait for a real fix.

I had a look at the source of the keyboard configuration in plasma and the xkbregistry.h file reference to see if it's something that I can fix myself within a reasonable time. On the first look it seems as rewriting the functions in xkb_rules.cpp should be enough (not sure if it's the best approach though..). Understanding the xkb_registry API might need a while. So if you're already familiar with it, be my guest - you're probably going to get it done faster than me.

@whot

Two reasons: [..]

That explains a lot - thanks :)

---

On a somewhat related note, I am involved in a (lenghty) discussion about how to make multiple input remappers work together, that was started by this post: kermitfrog/inputmangler#2 (comment)

In short: Linux-tools that transform input don't really work together, which is needed for some power users and people with disabillities. So some kind of protocol is needed to enable this - the requirements are collected here: kermitfrog/inputmangler#3

Feel free to join, if you're interested.

[whot]

Understanding the xkb_registry API might need a while.

fwiw, the xkbregistry API is a wrapper around the evdev.xml file so if you understand the XML structure of that file, you'll understand the xkbregistry API quickly too.

[kermitfrog]

fwiw, the xkbregistry API is a wrapper around the evdev.xml file so if you understand the XML structure of that file, you'll understand the xkbregistry API quickly too.

Thanks, that actually helped. Once I really started, it was pretty easy to work with xkb_registry :). I guess I'm just not used to the cumbersome look of C APIs anymore.

I finally got around to write the changes in kcm_keyboard and it seems to work so far (at least the UI; user layouts are now detected properly. Not yet sure how to test applying those changes without possibly breaking my system) and will submit it as an MR soon.

One question though: can I rely on the API to give layouts in an order where the base layout comes first, followed by all it's variants (with nothing else in between), even if multiple files are merged? (If so, I can keep the code a bit simpler)
Or do I need to handle cases where a variant comes before it's base or another layout is in between?

[whot]

can I rely on the API to give layouts in an order where the base layout comes first, followed by all it's variants (with nothing else in between)

I don't think we make that particular promise though the implementation effectively enforces this just the way it works (and how the XML is structured). If you want to file a PR to update the documentation to make that promise that'd be fine with me.

[wismill]

FYI: plasma-desktopMR to use xkbregistry.