My ALT key gets stuck when switching between ttys

[clst]

Sometimes (happens not often in /bin/login almost alway in bash) my ALT key stays pressed in kmscon. Switching to a different tty running getty it works as expected.

It happens when switching with CTRL+ALT+Fx, ALT+Fx and ALT+Arrowkeys.
I haven't found a way to use the kmscon after this happens except to restart it. End Session hotkey still works but doesn't fix it. Restarting kmscon fixes it.

How would I try to debug this? There is no obvious output. And it also happens when started from bash.

EDIT: Funky enough, once I run Xorg on tty1 it doesn't seem to happen anymore (or I haven't triggered it yet) even when switching between the other gettys.