UTF-8 input doesn't work

[sternenseemann]

Describe the bug

ad treats all input as ASCII which doesn't match its buffers.

To Reproduce

Open ad, type or paste (via the terminal's clipboard functionality) non ASCII characters.
ad adds garbage to the buffer.

Possible Fixes

I wrote an unpolished and slightly incorrect fix for this (sternenseemann@375b30d) which fills an up to four byte buffer in try_read_char.

However, I think that try_read_char should be replaced since reading input char by char is quite slow (you can already observe that when pasting a large amount of text via the system/terminal clipboard). Instead, the TUI should probably read input in a buffered fashion. If the input handling code was to be rewritten in this fashion, implementing unicode input would be simpler, as a buffer for remaining bytes would already exist.

[sminez]

Ah, yeah the assumption that all characters coming from stdin are a single byte is an oversight from when I first implemented utf8 support. That said, I'm confused as to how you are hitting this through pasting: how are you triggering this? Pasting should directly access the clipboard (currently through xclip though I'm aware that needs improving as per #61) which works fine, and middle clicks should be getting intercepted and handled internally by ad rather than triggering a paste.

Reading input buffered isn't something I'm looking to implement at the current time and I'd need to understand better what the benefits were: as I said, the behaviour of pasting for me doesn't exhibit this behaviour at all.

[sternenseemann]

As I write, I'm not using the clipboard feature in this case. With an TUI editor users will do that occasionally. Besides, you can also just enter unicode characters directly via your keyboard.

when pasting a large amount of text via the system/terminal clipboard