Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal to change the default keybind from ctrl+space to alt+space to fully fix issue #3 #51

Open
ErrorCatDev opened this issue Nov 26, 2024 · 7 comments

Comments

@ErrorCatDev
Copy link
Contributor

The current default keybind is ctrl+space, but I feel that if we change it to alt+space, we would be able to solve known issue #3.

Investigations

Writing Tools with the ctrl+space keybind does not work on the following types of apps:

  • UWP/WinUI3/XAML Islands
  • Win32 Apps using standard UI Controls

When using it with all these apps, the text will be deleted before it can be copied

1.mp4
2.mp4

Writing tools sometimes work with apps like Word with custom-drawn UI elements (the document itself)

However, all the text gets formatted, causing the program to be intrusive. There are many wacky keybinds for different apps, so it will be hard for us to find an optimal keyboard shortcut.

3.mp4

Writing tools always work on the web

Why alt+space is different

Alt+space brings up the system menu, a low-level feature from Windows 1.0. Hardly anyone uses the system menu, and apps do not have any shortcuts linked to alt+space.

Moreover, alt+space gets processed by the window manager before the actual program does, so we have a chance to intercept it before it gets sent to the program's text field, ensuring WritingTools works all the time.

Some sources on alt+space being run before the message gets sent to the application: microsoft/terminal#10799

Benefits

  • Nobody uses this keyboard shortcut, so nobody will need to change their workflow
  • No apps uses this keyboard shortcut, so this keyboard shortcut will not be a copy
  • This keyboard shortcut gets processed before it gets sent to the program, allowing us to intercept it

Roadmap to implement

Changing the keybind

If we change the keybind on the current app, it will not work as the System Menu gets called first. Since the system menu is a very old API and requires C++ to change it, many applications do not, leaving Alt+Space as a free keyboard shortcut and a double-edged sword. I'm sure we all also don't want to be messing with goofy Windows APIs from 1995 when we can be adding more features to this program using Python lol.

Implement alt+space

My suggestion is to bundle AutoHotKey 1.0 with each installation of writing tools. AutoHotKey was written in C++ and works closely with the windows APIs. By running this script, we can disable the default functionality of alt+space and map it to our desired functionality, to open the program.

!Space:: return

Size of executable

AutoHotKey 1.0 is very lightweight (1mb). My suggestion is to have an installer for AutoHotKey and WritingTools. This will both allow us to always run WritingTools on startup, and to use alt+space for the shortcut.

Conclusion

I strongly believe that this implementation can help bring this project with huge potential to greater heights. Not only can we make WritingTools appear 100% of the time, but we can also ensure that this keyboard shortcut does not interfere with anyone's workflow or a program s keyboard shortcut. This can, with the installer, also improve UX as the user does not need to manually drag the program to shell:startup but it does it automatically for them.

Please feel free to share your thoughts below, it will be much appreciated! If not, I'll probably link some pull requests here in the coming days as I already have a prototype working.

@user1823
Copy link

Nobody uses this keyboard shortcut, so nobody will need to change their workflow

Alt + Space is the default shortcut of PowerToys Run.

@theJayTea
Copy link
Owner

Hello @ErrorCatDev!

This is so comprehensive; thank you so much for your willingness to help bolster Writing Tools.

While I am not super free the next few days, I will come back to this in more detail.


It's very curious that Writing Tools does not work for you on UWP/WinUI3/XAML Islands, and Win32 Apps using standard UI controls. It's been working for me on all those types of apps, on multiple PCs (Win 10 and 11), including the exams you screen-recorded there. I wonder if other people have your issue too, but there haven't been any reports. #3 is actually different from the issue you showed, as you will soon see.

The way Writing Tools works as a broad, abstracted overview is this:

  • You select text.
  • When you invoke the writing tools shortcut, it internally saves your existing clipboard content to a temporary variable, and then simulates a Ctrl+C.
  • Now that your clipboard has the text, it sends it to the LLM accordingly and gets the response.
  • The response is pasted back with a simulated Ctrl+V keypress, and then uses an API to replace the clipboard back with the previously saved variable.

Issue #3 was caused by the Ctrl+C keypress happening and then Writing Tools trying to read the clipboard mere milliseconds after that, before the clipboard could update on some (slower?) PCs (especially in MS Word, which updated the clipboard slower). Thus, it thought there was nothing in the clipboard/selected ("no text selected" in the earlier versions and the mini chat popup in the more recent versions).

This was fixed with PR #39, with a system that automatically retries with a longer sleep timer if the PC was too slow/unreliable to work with the original sleep timer.

Conclusion (for TLDR):
What you showed with the text getting removed in your screen recording is actually a different issue than #3 that is worth exploring.

Would you mind running Writing Tools from source (with python main.py), according to these instructions and sharing the terminal log before and after the text disappearing bug happens?

Thank you again.


Regardless of the above investigation, the AutoHotkey suggestion is interesting. I'm not happy with the existing Python keyboard shortcut API (known bug #1 in the README). I'll look into this for the future.

Have a great day!

@theJayTea
Copy link
Owner

You also mentioned an issue with formatting being cleared in Word, which can easily be fixed by unmapping the "Clear Formatting" default keyboard shortcut in Word away from Ctrl+Space.

Here's how:
image

@ErrorCatDev
Copy link
Contributor Author

ErrorCatDev commented Dec 23, 2024

It's very curious that Writing Tools does not work for you on UWP/WinUI3/XAML Islands, and Win32 Apps using standard UI controls. It's been working for me on all those types of apps, on multiple PCs (Win 10 and 11), including the exams you screen-recorded there.

Hmm, interesting. For me and the other PCs I have used, Writing Tools does not work with the ctrl+space keyboard combination. Can I check if running WritingTools with elevated privileges is needed?

... Issue #3 was caused by the Ctrl+C keypress happening and then Writing Tools trying to read the clipboard mere milliseconds after that, before the clipboard could update on some (slower?) PCs (especially in MS Word, which updated the clipboard slower). Thus, it thought there was nothing in the clipboard/selected ("no text selected" in the earlier versions and the mini chat popup in the more recent versions).

Ahh I see. Thanks for your detailed explanation of how WritingTools work! It will really help me when I navigate the codebase in the future. :-)

Would you mind running Writing Tools from source (with python main.py), according to these instructions and sharing the terminal log before and after the text disappearing bug happens?

It appears that when I press ctrl+space, the space key gets registered but the ctrl key does not. Essentially I was only pressing the space key, therefore the selected text was empty.

DEBUG - Initializing WritingToolApp
DEBUG - Loading config from config.json
DEBUG - Config loaded successfully
DEBUG - Config found, setting up hotkey and tray icon
DEBUG - Creating system tray icon
DEBUG - Tray icon dark
DEBUG - Tray icon displayed
DEBUG - Registering hotkey
DEBUG - Registering global hotkey for shortcut: <ctrl>+<space>
DEBUG - Hotkey registered
DEBUG - triggered hotkey
DEBUG - Hotkey pressed
DEBUG - Cancelling current provider's request
DEBUG - Showing popup window
DEBUG - Clipboard backup: "python pyinstaller-build
DEBUG - Simulating Ctrl+C
DEBUG - Waited 0.2s for clipboard
DEBUG - Selected text: ""
DEBUG - No text captured, retrying with longer sleep
DEBUG - Clipboard backup: "python pyinstaller-build
DEBUG - Simulating Ctrl+C
DEBUG - Waited 0.5s for clipboard
DEBUG - Selected text: ""
DEBUG - Selected text: ""
DEBUG - Creating new popup window
DEBUG - Initializing CustomPopupWindow
DEBUG - Setting up CustomPopupWindow UI
DEBUG - CustomPopupWindow 
DEBUG - CustomPopupWindow shown. Geometry: PySide6.QtCore.QRect(534, 216, 386, 93)
DEBUG - Popup window moved to position: (936, 862)
DEBUG - Stopping the listener
DEBUG - Exiting application

Regardless of the above investigation, the AutoHotkey suggestion is interesting. I'm not happy with the existing Python keyboard shortcut API (known bug #1 in the README). I'll look into this for the future.

The reason I suggested to have an installer for WritingTools with AutoHotkey is so that alt+space does not interfere with the keyboard shortcut WritingTools is registering, but I have found more benefits over the month. Recently, newer PCs (including the one that I just got) are shipping with the Copilot key, but it only allows for UWP/WinUI3 apps remap I think. Since this key sends the Left Shift + Win+ F23 keyboard combo, we can also use AutoHotKey to open WritingTools instead of Copilot. Moreover, we can allow the user to choose if they want to start WritingTools on boot during the installation, making it more userfriendly.

(Apologies for late reply, my PC broke down for 2 weeks and I had to buy a new one, and I was on holiday for another week so I couldnt get the log results from my computer)

@theJayTea
Copy link
Owner

theJayTea commented Dec 24, 2024

Hey @ErrorCatDev ! Thanks for the reply, and merry Christmas!

I see, I’ve not really seen anyone else having such an issue with ctrl+Space

The only concerns I have with AutoHotkey is that it would break straightforward Linux compatibility, and in addition, it would be a little messy to get the user to run 2 programs in the background (they may see the taskbar tooltip of both Writing Tools and AutoHotkey, which may be weird).

Should more people have issues with the current keyboard implementation, we could look into this further, but for now I’m inclined to believe the existing implementation works well enough :3

@ErrorCatDev
Copy link
Contributor Author

ErrorCatDev commented Dec 25, 2024

@theJayTea I see, totally understand where you are coming from. I'm rather interested on this topic and may dive deeper into this to see if I could make an implementation without AutoHotKey, when I have time, so I'll keep this thread open.

Meanwhile, could I confirm that WritingTools for you with ctrl+space works on specifically notepad.exe? If you could make a short screen recording of it working (preferably with the logs) , it will really help me in troubleshooting.

Thank you, and Merry Christmas to you too! :)

@theJayTea
Copy link
Owner

theJayTea commented Dec 25, 2024

Thanks @ErrorCatDev !

By the way, as an aside, alt+space doesn't work for me (on multiple PCs) as the moment I press it, it brings up this Windows context menu, and the text loses focus:
image

Here's the screen recording of ctrl+space & Notepad for you:

WT.in.Notepad.with.Default.Hotkey.mp4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants