🥁 Local & Other Cloud LLM Support via OpenAI API, Chat Mode, and More! | v3

This is a colossal update!

1. 🎉 Writing Tools now supports local & other cloud LLMs! This is thanks to a huge refactor by @CameronRedmore 💛

It can now work with ANY OpenAI-Compatible API, giving you access to:

• Locally hosted LLMs (via Ollama, llama.cpp, KoboldCPP, TabbyAPI, vLLM, etc.)
• Other cloud LLMs that have OpenAI-compatible APIs (ChatGPT, Mistral AI, Anthropic, etc.)

This is exciting stuff!

The existing onboarding flow with the Gemini API is still an option, giving you easy access to a free cloud model that's still extremely intelligent.

2. 💬 There's a new chat mode when you invoke writing tools with no text selected!

(Credit: @CameronRedmore and @theJayTea )

3. 🚀 The Writing Tools options now work significantly faster due to optimisations! (Credit: @theJayTea )

4. 🧑🏼‍💻 You can now select code and ask Writing Tools to work on it (fix it, improve it, convert languages...) through Describe your change.... (Credit: @theJayTea )

5. 🐇 Streamed Responses (output starts showing up as soon as it's ready instead of all at once) is now an option, although it's highly experimental (credit: @CameronRedmore)

6. 🖌️ There's new on-boarding, Settings, and About UX! (Credit: @CameronRedmore and @theJayTea )

7. 🔧 Build Script improvements (Credit: @theJayTea )

This wouldn't be possible without @CameronRedmore's extensive work — thank you, truly, for being such an amazing contributor!

---

⬆️ Update Instructions:

Delete and re-download Writing Tools, as the config file format has changed.

🐞 Known Issues

1. (Potential fix being investigated) On some devices, Writing Tools does not work correctly with the default hotkey.
   To fix it, simply change the hotkey to ctrl+` or ctrl+j and restart Writing Tools.
   PS: If a hotkey is already in use by a program or background process, Writing Tools may not be able to intercept it. The above hotkeys are usualy unused.

2. (Fix almost ready!) If you lock and unlock your PC, Writing Tools may not be able to detect your hotkey unless you relaunch it.

3. (Fix almost ready!) Writing Tools may not work in Microsoft Word.

4. The initial launch of the Writing Tools.exe might take unusually long — this seems to be because AV software extensively scans this new executable before letting it run. Once it launches into the background in RAM, it works instantly as usual.