You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
The interface lags after a conversation gets a little long (around 10-20 messages with full code text, say about 5000 tokens in each message). When this happens, the chat box seems to slowly catch up to my typing and can take several seconds to start showing up. The problem goes away if I start a new chat...so I'm suspecting the history of the chat
To Reproduce
Use with a local LLM running on oobabooga
Expected behavior
No change in experience the longer the history gets, at least not in the interface, I can understand response from LLM may lag or even break if token limits are reached, but I don't have an issue with the response. As soon as the lag goes away and my text is in the box, I can hit enter and get the same token rate I get when it's not lagging...
Screenshots
N/A
API Provider
oobabooga
Chat or Auto Complete?
Chat
Model Name
any
Desktop (please complete the following information):
OS: Mac Os
The text was updated successfully, but these errors were encountered:
Can confirm this also on Mac using Twinny chat with a locally networked Ubuntu system with ollama serving llama3;
Text input starts lagging once tokens reach ~400 and ~2000 chars.
The "Code Helper (Renderer)" process spikes in CPU usage ~80%+ until the typed text has caught up, and then drops to idle around ~40%.
The lag doesn't recover unless a new chat is started, and instantly kicks in again if the old chat is loaded from hisotory.
No local ollama installed FWIW.
Temp workaround is to instruct the bot to state a warning when the tokens/char count reach the threshold (probably different for each system) then switch to a new chat.
Describe the bug
The interface lags after a conversation gets a little long (around 10-20 messages with full code text, say about 5000 tokens in each message). When this happens, the chat box seems to slowly catch up to my typing and can take several seconds to start showing up. The problem goes away if I start a new chat...so I'm suspecting the history of the chat
To Reproduce
Use with a local LLM running on oobabooga
Expected behavior
No change in experience the longer the history gets, at least not in the interface, I can understand response from LLM may lag or even break if token limits are reached, but I don't have an issue with the response. As soon as the lag goes away and my text is in the box, I can hit enter and get the same token rate I get when it's not lagging...
Screenshots
N/A
API Provider
oobabooga
Chat or Auto Complete?
Chat
Model Name
any
Desktop (please complete the following information):
The text was updated successfully, but these errors were encountered: