Continue button for Claude and other models #3079
peixotorms
started this conversation in
Feature Requests & Suggestions
Replies: 1 comment
-
This is done programmatically for OpenAI-like endpoints, if they return a context limit "finish_reason." It looks like Anthropic now has (they didn't before) stop reasons, so we can utilize this similarly, if it's available with streaming: https://docs.anthropic.com/en/api/migrating-from-text-completions-to-messages#stop-reason
I recommend making a preset and configuring max output tokens to a higher amount that won't result in an API error. It will make more space for the response and may actually make the AI respond longer. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi,
I installed LibreChat a couple of days ago, and there are a few issues I noticed that maybe can be improved.
When doing coding, it often exceeds the limit in tokens, so for OpenAI there is a "continue" button, and it works in some cases, but not all.
I found several cases where the text randomly stops in the middle of coding, and it just hangs there without response and without continue button. The stop button is available, meaning it's still waiting for something.
First, I would like to see some timeout on the response to stop this from happening. If there is no answer then it should stop.
If the tokens are at the limit, then the continue button should show up.
Currently, when asking to consolidate a big css file, it sometimes works or just hangs and never shows the continue button on OpenAI.
When using Claude for the same task, it just cuts and there is no continue button.
I would like to see the continue button on all models, perhaps looking at the response for the tokens used to understand if it needs it or not. If we used 100% of the tokens, probably it needs a continue button.
Second, when messages are cut with code boxes, and we write continue, the system doesn't automatically continue with formatted boxes. I think something should exist to check if the previous prompt ended with 100% tokens and if it was a code box... then likely the next should also be code, or better... continue by adding to the last box, like what chatgpt does.
Currently on Claude:
https://share.raisercdn.com/raul/MoBEcOZa22.png
And more importantly, at the end it seems to believe it's SQL.
https://share.raisercdn.com/raul/qEveFolI88.png
Beta Was this translation helpful? Give feedback.
All reactions