New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Track multiple and swap easily between contexts #5

Open

1 of 4 tasks

getnamo opened this issue Dec 24, 2023 · 1 comment

Open

1 of 4 tasks

Track multiple and swap easily between contexts #5

getnamo opened this issue Dec 24, 2023 · 1 comment

Labels

Owner

getnamo commented Dec 24, 2023 •

edited

Loading

NB: context meaning full prompt history

Swap contexts without tearing down thread and re-starting
Flush contexts to and from disk utility
Enable rollback of single prompt entries (e.g. regenerate concept)
Impersonate syntactic sugar

getnamo added the enhancement label

Owner Author

getnamo commented Feb 6, 2024

Likely need some form of batched inference with different contexts: https://github.com/ggerganov/llama.cpp/tree/master/examples/batched-bench

NB:

Refactor to wrap chat contexts as a list/map of chat name -> chat context + history with ability to rollback replies for e.g. regeneration

getnamo mentioned this issue

Allow changing conversation state via model state #16

Open

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment