Skip to content

Commit

Permalink
server : simplify state machine for slot (#9283)
Browse files Browse the repository at this point in the history
* server : simplify state machine for slot

* add SLOT_STATE_DONE_PROMPT

* pop_deferred_task

* add missing notify_one

* fix passkey test

* metrics : add n_busy_slots_per_decode

* fix test step

* add test

* maybe fix AddressSanitizer?

* fix deque ?

* missing lock

* pop_deferred_task: also notify

* Update examples/server/server.cpp

Co-authored-by: Georgi Gerganov <[email protected]>

---------

Co-authored-by: Georgi Gerganov <[email protected]>
  • Loading branch information
ngxson and ggerganov authored Sep 6, 2024
1 parent 134bc38 commit 9b2c24c
Show file tree
Hide file tree
Showing 4 changed files with 147 additions and 93 deletions.
Loading

0 comments on commit 9b2c24c

Please sign in to comment.