You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
2. Please use English, otherwise it will be closed.
Motivation
Currently, SGLang supports LoRA via the generate API, but it is not accessible when using the OpenAI Chat Completion API. Adding LoRA support to the OpenAI Chat Completion API would enable users to leverage LoRA functionality consistently across both APIs, providing a more seamless experience. https://github.com/sgl-project/sglang/blob/main/python/sglang/srt/openai_api/protocol.py#L257
vLLM currently provides LoRA support for v1 Chat Completion, demonstrating how LoRA can be integrated to improve training efficiency in language models. Examining vLLM's implementation may provide insights or code references that could assist in adding this feature to SGLang.
The text was updated successfully, but these errors were encountered:
Checklist
Motivation
Currently, SGLang supports LoRA via the generate API, but it is not accessible when using the OpenAI Chat Completion API. Adding LoRA support to the OpenAI Chat Completion API would enable users to leverage LoRA functionality consistently across both APIs, providing a more seamless experience.
https://github.com/sgl-project/sglang/blob/main/python/sglang/srt/openai_api/protocol.py#L257
Related resources
vLLM LoRA Support for v1 Chat Completion: https://github.com/vllm-project/vllm/blob/main/vllm/entrypoints/openai/serving_completion.py#L38
vLLM currently provides LoRA support for v1 Chat Completion, demonstrating how LoRA can be integrated to improve training efficiency in language models. Examining vLLM's implementation may provide insights or code references that could assist in adding this feature to SGLang.
The text was updated successfully, but these errors were encountered: