Skip to content

Issues: vllm-project/vllm

[Roadmap] vLLM Roadmap Q3 2024
#5805 opened Jun 25, 2024 by simon-mo
Open 5
Virtual Office Hours: July 9 and July 25
#5937 opened Jun 27, 2024 by mgoin
Open
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

977 Open 2,239 Closed
977 Open 2,239 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[Bug]: Illegal memory access for MoE kernel with large workloads bug Something isn't working
#5938 opened Jun 27, 2024 by comaniac
1
Virtual Office Hours: July 9 and July 25 misc
#5937 opened Jun 27, 2024 by mgoin
[New Model]: Florence-2 new model Requests to new models
#5934 opened Jun 27, 2024 by localbarrage
[Bug]: FP8 checkpoints with fused linear modules fail to load scales correctly bug Something isn't working
#5915 opened Jun 27, 2024 by mgoin
1
2
[Bug]: TRACKING ISSUE: CUDA OOM with Logprobs bug Something isn't working
#5907 opened Jun 27, 2024 by robertgshaw2-neuralmagic
[Bug]: Internal Server Error when hosting Salesforce/SFR-Embedding-Mistral bug Something isn't working
#5906 opened Jun 27, 2024 by markkofler
[Bug]: TRACKING ISSUE: AsyncEngineDeadError bug Something isn't working
#5901 opened Jun 27, 2024 by robertgshaw2-neuralmagic
1 of 10 tasks
[Misc]: How can I serve multiple models on a single port using the OpenAI API? misc
#5899 opened Jun 27, 2024 by SuiJiGuoChengSuiJiGuo
3
[Bug]: Inconsistent Responses with VLLM When Batch Size > 1 even temperature = 0 bug Something isn't working
#5898 opened Jun 27, 2024 by gjgjos
1
[Bug]: Incoherent error message when using MLPSpeculator and num_speculative_tokens is set too high bug Something isn't working
#5893 opened Jun 27, 2024 by tdoublep
[Feature]: Add distributed inference support for lora adapters. feature request
#5891 opened Jun 27, 2024 by xuanyaoming
[Bug]: Query with logprobs and echo crashes vllm (llama-3-8b-instruct) bug Something isn't working
#5890 opened Jun 27, 2024 by yaronr
1
[Installation]: Wrong torch header file referenced when compiling from source installation Installation problems
#5889 opened Jun 27, 2024 by crazy-JiangDongHua
[Bug]: Exception in ASGI application bug Something isn't working
#5881 opened Jun 27, 2024 by houshuai-cs
7
[Bug]: vllm stuck when using prompt_token_ids and setting prompt_logprobs bug Something isn't working
#5872 opened Jun 26, 2024 by xinyangz
1
2
[Bug]: LLaVa Next Value Error - "Incorrect type of image sizes" when running in Docker bug Something isn't working
#5868 opened Jun 26, 2024 by FennFlyer
@ywang96
1
[Bug]: The end_sync operation inside the cross_device_reduce_2stage kernel sometimes deadlocks because it can't wait for the end signal. bug Something isn't working
#5866 opened Jun 26, 2024 by JiantaoXu
[Usage]: Can I get the streaming output when using offline inference? usage How to use vllm
#5862 opened Jun 26, 2024 by jiangjiadi
1
[Bug]: ray version problem bug Something isn't working
#5861 opened Jun 26, 2024 by zxcdsa45687
@Yard1
2
[Feature]: support logging input and output feature request
#5859 opened Jun 26, 2024 by NiuBlibing
[Feature]: Supported For Quanto Models feature request
#5858 opened Jun 26, 2024 by RanchiZhao
[Misc]: CUDAGraph captured generation stuck with custom_all_reduce and tensor_parallel=2 misc
#5854 opened Jun 26, 2024 by nuzant
2
[Bug]: server error when hosting TheBloke/Llama-2-7B-Chat-GPTQ with chunked-prefill bug Something isn't working
#5853 opened Jun 26, 2024 by George-ao
1
[Bug]: OutOfMemoryError when loading a small model with a huge context length bug Something isn't working
#5847 opened Jun 25, 2024 by alugowski
3
[Feature]: Support in distributed speculative inference feature request
#5835 opened Jun 25, 2024 by keyboardAnt
ProTip! Adding no:label will show everything without a label.