Skip to content

Pull requests: vllm-project/vllm

New pull request New
315 Open 2,149 Closed
315 Open 2,149 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

[Hardware][Intel CPU]use ipex varlen attention to compute prompts for better performance.
#5943 opened Jun 28, 2024 by jikunshang Loading…
[core][optimization] use a pool of numpy ndarray to hold seq data
#5942 opened Jun 27, 2024 by youkaichao Loading…
6
[Kernel] Add per-token AZP epilogue
#5941 opened Jun 27, 2024 by ProExpertProg • Draft
[ Misc ] Refactor w8a8 to use process_weights_after_load (Simplify Weight Loading)
#5940 opened Jun 27, 2024 by robertgshaw2-neuralmagic Loading…
6
[Kernel] Raise an exception in MoE kernel if the batch size is larger then 65k
#5939 opened Jun 27, 2024 by comaniac Loading…
3
[Bugfix] Only add Attention.kv_scale if kv cache quantization is enabled
#5936 opened Jun 27, 2024 by mgoin Loading…
[Frontend]openai base64 embedding: remove the message blocker for base64 embedding
#5935 opened Jun 27, 2024 by llmpros Loading…
[Bugfix] Fix compute datatype for cutlass 3.x epilogues
#5931 opened Jun 27, 2024 by tlrmchlsmth Loading…
4
[Kernel] Add punica dimensions for Granite 3b and 8b
#5930 opened Jun 27, 2024 by joerunde Loading…
Pp threads
#5929 opened Jun 27, 2024 by Anirudhaagrawal Loading…
[ Misc ] Remove fp8_shard_indexer from Col/Row Parallel Linear (Simplify Weight Loading)
#5928 opened Jun 27, 2024 by robertgshaw2-neuralmagic Loading…
@comaniac
1
[Distributed] Make it clear that % should not be in tensor dict keys.
#5927 opened Jun 27, 2024 by xwjiang2010 Loading…
1
[Bugfix] fix missing last itl in openai completions benchmark
#5926 opened Jun 27, 2024 by mcalman Loading…
1
[Misc] Extend vLLM Metrics logging API
#5925 opened Jun 27, 2024 by SolitaryThinker Loading…
11
[Frontend] Support for chat completions input in the tokenize endpoint
#5923 opened Jun 27, 2024 by sasha0552 Loading…
[ Bugfix ] Enabling Loading Models With Fused QKV/MLP on Disk with FP8
#5921 opened Jun 27, 2024 by robertgshaw2-neuralmagic Loading…
1
9
[Model] Initial support for BLIP-2
#5920 opened Jun 27, 2024 by DarkLight1337 • Draft
1
[Bugfix] fix detokenizer shallow copy
#5919 opened Jun 27, 2024 by aurickq Loading…
4
[Kernel] Prototype integration of bytedance/flux kernels
#5917 opened Jun 27, 2024 by tlrmchlsmth • Draft
[PoC][Spec Decoding] Stop speculating when confidence is low
#5914 opened Jun 27, 2024 by njhill • Draft
2
[Bugfix] Fix precisions in Gemma 1
#5913 opened Jun 27, 2024 by WoosukKwon Loading…
4
Warn if user max_model_len is greater than derived max_model_len
#5911 opened Jun 27, 2024 by fialhocoelho • Draft
3
[Model] Support for models with head size 120
#5910 opened Jun 27, 2024 by g-eoj • Draft
[Lora] Use safetensor keys instead of adapter_config.json to find unexpected modules.
#5909 opened Jun 27, 2024 by rkooo567 Loading…
1
[VLM][BugFix] Make sure that multi_modal_kwargs can broadcast properly with ring buffer.
#5905 opened Jun 27, 2024 by xwjiang2010 Loading…
@DarkLight1337
ProTip! Type g i on any issue or pull request to go back to the issue listing page.