Popular repositories Loading

1. vllm vllm Public

   Forked from vllm-project/vllm

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python

2. sglang sglang Public

   Forked from sgl-project/sglang

   SGLang is a fast serving framework for large language models and vision language models.

   Python

3. flashinfer flashinfer Public

   Forked from flashinfer-ai/flashinfer

   FlashInfer: Kernel Library for LLM Serving

   Cuda

4. opencompass opencompass Public

   Forked from open-compass/opencompass

   OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

   Python

5. evalscope evalscope Public

   Forked from modelscope/evalscope

   A streamlined and customizable framework for efficient large model evaluation and performance benchmarking

   Python

6. omniserve omniserve Public

   Forked from mit-han-lab/omniserve

   [MLSys'25] QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving; [MLSys'25] LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attention

   C++