Pinned Loading

1. NVIDIA/TensorRT-LLM NVIDIA/TensorRT-LLM Public

   TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. Tensor…

   Python 14.1k 2.6k

2. lightseekorg/tokenspeed lightseekorg/tokenspeed Public

   TokenSpeed is a speed-of-light LLM inference engine.

   Python 1.6k 189

3. lightseekorg/smg lightseekorg/smg Public

   Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing,…

   Rust 391 116

4. ai-dynamo/dynamo ai-dynamo/dynamo Public

   A Datacenter Scale Distributed Inference Serving Framework

   Rust 7.5k 1.3k