Engine-agnostic signal contract for model servers on Kubernetes: common metric semantics, engine profiles (e.g., vLLM, SGLang, TensorRT-LLM), a mapping exporter, a conformance suite, and autoscaling integration.
This project is sponsored by:
- Kubernetes SIG Autoscaling
- Kubernetes SIG Instrumentation
- OpenTelemetry GenAI SIG
See the model server contract for the complete metric table, including standard names, types, units, semantics, and engine-specific metric mappings.
The motivation, scope, and design of this project are described in the model-serving-signals proposal doc.
The project website is built from site-src/ with MkDocs and
published via Netlify. site-src/ is the single source of truth for docs;
build locally with make build-docs (or make live-docs to preview).
Learn how to engage with the Kubernetes community on the community page.
You can reach the maintainers of this project at:
Participation in the Kubernetes community is governed by the Kubernetes Code of Conduct.