Inference Playbooks

Deployment playbooks, manifests, benchmark scripts, and guides for serving large language models on OpenShift/Kubernetes with GPU accelerators.

Directory Convention

models/<model>/<framework>/<version>/<topology>/
                                     ├── manifests/
                                     ├── guides/
                                     ├── benchmarks/
                                     └── results/

• model — model family and quantization (e.g. glm-5.2 for GLM-5.2-FP8)
• framework — serving framework (vllm, llm-d, rhoai)
• version — framework version (e.g. v0.23.0)
• topology — deployment shape (single-node, multi-node-lws)
• model-ops/ — framework-agnostic download/sync jobs, at model level

Models

Model	Framework	Versions	Start Here
GLM-5.2-FP8	vLLM	v0.23.0	Deployment Guides
GLM-5 / GLM-5-FP8	vLLM	latest	Model Ops
GLM-5.2-FP8	RHOAI	3.5	Deployment Guides

Planned

• llm-d — prefix-cache-aware routing, P/D disaggregation, KV tiering

Follow the same <version>/<topology>/ convention when adding content under these frameworks.