Skip to content

CHANGELOG.md

Latest commit

 

History

History
1495 lines (927 loc) · 135 KB

CHANGELOG.md

File metadata and controls

1495 lines (927 loc) · 135 KB
Raw

Changelog

All notable changes to LLMKube will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

0.9.6 (2026-07-14)

Features

  • api: add GPUQuota CRD types for multi-tenant GPU governance (#1101) (5dd867a)
  • controller: add GPUQuota status reconciler (#1117) (1d583e4)
  • controller: add GPUQuota validating webhook for InferenceService (#1118) (26804bd)
  • controller: add s3:// model source via curl --aws-sigv4 (#1098) (#1125) (ed35142)
  • foreman: preserve a coder's gate-failed branch instead of discarding it (#1115) (bc39c77)
  • helm: gate InferenceService quota webhook and tenant RBAC behind multitenancy toggle (#1122) (26a994e)
  • quota: add GPUQuota admission decision function (#1107) (4897093)
  • runtime: close out the SGLang kitchen-sink (#1060): minor flags, accept thresholds, typed LoRA adapters, LoRAAdapter CRD (#1103) (8edf8bd)

Bug Fixes

  • foreman: honor GateProfile source extensions in scope-overlap issue-ref extraction (#1120) (0c20431)

Documentation

  • samples: add air-gapped local file-path model source example (#1099) (#1124) (913ff3d)

0.9.5 (2026-07-13)

Features

  • ci: validate config/samples against CRD schemas (Fixes #1021) (#1083) (b3ae847)
  • cli: add --planner-token for gateway-routed planner (Fixes #1053) (#1090) (f71d2cc)
  • controller: add extraVolumes/extraVolumeMounts passthrough on InferenceService (#1079) (80e004c)
  • controller: extend drain-before-roll idle checks to vLLM, TGI, SGLang, and multi-replica services (#1088) (1d884df)
  • foreman: expose CRD status as Prometheus metrics via CRS (Fixes #1001) (#1086) (a459748)
  • foreman: honest-verdict slice 1 (claim evidence + work-class policy) (#1078) (6f2c216)

Bug Fixes

Documentation

  • proposals: honest-verdict harness design (declare-then-verify coder gates) (#1076) (630de33)

0.9.4 (2026-07-11)

Features

  • chart: support multiple agent fleets in one release (#1028) (a9a6c6b)
  • cli: --revise-from-branch on foreman dispatch for in-place PR refresh (#1043) (26f807b)
  • cli: llmkube foreman slice renders a sliced Workload from a plan (#1049) (ce586ed)
  • cli: plan an issue into slices in llmkube foreman slice (#1050) (c4a967c)
  • foreman: flag unverifiable facts instead of inventing them (slicer anti-confabulation) (#1062) (81bbfe9)
  • foreman: integrate + reconcile AgenticTask kinds + slice payload (#1037) (0899e9f)
  • foreman: integrate library for disjoint slice union (#1036) (39131f6)
  • foreman: make the EditFreeStreak guard grounding-aware (#1067) (63a71ae)
  • foreman: reconcile library for sliced-workload interface drift (#1034) (b14f687)
  • foreman: run_integrate tool for sliced-workload union (#1045) (5098b4e)
  • foreman: run_reconcile tool for sliced-workload drift check (#1046) (6bde143)
  • foreman: ship default integrate/reconcile Agents for sliced workloads (#1052) (00dd7c0)
  • runtime: add SGLang runtime backend (#974) (#1059) (9daa186)

Bug Fixes

  • cli: keep the slicer planner output to YAML only (fixes #1062 regression) (#1064) (5d0eb9c)
  • foreman: add payload.branchStrategy to stop stale revisions reverting merged work (#1042) (9f806d4)
  • foreman: anchor recovery soft-reset at merge-base branch point (#1044) (2492d15)
  • foreman: default to rebase when reviseFromBranch is set without branchStrategy (#1065) (2792634)
  • foreman: set fsGroup on the gate pod so non-root gates can write XDG_DATA_HOME (#1068) (7465e2b)
  • foreman: stop the coder from running envtest it cannot run (#1070) (bf203fe)
  • release: publish a cross-platform Homebrew formula, not a macOS-only cask (#1041) (32dcfea)
  • slicer: give the union commit an explicit git identity (#1057) (15d906f)

0.9.3 (2026-07-09)

Bug Fixes

  • controller: drop invalid --enable-metrics flag from vLLM runtime (#1031) (9b9450d)

0.9.2 (2026-07-09)

Features

  • foreman: coder grounding rail (flag doc-contradicting metric writes) (#1017) (aa969ed)
  • foreman: distinct ALREADY-RESOLVED coder outcome (#970) (#1011) (8e041e0)
  • foreman: flag a GO that changes no functional production code (#1024) (3e1a6fb)
  • foreman: MCP client for agents (Phase 1: HTTP transport, context7) (#1014) (d34343f)
  • foreman: semantic issueAsk verification instead of verbatim substring (#1020) (85d92ee)
  • foreman: str_replace escalates to write_file after repeated failures (#1026) (de1b8cb)

Bug Fixes

  • foreman: exempt REJECT from grounded-finding demotion (#1008) (18098cb)
  • foreman: gate Job downloads the go.mod toolchain (GOTOOLCHAIN=auto) (#1019) (5f4c51c)
  • foreman: guard file-less findings + resolve bare/absolute paths in grounding rail (#1009) (304a2b8)
  • foreman: reap audit ConfigMaps older than retention (#990) (#1027) (08d06d6)
  • foreman: reviewer diffs against the upstream base, not stale local fork main (#1006) (6ef7278)
  • metrics: remove dead metrics + phantom PrometheusRule error-rate rule (#1010) (7c92f3f)

0.9.1 (2026-07-07)

Features

  • chart: gate CRD installation behind crds.enabled toggle (#998) (acf2e6a)
  • foreman: append actionable steers to structural-lint gate feedback (#984) (4d496c6)
  • foreman: closest-line fallback when str_replace has no unique anchor (#1000) (02ec677)
  • foreman: grounded-finding rail for reviewer NO-GO verdicts (#988) (cadbd30)
  • foreman: verdict-from-findings rail (promote GO with a grounded blocker) (#992) (b82fcca)

Bug Fixes

  • cachekey: unify model cache-key derivation across controller and CLI (#985) (1fb122c)
  • foreman: grant coder Job pods get on secrets for cloud-proxy auth (#987) (ffe45d7)
  • foreman: key test-presence gate on net-new functions only (#983) (0236630)
  • foreman: tolerate self-committed model work and detect git apply edits (#995) (730d1cd)
  • foreman: truncate over-length submit_result summaries instead of rejecting (#999) (8c36edd)

Documentation

  • cachekey: note the metal literal mirrors acceleratorMetal (#993) (ae9e20d)

0.9.0 (2026-07-05)

⚠ BREAKING CHANGES

  • security: gate local/hostPath model sources + block controller SSRF (GHSA-jw3m-8q7m-f35r) (#981)

Features

  • foreman: use the reviewer's summary as the opened-PR body (#975) (e031ff5)

Bug Fixes

  • foreman: populate FleetNode.Status.CurrentTask so the scheduler spreads work (#978) (d731a90)
  • foreman: reap orphaned Draining FleetNodes so they stop leaking (#980) (241ac4a)
  • runtime: bind inference servers to :: for IPv6-only clusters (#973) (398d5c1)
  • security: gate local/hostPath model sources + block controller SSRF (GHSA-jw3m-8q7m-f35r) (#981) (ec2647f)

0.8.28 (2026-07-04)

Features

  • foreman: bounded fix iteration on reviewer NO-GO instead of terminal failure (#959) (d820fff)
  • foreman: coder-tier escalation, re-dispatch a failed coder to a larger model (#964) (ce8f655)
  • foreman: executor-owned revise-from-branch restore for revision tasks (#967) (b76051c)
  • foreman: open the pull request on review GO (#956) (fd852e1)
  • inference: add spec.modelCache.claimName for user-owned cache PVCs (#960) (aab5a58)

Bug Fixes

  • foreman: accept workspace-internal absolute paths in resolveInside (#957) (34b126c)
  • foreman: defer generic self-gate when runtime is missing from coder image (#958) (df185ec)
  • foreman: reject no-op str_replace where old_string equals new_string (#969) (c71f38b)
  • foreman: scope-overlap check catches Go files in new directories (#962) (486a944)
  • inference: warn when modelCache.claimName is silently ignored (#966) (d49cd22)

Documentation

  • register the Karpenter GPU autoscaling guide in nav.yaml (#954) (88b9c7d)

0.8.27 (2026-07-02)

Features

  • foreman: str_replace fuzzy recovery for old_string drift + write_file steering (#943) (bab4631)

Bug Fixes

  • foreman: agent claims tasks depth-first so verify/review don't starve (#947) (323423d)
  • foreman: chart exposes --foreman-namespace so gate Jobs land with the cache PVC (#939) (0a9ff11)
  • foreman: do not resynthesize children on a terminating Workload (#950) (f90cc49)
  • foreman: replace stale agent-owned branches on push rejection (force-with-lease) (#948) (c55c6ce)
  • foreman: substitute placeholder for empty assistant replies to avoid poisoning history (#940) (6eb67f3)

Documentation

  • metrics-driven autoscaling tutorial for InferenceService (HPA, custom metrics, node autoscaling) (#952) (ebd5996)

0.8.26 (2026-07-02)

Features

  • foreman: two-tier gate-check suite for the coder gate (#931) (f7f469e)

0.8.25 (2026-07-01)

Features

  • foreman: coder clones/pushes each task's own repo when no static remote (#924) (88c334d)

0.8.24 (2026-07-01)

Bug Fixes

  • preserve init-container SecurityContext in podTemplatesDiffer (#922) (#923) (859841e)

Documentation

  • add Jory Irving as Foreman area maintainer (#926) (7513176)

0.8.23 (2026-07-01)

Features

  • controller: defer rollouts until idle (#913) (aa404ff)
  • expose oMLX TurboQuant KV cache quantization settings (#905) (68e0291)
  • foreman: controller/metal-agent runtime-arg parity guard (#918) (92d3c1b)
  • foreman: propagate gateProfile from Workload to decomposed tasks (#920) (270b2fa)
  • foreman: reference-grounding check in the coder gate (#906) (6296496)

Bug Fixes

  • cache hf:// multi-file models instead of emptyDir (#912) (8e69d74)
  • foreman: recover str_replace from near-miss old_string (#917) (4ce29e8)
  • grant events.k8s.io to the operator event recorder (#914) (5b23c4b)

0.8.22 (2026-06-29)

Features

  • foreman: coder gate mutation checks (test-presence + neuter-survival) (#899) (b235932)

Bug Fixes

  • foreman: don't force-terminate a coder mid-verification after it has edited (#896) (#898) (b097c29)
  • foreman: stamp real AgenticTask name on the post-push gate Job (not 'unknown') (#894) (f8096d4)
  • metal-agent: reap orphaned legacy Endpoints so mirror slices stop blackholing traffic (#895) (6274fc5)

0.8.21 (2026-06-28)

Bug Fixes

  • controller: run model-cache-prep init as root again (0.8.20 regression) (#888) (8e98e50)

Documentation

  • strix: document amdgpu.lockup_timeout for device-lost resets (#886) (4468d62)

0.8.20 (2026-06-28)

Features

  • add spec.mode to auto-serve embedding and rerank models (#866) (a7fa09f)
  • cli: add llmkube foreman dispatch to fan issues across coder agents (#879) (1530754)
  • controller: run model-cache-prep init non-root + document PSA/multi-tenancy caveats (#883) (a8049f1)
  • foreman: add a goreleaser-config check to the coder gate (#882) (c8535a7)
  • foreman: add vulkan to the accelerator routing enum (#878) (315dadc)
  • foreman: branch the coder off upstream/main, not the stale fork base (#875) (f4c382c)
  • foreman: deterministically resolve codegen drift in the coder gate (#874) (332f7eb)
  • foreman: durable FleetNode labels via --node-labels (survive pod restart) (#885) (fdda5a8)
  • foreman: scope-overlap guard catches wrong-subsystem coder drift (#876) (795bf23)

Bug Fixes

  • foreman: raise coder agent memory limit default to 8Gi (#884) (8b3ee32)

Documentation

  • add AI-assisted contribution policy (#872) (e65f2fa)
  • foreman: enable a coder-capable in-cluster agent (image in llmkube-runtimes) (#877) (9bc3d2c)

0.8.19 (2026-06-27)

Features

  • foreman: detect changed envtest packages + gate-job delegation seam (part 1 of #859) (#860) (c745127)
  • foreman: ModelProfile CRD for per-model harness tuning (#862) (#863) (90ad28b)
  • foreman: post-push envtest gate (closes the in-loop timing flaw) (#859) (#864) (71da4e2)

Bug Fixes

  • controller: make model cache writable on fsGroupPolicy=None CSIs (#855) (#858) (8e1a30d)

0.8.18 (2026-06-26)

Features

  • model: stage multi-file artifacts (#853) (ca22b84)
  • router: TTFT + budget-utilization metrics, backend.provider span attr, Grafana dashboard (#852) (b82cde1)

Bug Fixes

  • controller: propagate backend displayName into router-proxy config (#849) (9f3b360)

Documentation

  • foreman: fix overview link that 404s the site build (#848) (2f9b82b)
  • foreman: non-Go projects (language gates) page (#845) (d48cf6d)

0.8.17 (2026-06-25)

Features

  • cli: explicit model-cache control flags on deploy/delete (Fixes #722) (#830) (9fc4187)
  • foreman: durable audit-log record + llmkube audit export (#837) (#838) (57d2ad0)
  • foreman: GateProfile API type + language presets + resolver (Addresses #839) (#840) (165a375)
  • foreman: run the fast gate from the resolved GateProfile (Addresses #839) (#841) (4225da4)
  • foreman: scope guard reads source extensions from GateProfile (Addresses #839) (#842) (c36ac87)
  • foreman: verify-gate Job image from GateProfile (Addresses #839) (#843) (327dd88)
  • foreman: verify-gate Job runs resolved commands for non-Go profiles (Addresses #839) (#844) (84994fd)
  • inferenceservice: speculativeDecoding for llama.cpp MTP/draft (Fixes #502) (#827) (564726a)
  • router: optional backend displayName for /v1/models id (Fixes #792) (#826) (7090f83)
  • router: Prometheus metrics + OTel spans for routing decisions (Addresses #433) (#834) (b8fa96d)

Bug Fixes

  • foreman: scope guard skips check on zero-Go-file diffs; NUL-safe git status (Fixes #800) (#823) (92e5501)

Documentation

  • examples: spot-capacity GPU NodePool for Foreman gate Jobs (Fixes #659) (#824) (9fa4846)
  • Karpenter GPU autoscaling guide (Fixes #658) (#825) (2d586ef)

0.8.16 (2026-06-24)

Bug Fixes

  • derive BTP perRetry timeout from configured router timeout (Fixes #817) (#818) (5c41d1e)
  • foreman: retry transient SSE/transport errors in the OAI client (#815) (#816) (1bf8d3d)

Documentation

  • readme: lead hero with heterogeneous hardware + Foreman; add AMD to comparison (#820) (1439ae5)

0.8.15 (2026-06-23)

Features

  • foreman: reviewer.md single source of truth for reviewer prompts (#804) (#812) (7b62b17)

Bug Fixes

  • foreman: make the verify-gate bite check actually run, enable by default (#805) (acbe631)
  • metrics: remove inert InferenceTTFT and InferenceRequestErrors metrics (#786) (#810) (9e42217)

0.8.14 (2026-06-22)

Features

  • add defaultRouteStrategy with BackendNameMatch to ModelRouter (#791) (b7da779)
  • add topologySpreadConstraints and affinity to InferenceService (#793) (#794) (0a24fed)
  • foreman: bite check in the verify-gate Job (reject non-biting tests) (#803) (6f14b43)
  • foreman: reviewer rubric checks for test/prod fidelity (real values + wired-up) (#802) (4353f3a)

Bug Fixes

  • chart: disable model cache when modelCache.enabled is false (#801) (638663b)

0.8.13 (2026-06-21)

Bug Fixes

  • metal-agent: register llamacpp runtime alias for llama.cpp executor (#785) (73b7c90)

0.8.12 (2026-06-21)

Features

  • cli: download heartbeat for llmkube deploy --wait (#719) (#780) (e6f9774)
  • cli: make cache list per-InferenceService cache aware (#731) (#767) (d03c9ee)
  • controller: shield inference pods from disruption during startup (#660) (#781) (370084a)
  • foreman: codegen-drift check in the coder fast gate (#775) (#778) (b04a28d)
  • metal-agent: event-driven endpoint withdrawal/recovery on health transitions (#662) (#771) (7848d18)
  • metal-agent: honor spec.runtime per-CR for multi-runtime dispatch (#525) (#783) (aa210c3)
  • metrics: TTFT histogram, request error counter, vLLM metrics flag (#409) (#769) (563b0c9)

Bug Fixes

  • controller: make checkAcceleratorAvailability DRA-aware (#754) (#776) (804063b)
  • controller: preserve agent-written schedulingStatus on reconcile (#643) (#774) (4321028)
  • controller: reconcileService updates existing Service on endpoint changes (#720) (#773) (70222ae)
  • metal-agent: clear SchedulingStatus on memory-check-pass (#777) (#779) (ed1c4eb)
  • metal-agent: prevent zero-byte stub files from failed model downloads (#642) (#772) (d737e47)

0.8.11 (2026-06-20)

Features

  • ci: publish Helm charts as OCI artifacts to GHCR (#758) (da07772)
  • foreman: add unit-test tier to coder gate (#763) (dfb3d75)
  • foreman: per-Agent contextStrategy with session mode (stable cache-friendly prefix) (#757) (3b1ea1c)

Bug Fixes

  • ci: pin GitHub Actions to commit SHAs in helm-chart workflow (#751) (e1a5b9e)
  • foreman: scope coder-gate golangci-lint cache per workspace (#760) (cee04e8)
  • router: inject backend Model on dispatch so fallback chains degrade (#766) (bee7f1c)

Documentation

  • add Strix Halo node onboarding quickstart (#724) (b1859f0)

0.8.10 (2026-06-20)

Bug Fixes

  • crd: guard GPU resourceClaims/resourceName CEL rule against absent field (#753) (26be1aa)

0.8.9 (2026-06-19)

Features

  • add vulkan accelerator enum and make readiness-check Vulkan-aware (#735) (76cf370)
  • cli: add --node-port to pin a stable NodePort on InferenceService (#737) (a6c1a03)
  • cli: add llmkube scale subcommand to scale InferenceService replicas (#736) (bdb89f8)
  • controller: vendor-neutral DRA (resource.k8s.io/v1) scheduling for InferenceService (#750) (6eb0f27)
  • foreman: deterministic coder verification gate with feedback loop (#749) (8cf3295)
  • foreman: loop convergence forcing (EditFreeStreak + final-turns submit) (#741) (cd3f068)

Bug Fixes

  • cli: correct --node-port help text and add NodePort test coverage (#742) (ef55900)
  • controller: honor GPU resourceName override in checkAcceleratorAvailability (#747) (5aa3152)
  • controller: honor RouterRule.Timeout in gateway-mode AIGatewayRoute generation (#748) (1978a72)
  • foreman: make install-foreman-agent produce a working plist (#743) (b946fef)
  • foreman: scope-overlap rail rescues honest paraphrase from false NO-GO (#746) (00ae36e)

0.8.8 (2026-06-17)

Features

  • AMD/Vulkan runtime image selection (hardware.gpu.runtime) (#727) (1a4544f)
  • crd: make GPU resource name configurable to support AMD/Vulkan/Intel scheduling (#709) (c88becf)
  • gateway: active HTTP health checks on the ModelRouter BTP for fast backend ejection (#662) (#704) (ba99060)
  • gateway: event-driven route-level ejection of unhealthy backends (#662) (#706) (815f2bf)
  • gateway: gateway-scoped audit access log + fail-loud auditLog in Gateway mode (2c) (#703) (b874b5e)
  • gateway: header-only data-classification routing + fail-closed sensitive guard (2e-core) (#707) (0249665)
  • gateway: InferenceService Envoy AI Gateway exposure (MVP) (#692) (3b095dc)
  • gateway: ModelRouter dataPlane Gateway mode with cross-tier failover (2a) (#693) (2842634)
  • gateway: ModelRouter JWT authentication via SecurityPolicy (2d-core) (#695) (73a2ea9)
  • gateway: ModelRouter per-team model allowlists via SecurityPolicy authorization (2d.2) (#702) (94428b4)
  • gateway: ModelRouter token budgets and 429 enforcement (2b) (#694) (627e85a)
  • metal-agent: withdraw endpoint when runtime is unhealthy (#662) (#705) (5ed9395)
  • selfupdate: bound download size + GC old agent versions (#690) (5205a62)
  • webhook: ModelRouter validating webhook for apply-time honest-boundary rejection (#708) (13d9321)

Bug Fixes

  • cache: restore shared model cache as the default (perService becomes opt-in) (#732) (44ab7dc)
  • per-node model cache so GPU on a second node can schedule (#728) (#729) (79bccce)

Documentation

  • DGX Spark (GB10) on MicroK8s setup guide (#717) (bf7d7a7)
  • fix DGX Spark guide for ARM64 (GPU operator + GB10 image) (#718) (45a4237)
  • proposal for owned AMD/Vulkan runtime image and build pipeline (#726) (3a1a150)

0.8.7 (2026-06-14)

Features

  • foreman: validating webhook for Agent + AgenticTask CRs (#685) (1935da7)

Bug Fixes

  • foreman: drop deprecated Result.Requeue in workload terminal test (#688) (a97056c)

0.8.6 (2026-06-13)

Bug Fixes

  • selfupdate: resolve current symlink in managed-root detection (#679) (a6e7223)

0.8.5 (2026-06-13)

Features

  • foreman,agent: report agent version to the cluster (#672) (8860cb6)
  • foreman,selfupdate: agent self-update so approved AgentReleases roll the fleet (#676) (5917715)
  • foreman: add AgentRelease CRD (scaffold for managed fleet updates) (#674) (55f7d9f)
  • foreman: AgentRelease rollout controller (staged, health-gated, approved) (#675) (aa6bbd0)

0.8.4 (2026-06-12)

Features

  • agent,controller: heartbeat liveness for metal endpoint registrations (#666) (6a8cfe8)

Bug Fixes

  • agent: retry endpoint registration with correct update semantics (#664) (82cfc35)
  • foreman: ERROR verdicts become terminal INCOMPLETE outcomes; watcher can no longer wedge (#667) (46405b6)
  • foreman: expire stale claims so tasks survive agent death (#669) (8af7ac6)
  • foreman: patchTerminal stands down when the task is no longer owned (#670) (33af907)
  • foreman: workspace reset survives read-only files from prior runs (#671) (0a7de0d)

0.8.3 (2026-06-11)

Features

  • foreman/loop: support hybrid-thinking models via reasoning_content (#651) (b178de2)
  • foreman/reviewer: computable scope-overlap check demotes drifted GO verdicts (#648) (8afe0c1)
  • foreman/reviewer: demote unverifiable review verdicts to NO-GO (#645) (188dacd)

Bug Fixes

  • foreman/executor: map reasoning-only exhaustion to INCOMPLETE (#652) (d2f453e)
  • foreman/loop: keep wire-stripped reasoning-only turns template-valid (#653) (5ec14f6)

0.8.2 (2026-06-10)

Features

  • controller: add Model refreshPolicy + upstream drift detection (#635) (7dc5b14)
  • controller: real accelerator availability check for Model status (#230) (#610) (f8652fd)
  • foreman: add opt-in NetworkPolicy to the Foreman Helm chart (#625) (8872b4d)
  • foreman: escalation-only reviewer emission on base NO-GO (#639) (d596ce3)
  • foreman: Job execution strategy for remote-model coder/reviewer Agents (#620) (#621) (2317ac0)
  • metal-agent: stable host-side client proxy on a fixed port (#406) (#608) (5b4ea34)

Bug Fixes

  • bump Go toolchain to 1.26.4 to patch standard-library CVEs (#626) (6d8fbb7)
  • controller: recreate InferenceService Deployment on immutable selector change (#606) (#607) (faf9151)
  • controller: revalidate http/https model sources in init container (#636) (64812e5)
  • foreman: detect stale FleetNodes via heartbeat window (#633) (515bdfe)
  • foreman: harden the agent loop against false-negative INCOMPLETE terminations (#623) (edc93e4)
  • foreman: propagate Job-mode branch + commit SHA to AgenticTask status (#634) (3586a14)
  • metal-agent: fail closed when memory admission check cannot complete (#641) (cfe53aa)

Documentation

  • roadmap: rewrite around the heterogeneous sovereign fleet thesis + milestones (#632) (d98b257)

0.8.1 (2026-06-01)

⚠ BREAKING CHANGES

  • foreman: Agent.spec.requestTimeoutSeconds changes meaning from a per-request HTTP timeout to a loop-wide wall-clock budget, and its default moves from 600 to 3600. The former per-request bound is now the new Agent.spec.requestTurnTimeoutSeconds (default 120). Re-apply your Agent CRs after upgrade so existing Agents pick up explicit values.

Features

  • inferenceservice: typed spec.ropeScaling for RoPE/YaRN context extension (#507) (#600) (a554aee)

Bug Fixes

  • foreman: recover orphaned phase=Running tasks on agent restart (#542) (#598) (6dd2c44)
  • foreman: split per-turn timeout from loop-wide budget (#532) (#602) (41e7663)
  • foreman: warm-path reviewer scheduling on macOS (#578, #579) (#597) (a94d1ef)
  • metal-agent: prefer routable interface for host-IP auto-detect (#526) (#599) (c780795)

Documentation

  • foreman: absolute paths in overview README cross-refs (fix llmkube-web prerender) (#596) (b5f6f94)
  • foreman: move docs/foreman to docs/site/foreman + register in site nav (#594) (9fd85bb)

Miscellaneous

  • pin next release to 0.8.1 (Release-As) (#605) (a876cc6)

0.8.0 (2026-05-28)

Features

  • foreman/api: structured AgenticTaskFailureReason taxonomy (#565) (6e72e85)
  • foreman/loop: observation masking for context-window management (#563) (d17c3e0)
  • foreman/loop: stuck-loop detector with nudge-then-force protocol (#544) (#569) (2172ece)
  • foreman/reviewer: fetch_issue tool replaces gh issue view subshell (#581) (0253e43)
  • foreman/tools: distinguish whitelist-excluded from unknown tool calls (#564) (089e9ca)
  • foreman/v0.2: hybrid cloud reviewer Agent + sovereignty toggles (#553) (65a7cb8)
  • foreman/v0.2: WorkloadSpec.reviewerAgentRefs (plural) + third pipeline stage (#551) (831ae8c)
  • foreman: add repo-map localization for coder Agents (#560) (#566) (f6bf8c0)
  • foreman: AgenticTask branches include workload name (#573) (#574) (2986906)
  • foreman: executor fetches GitHub issue body when payload prompt is empty (#571) (#572) (2b5bd31)
  • foreman: post-M4 stability follow-ups for v5-batch readiness (#535) (a841612)
  • foreman: v0.4 reviewer agent — tool-using reviewer with sharpened prompt + structured findings (#575) (#576) (06091a9)
  • foreman: workspace-scoped bash + WORKSPACE_ROOT contract (#567) (#568) (061eb41)
  • gpu: add Intel GPU (oneAPI/SYCL) support across controller, CLI, docs, and e2e (#557) (741ef5d)
  • metal-agent: InferenceService name allowlist for multi-Mac fleets (#555) (67361f3)

Bug Fixes

  • foreman/build: include gate_job_template.yaml in Docker context (#554) (def535d)
  • foreman/executor: resolve InferenceService port from live Endpoints, not stale install-time override (#550) (4351608)
  • foreman/executor: route reviewer-role GO through modelDecidedResult (#545) (16943a5)
  • foreman/loop: force-terminate returns clean Terminal envelope (#544 follow-up) (#570) (19500c9)
  • foreman/oai: always emit content on non-assistant messages (#562) (f6dc8e1)
  • foreman/reviewer: ground-truth filesTouched + bump qwen maxTurns + tighten confabulation defenses (#584) (b3e21f0)
  • foreman/reviewer: ground-truth issueAsk + cap qwen Section D (#587) (b66006c)
  • foreman/reviewer: role-aware stuck-loop detector + non-empty reviewer user prompt (rerun-7 follow-up) (#577) (19590f6)
  • foreman/tools: cmd.WaitDelay + process-group kill so BashTool can't deadlock on grandchild-held pipes (#547) (c12f6f8)
  • foreman: cascade + Workload rollup gate on phase AND verdict, not phase alone (#548) (1b72a7c)

Documentation

  • foreman: v0.8.0 release-prep docs + README Foreman section (#591) (5e41df1)
  • mention make lint-all in AGENTS.md and CONTRIBUTING.md (#588) (39da983)

Miscellaneous

  • release Foreman debut as 0.8.0 (take 2) (#593) (a8f0368)

0.7.12 (2026-05-24)

Features

  • foreman/m6: Workload reconciler with stub planner (explicit pipeline + issue-batch shortcut) (#533) (dbdcd46)

Bug Fixes

  • foreman/m4: gate Job honors payload.branch + clones from --git-remote-url (#529) (905a269)
  • foreman: chart wires --workspace-dir + tightens docs for native-mode required values (#534) (1c43c69)

0.7.11 (2026-05-23)

Bug Fixes

  • foreman: drop chart-level subchart dep on llmkube (unblock v0.7.11 chart-releaser) (#519) (207ddc6)

0.7.10 (2026-05-23)

Features

  • add --llama-server-port for a fixed llama-server runtime port (#499) (cc30b0d)
  • add make lint-all target for cross-arch linting (#508) (f57dd5b)
  • capability-aware scheduler + AgenticTaskWatcher + stub executor (Foreman v0.1 M2) (#504) (74b3d6e)
  • foreman: gate-role Agent on a verifier node (M4) (#518) (40a340e)
  • foreman: native agent loop + Agent CRD + coder role on M5 Max (M3) (#509) (6661343)
  • scaffold Foreman as an opt-in add-on (M0 + M1) (#501) (cd40491)

Bug Fixes

  • report Stopped phase when InferenceService.spec.replicas=0 on Metal path (#498) (7787239)

Documentation

  • add AGENTS.md (#496) (89d3766)
  • bump broken bartowski phi-4-mini URL to renamed repo (#514) (9f15d98)
  • macos-metal: derive curl port from Endpoints (follow-up to #513) (#515) (83085c2)
  • macos-metal: replace broken port-forward step with host-localhost curl (#513) (0f7f7a7)

0.7.9 (2026-05-18)

Features

Bug Fixes

  • clear stale conditions when a model reaches Ready without a download (#476) (06325b0)
  • inference PodMonitor selector matched no pods (#481) (31ee4d6)
  • mark Metal local-path models Ready instead of stuck Copying (#472) (c513c84)
  • metal-path InferenceService status and memory pre-flight (#488) (98ef2c4)
  • point metal-agent mlx-server install hint at the Homebrew formula (#477) (74b3333)
  • prevent concurrent runtime respawn in metal-agent (#469) (f34640b)
  • stop the operator fighting the HPA over Deployment replicas (#485) (8fc70e2)

Documentation

  • add MAINTAINERS file and recommend private vulnerability reporting (#479) (aaccb4d)

0.7.8 (2026-05-14)

Features

  • configurable proxy + per-route/backend timeouts (closes #457, #458) (#461) (03d222a)
  • external provider URL defaults + cluster-wide LiteLLM URL (closes #438) (#451) (26cd5ae)
  • Helm packaging, sample manifest, and concept doc for ModelRouter (#448) (a513fdc)
  • ModelRouterReconciler skeleton with spec validation (#445) (9b1a259)
  • reconcile router-proxy Deployment, Service, and ConfigMap (#447) (856ecc3)
  • router-proxy binary with OpenAI streaming passthrough (#446) (942d09a)
  • router-proxy cluster e2e + runtime fail-closed 503 (closes #430) (#450) (75151fa)
  • scaffold ModelRouter CRD types and deepcopy (#442) (e6c60b3)

Bug Fixes

  • close cloud-tier conns + drop local idle timeout (closes #459) (#460) (173c26a)
  • don't quarantine backends on per-attempt context deadline (closes #462) (#463) (80ef9c8)
  • e2e: unblock MicroShift SCC diagnostics + bump bootstrap timeout (#466) (0c793b7)
  • half-open circuit breaker on proxy + scale-to-zero status (closes #452, #453) (#454) (ac9302c)
  • preserve external annotations on reconciler Deployment updates (#468) (de580c1)

Documentation

  • add consumer-hardware model matrix guide (#444) (dd07397)
  • readme: land ModelRouter prominently for the 0.7.8 release (#464) (deb24bb)
  • site: air-gapped, OpenShift, macOS Metal guides + architecture refresh (Tier 1) (#465) (5996a1e)
  • site: drop stale "fifteen lines" claim in openshift-install Reference (#467) (ec52ca8)

0.7.7 (2026-05-11)

Features

  • agent: vllm-swift runtime + TurboQuant passthrough (#391) (#393) (2691e67)
  • ci+chart: make OpenShift a first-class deploy target (closes #421) (#422) (798a13e)
  • crd: add gpuMemoryUtilization and cpuOffloadGB to VLLMConfig (#394) (6883f78)
  • metal-agent: emit Kubernetes events for memory-pressure transitions, evictions, skips, and respawn blocks (closes #390) (#411) (e0d17d1)
  • observability: runtime label on inference pods + recording rules + starter dashboard (refs #409) (#410) (71743ed)

Bug Fixes

  • controller: default FSGroup to curl_group + Longhorn-backed e2e job (closes #418, closes #420) (adce90f)
  • controller: stop hot-spinning on unreachable file:// model sources (closes #405) (#412) (4ac6f57)

Documentation

  • add NVIDIA Blackwell B200 (sm_100) validation matrix (refs #413) (#414) (bfda149)
  • operations: seed runbooks index + first 2 entries (file:// hot-spin, metal-agent memory pressure) (#417) (d3bce8d)
  • port concepts/comparison to markdown (first Phase 1C content port) (#403) (51c396b)
  • readme: HN-launch readiness fixes (broken link, Apple Silicon CTA, quickstart memory) (#401) (3e44bfb)
  • refresh quickstart cast for v0.7.6 (HN launch) (#404) (5abaddb)
  • split docs/ into site/ and contributors/, prep for site rendering (#396) (9299a31)
  • upgrade: OpenShift / OKD / MicroShift installs must use helm ... -f charts/llmkube/values-openshift.yaml so restricted-v2 SCC can inject fsGroup from the namespace's allocated range (adce90f)
  • upgrade: operators using a custom --init-container-image whose user is not curl (uid=101 gid=102) should set spec.podSecurityContext on each InferenceService or pass --default-fsgroup=<gid> to the controller (adce90f)
  • upgrade: v0.7.7 rolls every InferenceService Pod once on first reconcile (Deployment template gains fsGroup=102 and the new inference.llmkube.dev/runtime label) (adce90f)

0.7.6 (2026-05-03)

Features

  • agent: eviction safety floor + evictionProtection opt-out + late-spawn condition fix (#186) (#384) (6544747)
  • agent: memory-pressure eviction and respawn protection (#186) (#382) (65a78b5)
  • api: add podAnnotations and podLabels passthrough (closes #326) (#381) (baecd68)
  • api: expose runtimeClassName on InferenceServiceSpec (closes #375) (#380) (cc44ff5)
  • crd: add ParallelSlots support for vllm and fix llamacpp (#340) (d81babb)

Bug Fixes

  • catalog: default phi-4-mini context to 8K (closes #386) (#387) (7bcd685)
  • controller: drop model label from Deployment selector to make modelRef mutable (closes #301) (#385) (a1de3bf)
  • derive metal InferenceService phase from Endpoints, not desiredReplicas (closes #374) (#376) (350dafe)

Documentation

  • fix broken phi-3-mini command and dead benchmark link (#369) (6a1fd58)
  • HN launch prep README polish (llama.cpp credit, vLLM, KubeAI/llm-d) (#371) (9d27774)

0.7.5 (2026-04-30)

Bug Fixes

  • chart: sync Helm CRDs from kubebuilder source and add CI guard (#367) (73bd2b4)

0.7.4 (2026-04-29)

Features

  • controller: pin vLLM default image to v0.20.0 (#362) (d2ae561)

Bug Fixes

  • controller: defer HTTP(S) Model downloads to the workload init container (#364) (469f542)

0.7.3 (2026-04-29)

Features

  • agent: cache-type-aware memory estimator + TurboQuant docs (#355) (0697afd)
  • api: add cacheTypeCustomK/V for non-enum llama.cpp KV cache types (#351) (71bd762)
  • api: add kvCacheCustomDtype for non-enum vLLM KV cache types (#359) (5e796d0)

Bug Fixes

  • agent: respawn on InferenceService spec drift, honor replicas=0, and plumb full spec to llama-server flags (#353) (ff54cad)
  • controller: use GGUF metadata name for downloaded model file basename (#347) (e932c7a)
  • vllm: set enableServiceLinks=false on vLLM Pod spec (#361) (01eb5c5)
  • vllm: use positional model argument instead of deprecated --model (#360) (a17566c)

0.7.2 (2026-04-27)

Features

  • agent: expose Apple Silicon power gauges via powermetrics (#334) (58a94a7)
  • make: one-command install-powermetrics-sudo + uninstall targets (#336) (af48077)

Bug Fixes

  • agent: make executor startup timeouts configurable; raise defaults to 120s (#330) (5aa5fa2)
  • agent: reconcile orphaned Service+Endpoints on agent startup (#332) (d88c541)

0.7.1 (2026-04-25)

Features

  • agent: pass Apple Silicon-optimized flags to llama-server (#327) (a69ab6a)
  • chart: add values.schema.json for Helm value validation (#322) (1f8a34d)
  • crd: extend InferenceService vLLM config for agentic-coding flags (#306) (cb2aa6a)
  • security: supply-chain MVP — checksum install, govulncheck, gosec, codecov (#310) (f17f59d)

Bug Fixes

  • agent: detect stalled K8s polling and exit for supervisor restart (#328) (c0636cc)
  • agent: let the kernel pick free ports for llama-server (#321) (8111395)
  • bump InferenceService spec.contextSize cap from 131072 to 2097152 (#300) (a46a1bf)

Documentation

  • add ADOPTERS.md inviting public user listings (#324) (871a0cb)
  • backfill ⚠ BREAKING CHANGES section into 0.7.0 changelog (#296) (2ad4640)

0.7.0 (2026-04-18)

⚠ BREAKING CHANGES

  • sharding: sharding.strategy: tensor on a Model now correctly maps to llama.cpp's --split-mode row instead of silently falling back to --split-mode layer. Configs that set strategy: tensor expecting layer behavior may see performance regressions or new failure modes under concurrent load (particularly on consumer PCIe multi-GPU setups with quantized models). Explicitly set strategy: layer to retain the previous behavior. (#291)
  • vllm: InferenceService spec.extraArgs is now forwarded to the vLLM runtime. Previously extraArgs was silently ignored when runtime: vllm. Configs that placed llama.cpp-only flags in extraArgs on a vLLM InferenceService will start failing at pod startup. Audit any vLLM InferenceService that sets extraArgs before upgrading. (#291)

Features

  • add hybrid GPU/CPU offloading support for MoE models (#281) (2287f66)
  • add tensor overrides and batch size controls for hybrid offloading (#283) (8be4adc)
  • expose additional runtime controls for llama.cpp and vllm (#291) (2245718)
  • recognize runtime-resolved sources (HF repo IDs) in Model controller (#293) (953e8a7)

Bug Fixes

  • inherit runAsUser/runAsGroup from podSecurityContext (#274) (72b9b5c)

Documentation

  • surface breaking behavior changes for 0.7.0 (#294) (e234a40)

0.6.0 (2026-04-08)

⚠ BREAKING CHANGES

  • update default CUDA image to server-cuda13 for Qwen3.5 and Blackwell support (#262)

Features

  • add first-class PersonaPlex (Moshi) runtime backend (#272) (2b1c948)
  • add Grafana inference metrics dashboard (#269) (be376c6)
  • add HPA autoscaling for InferenceService (#260) (2d16502)
  • add pluggable runtime backends for non-llama.cpp inference engines (#271) (bb1576c)
  • add vLLM and TGI runtime backends with per-runtime HPA metrics (#273) (441c7c7)
  • separate image registry from repository in Helm chart (#268) (5c059a4)
  • support custom layer splits from GPUShardingSpec (#267) (a37701c)
  • update default CUDA image to server-cuda13 for Qwen3.5 and Blackwell support (#262) (cc9a95e)

0.5.3 (2026-04-01)

Features

  • add KV cache type configuration and extraArgs escape hatch (#256) (7a4b855)
  • add Ollama as runtime backend for Metal agent (#258) (6148b89)
  • add oMLX as alternative runtime backend for Metal agent (#257) (eaf9045)

Bug Fixes

0.5.2 (2026-03-27)

Features

  • add pod security context defaults and CRD overrides (#239) (904432b)

Documentation

0.5.1 (2026-03-16)

Features

  • add memory pressure watchdog with runtime monitoring (#216) (5fa6d54)
  • add pvc:// model source and SHA256 integrity verification (#229) (1b94f5d)
  • auto-detect llama-server from Homebrew paths on macOS (#215) (a1e4302)

Bug Fixes

  • controller metrics port declarations and ServiceMonitor consistency (#214) (296ec99)
  • correct CHANGELOG entry from 0.4.21 to 0.5.0 (#212) (f7f703a)
  • quote job-level if expression to fix YAML parsing in helm-chart workflow (8714b9f)

0.5.0 (2026-03-04)

Features

  • add pre-flight memory validation for Metal agent (#204) (ba252ef)
  • add health checks, metrics, and continuous monitoring to Metal agent (#205) (a113fd1)
  • add per-model memoryBudget and memoryFraction CRD fields (#206) (e632369)

Bug Fixes

  • agent: unregister service endpoints on metal process delete (#168) (147b9bc)
  • enable controller metrics endpoint in Helm chart (#195) (70940af)
  • prevent model re-download of cached models after helm upgrade (#203) (a8f9a88)
  • use Recreate strategy for GPU workloads to prevent rolling update deadlock (#196) (2e45181)

Documentation

  • rewrite README for clarity, positioning, and growth (#190) (a7fc152)

0.4.20 (2026-02-28)

Features

  • add license compliance scanning for GGUF models (#188) (c26400a)
  • add Prometheus metrics, OpenTelemetry tracing, and inference observability (#189) (c653ff1)
  • add PVC inspection to cache list for orphaned entry detection (#183) (2723d92)
  • agent: add structured zap logging to metal agent (#164) (e9d143c)
  • deps: upgrade to Kubernetes 1.35 and controller-runtime v0.23.1 (#175) (3c323f4)

Bug Fixes

  • correct Metal quickstart docs for selectorless services (#173) (89471ec)
  • prevent command injection in init container shell commands (#172) (3aa9cc3)
  • remove mutable latest tags and pin container images (#174) (3c4569a)

Documentation

  • add Apple Silicon Metal option to bug report template (#169) (e7689d8)

0.4.19 (2026-02-21)

Features

  • add --jinja flag for tool/function calling support (#162) (47624ca)

0.4.18 (2026-02-20)

Bug Fixes

  • agent: read contextSize from InferenceService CRD (#160) (17f58d4)

Documentation

  • update README and Metal Agent guide for remote K8s architecture (#156) (79145b2)

0.4.17 (2026-02-20)

Bug Fixes

  • agent: filter InferenceServices by Metal accelerator type (#157) (5737bb7)

0.4.16 (2026-02-20)

Features

  • agent: add --host-ip flag for remote K8s cluster support (#155) (b425569)

Documentation

  • Add Metal Agent (Apple Silicon) support to README (#151) (3579426)

0.4.15 (2026-02-15)

Bug Fixes

  • inference: pass value to --flash-attn for newer llama.cpp versions (#148) (25e08d0)

0.4.14 (2026-02-15)

Features

  • gguf: add native Go GGUF parser with CRD integration and CLI inspect (#140) (9d96ed4)
  • inference: add flashAttention and contextSize to sample manifest (914c929), closes #145

0.4.13 (2026-02-07)

Features

  • controller: make init container image configurable (#128) (38ccdf0)
  • expose llama.cpp parallel slots in InferenceService CRD (#133) (cae7b52)
  • helm: add optional NetworkPolicy for controller manager (#135) (8d61ce3)
  • update model catalog with DeepSeek R1 and refresh stale entries (#131) (89eb5a6)

0.4.12 (2026-01-22)

Features

  • add custom CA support and fix deprecated image tags (#124) (5ec912e)

0.4.11 (2026-01-22)

Bug Fixes

  • cli: use numeric comparison for version checking (#109) (05e0025)
  • controller: use fully qualified image names for curl (#121) (213660b)

0.4.10 (2025-12-07)

Features

  • add 32B models to catalog with --context flag (#88) (6c06602)
  • add air-gapped deployment support for local model paths (#85) (31fe8d0)
  • add GPU observability config and Grafana dashboard (#105) (571643f)
  • cli: add comprehensive benchmark test suites and sweeps (#107) (323a28a)
  • cli: add stress testing mode to benchmark command (#104) (530c82e)

Documentation

  • add community standards and security policy (#92) (e7c9cad)
  • update documentation for v0.4.9 GPU scheduling features (#83) (0934e8f)

0.4.9 (2025-12-01)

Features

  • add GPU contention visibility, queue position, and priority classes (#81) (c0220e5)

Documentation

  • add getting started video to README (#76) (ceb83d7)

0.4.8 (2025-11-27)

Features

  • Support configurable context size for llama.cpp server (#73) (6f8e04b)

0.4.7 (2025-11-26)

Bug Fixes

  • Don't mark Helm chart release as latest (#70) (761b154)

0.4.6 (2025-11-26)

Bug Fixes

  • Set empty component to prevent llmkube- prefix in releases (#68) (45b61c6)

0.4.5 (2025-11-26)

Bug Fixes

  • Clean up release process - single release with proper notes (#66) (4deae85)

0.4.4 (2025-11-26)

Bug Fixes

  • Trigger GoReleaser and Helm release from Release Please workflow (#64) (9a37a77)

0.4.3 (2025-11-26)

Features

  • Add benchmark command and reorganize documentation (58307be)
  • Add benchmark command and reorganize documentation (ac8888e), closes #6
  • Add Helm chart for easy installation (5718804)
  • Add Helm chart for easy installation with comprehensive CI testing (3ea3bfd), closes #9
  • Add Metal GPU support for macOS (Apple Silicon) (f673c26), closes #33
  • Add model catalog with 10 pre-configured models (404d722)
  • Add model catalog with 10 pre-configured models (Phase 1) (0fd969a)
  • Add persistent model cache to avoid re-downloading (83f844f), closes #52
  • Add Release Please automation and version-agnostic docs (dc2d54e)
  • helm: Add image digest support for production deployments (a38801d)
  • Implement automatic port forwarding for benchmark command (472b3ae)
  • Multi-GPU support with layer-based sharding (#47) (4797609)
  • Persistent model cache with per-namespace PVC support (ab04261)
  • Set up Helm repository on GitHub Pages (8d62737)
  • Support per-namespace model cache PVCs (c3cb891)

Bug Fixes

  • Add cacheKey to CRD and restrict cache to llmkube-system namespace (464c23d)
  • Add CRD keep policy and improve security test reliability (ff32296)
  • Add Helm chart publishing to release workflow (8baf9c4)
  • Add Helm chart publishing to release workflow (03bab72)
  • Add Homebrew archive IDs and v0.3.0 release notes (cea933b)
  • Address lint issues in benchmark command (bf80610)
  • Address linter errors in catalog implementation (8932e4f)
  • Address linter issues in Metal agent code (3f1f678)
  • controller: Add Model watch to InferenceService controller (cb4e201)
  • Correct CLI binary path in E2E tests (41af555)
  • Fix GoReleaser Homebrew tap configuration for v0.3.0 (4e95c04)
  • Further increase Helm CI timeout and readiness probe delay (5453d66)
  • Further increase Helm CI timeout and readiness probe delay (fd577d3)
  • Handle resp.Body.Close error in version check (linter) (fb3adf5)
  • Increase Helm chart CI timeout from 2m to 5m (7a08b45)
  • Increase Helm chart CI timeout from 2m to 5m (ced2210)
  • InferenceService stuck in Pending when Model becomes Ready (4d20aec)
  • Metal agent production fixes and testing improvements (8744c7b)
  • Resolve Helm chart CI test failures (9919696)
  • Resolve staticcheck SA5011 lint errors and update CONTRIBUTING.md (#60) (c0b5824)
  • Sanitize Service names for DNS-1035 compliance (v0.3.3) (db81990)
  • Sanitize Service names to comply with DNS-1035 requirements (b431986)
  • Skip containerized Deployment for Metal accelerator and add version check (d300e64)
  • Skip containerized Deployment for Metal accelerator and add version check (8dab955)
  • Suppress Endpoints API deprecation warnings (e70a4b3)
  • Update operator deployment to use correct container image (00fee75)
  • Update operator deployment to use correct container image (4c67a78)
  • Update version.go to 0.2.1 and add automation for future releases (8dd613d)
  • Update version.go to 0.2.1 and add automation for future releases (2ff68bd)
  • Use simple v* tag format for releases (#62) (bda9f19)
  • Use workspace path for kubeconform validation (fc066d8)

Documentation

  • Add CLI option to quick start, keep kubectl as fallback (f6829ee)
  • Add release notes for v0.3.2 (177abf8)
  • Add release notes for v0.3.2 (ca1bb12)
  • Add release notes for v0.4.0 (144b960)
  • Add release notes for v0.4.0 (a61321f)
  • Overhaul README and roadmap for public launch (b42c17e)
  • Update binary download links to version 0.2.1 (fad530a)
  • Update binary download links to version 0.2.1 (63bb0fa)
  • Update Helm installation to use GitHub Pages repository (477e037)
  • Update MODEL-CACHE.md for per-namespace PVC pattern (0be3f46)

0.4.2 (2025-11-26)

Bug Fixes

  • Resolve staticcheck SA5011 lint errors and update CONTRIBUTING.md (#60) (c0b5824)

0.4.1 (2025-11-26)

Features

  • Add benchmark command and reorganize documentation (58307be)
  • Add benchmark command and reorganize documentation (ac8888e), closes #6
  • Add persistent model cache to avoid re-downloading (83f844f), closes #52
  • Add Release Please automation and version-agnostic docs (dc2d54e)
  • helm: Add image digest support for production deployments (a38801d)
  • Implement automatic port forwarding for benchmark command (472b3ae)
  • Persistent model cache with per-namespace PVC support (ab04261)
  • Support per-namespace model cache PVCs (c3cb891)

Bug Fixes

  • Add cacheKey to CRD and restrict cache to llmkube-system namespace (464c23d)
  • Address lint issues in benchmark command (bf80610)

Documentation

  • Update MODEL-CACHE.md for per-namespace PVC pattern (0be3f46)

[0.3.0] - 2025-11-23

Added

Metal GPU Support for macOS (Apple Silicon)

  • Native Metal GPU Acceleration: Full support for Apple Silicon (M1/M2/M3/M4) GPUs
    • 60-120 tok/s generation on M4 Max (Llama 3.1 8B: 40-60 tok/s, Llama 3.2 3B: 80-120 tok/s)
    • Native llama-server processes with Metal GPU offloading
    • Hybrid architecture: Kubernetes orchestration + native Metal performance
  • Metal Agent: Background daemon for macOS that manages llama-server processes
    • Watches InferenceService CRDs and spawns native processes
    • Automatic Service and Endpoints creation for cluster integration
    • Health checking and process lifecycle management
    • Configurable via LaunchAgent (deployment/macos/com.llmkube.metal-agent.plist)
  • Platform Detection: Automatic detection of Metal availability and GPU capabilities
  • CLI Metal Support: --accelerator metal flag for one-command Metal deployments
    • llmkube deploy llama-3.1-8b --accelerator metal
    • Automatic GPU layer configuration and optimization
  • Multi-Accelerator Support: Unified CLI for CUDA (cloud) and Metal (local) deployments
    • Same Kubernetes CRDs work across both platforms
    • Test locally on Mac, deploy to cloud with same configs

Developer Experience

  • GoReleaser Configuration: Multi-platform CLI builds for macOS, Linux, Windows
    • Separate Metal agent binary for macOS (Intel + Apple Silicon)
    • Automated release workflow with GitHub Actions
  • Metal Quick Start Guide: Comprehensive guide at examples/metal-quickstart/README.md
    • Architecture diagrams and explanations
    • Step-by-step setup instructions
    • Troubleshooting and performance tuning
  • macOS Deployment Guide: Production deployment instructions at deployment/macos/README.md

Changed

  • Deploy Command: Enhanced to support Metal accelerator alongside GPU flag
  • Service Registry: Added support for manual Endpoints management to bridge native processes

Fixed

  • Endpoints API deprecation warnings (SA1019) with appropriate nolint directives
  • Metal agent linter issues and production stability improvements

Documentation

  • New: examples/metal-quickstart/README.md - Metal GPU quick start guide
  • New: deployment/macos/README.md - macOS deployment and setup
  • New: cmd/metal-agent/main.go - Metal agent binary implementation
  • New: pkg/agent/ - Agent, executor, watcher, and registry implementations
  • New: internal/platform/detect.go - Platform and GPU detection
  • Updated: README with Metal support documentation

[0.2.2] - 2025-11-23

Added

Model Catalog (Phase 1)

  • Pre-configured Model Catalog: 10 battle-tested LLM models with optimized settings
    • Small models (1-3B): Llama 3.2 3B, Phi-3 Mini
    • Medium models (7-8B): Llama 3.1 8B, Mistral 7B, Qwen 2.5 Coder 7B, DeepSeek Coder 6.7B, Gemma 2 9B
    • Large models (13B+): Qwen 2.5 14B, Mixtral 8x7B, Llama 3.1 70B
  • Catalog CLI Commands:
    • llmkube catalog list - Browse all available models with specifications
    • llmkube catalog info <model-id> - View detailed model information
    • llmkube catalog list --tag <tag> - Filter models by tags (code, small, recommended, etc.)
  • One-Command Deployments: Deploy catalog models without specifying source URLs
    • llmkube deploy llama-3.1-8b --gpu - No need to find GGUF URLs
    • Automatic application of optimized settings (quantization, resources, GPU layers)
    • Flag overrides still work for customization
  • Embedded Catalog: YAML catalog embedded in CLI binary for offline usage

Developer Experience

  • Enhanced Deploy Command: Made --source flag optional for catalog models
  • Smart Defaults: Catalog models come with pre-configured CPU, memory, GPU layers, and quantization
  • Better Error Messages: Helpful suggestions when model not found in catalog
  • Documentation Updates: README showcases catalog feature prominently

Changed

  • CLI Help Text: Updated deploy command examples to highlight catalog usage
  • README: Added catalog section to features and quick start

Fixed

  • Line length and linter compliance in catalog implementation
  • E2E test binary path for catalog tests

Documentation

  • New: pkg/cli/catalog.yaml - Embedded model catalog with 10 models
  • New: Comprehensive unit tests (13 test functions, 50+ test cases)
  • New: E2E tests for catalog commands
  • Updated: README with catalog usage examples
  • Updated: Deploy command help text with catalog examples

[0.2.0] - 2025-11-17

Added

GPU Acceleration (Phase 0-1)

  • 17x Performance Improvement: GPU-accelerated inference on NVIDIA GPUs (L4, T4, A100, V100)
    • 64 tok/s generation on Llama 3.2 3B (vs 4.6 tok/s CPU)
    • 1,026 tok/s prompt processing (66x faster than CPU)
    • 0.6s total response time (17x faster than CPU's 10.3s)
  • Automatic GPU Scheduling: GPU resource requests, tolerations, and node selectors configured automatically
  • GPU Layer Offloading: Automatic detection and configuration of optimal GPU layer count
  • CLI GPU Support: --gpu flag for one-command GPU deployments
  • Multi-GPU API: Future-proof CRD design supporting up to 8 GPUs per model
  • GPU Configuration Flags: --gpu-count, --gpu-memory, --gpu-layers, --gpu-vendor

Observability Stack (Phase 1)

  • Prometheus Integration: Full kube-prometheus-stack deployment with ServiceMonitors
  • DCGM GPU Metrics: 10+ GPU metrics (utilization, temperature, power, memory)
  • Grafana Dashboard: Pre-built GPU monitoring dashboard (config/grafana/llmkube-gpu-dashboard.json)
    • 3 gauge panels: GPU utilization, temperature, power
    • 3 timeseries panels: Memory, utilization over time, power over time
    • Auto-refresh every 10 seconds
  • SLO Alert Rules: Production-ready alerts for GPU health and service availability
    • GPUHighUtilization, GPUHighTemperature, GPUMemoryPressure, GPUPowerLimit
    • InferenceServiceDown, ControllerDown

Infrastructure & Testing

  • GKE GPU Cluster Terraform: Complete GPU cluster setup with NVIDIA L4 GPUs
    • Spot instance support (~70% cost savings)
    • Auto-scale to 0 for cost optimization
    • NVIDIA GPU Operator installation
  • E2E Test Suite: Comprehensive 8-test validation suite (test/e2e/gpu_test.sh)
    • GPU scheduling verification
    • Inference endpoint testing
    • GPU metrics validation
    • Alert rules validation
  • GPU Quickstart Example: Complete working example (examples/gpu-quickstart/)
    • Model and InferenceService YAML files
    • Automated test script
    • Comprehensive documentation with troubleshooting

Changed

  • Controller Image: Updated to support GPU layer offloading automatically
  • CLI Deploy Command: Enhanced with GPU-specific flags and auto-detection
  • Documentation: Complete rewrite of README, launch materials, and performance benchmarks
  • Version: Bumped from 0.1.0 to 0.2.0

Fixed

  • GPU Layer Offloading: Controller now correctly applies --n-gpu-layers 99 for automatic offloading
  • CUDA Image Selection: CLI automatically selects CUDA image when --gpu flag is set

Performance

  • Llama 3.2 3B Q8_0 on NVIDIA L4:
    • Generation: 64 tok/s (17x faster than CPU)
    • Prompt Processing: 1,026 tok/s (66x faster than CPU)
    • Total Response: 0.6s (17x faster than CPU)
    • GPU Layers: 29/29 (100% offloaded)
    • GPU Memory: 4.2GB / 24GB
    • Power: 35W
    • Temperature: 56-58°C

Documentation

  • New: RELEASE_NOTES_v0.2.0.md - Comprehensive v0.2.0 release notes
  • New: examples/gpu-quickstart/ - GPU deployment quickstart guide
  • New: config/grafana/llmkube-gpu-dashboard.json - GPU monitoring dashboard
  • New: config/prometheus/llmkube-alerts.yaml - SLO alert rules
  • New: test/e2e/gpu_test.sh - E2E test suite
  • Updated: README.md - GPU sections, performance benchmarks
  • Updated: ROADMAP.md - Phase 0-1 completion status
  • Updated: LAUNCH_ANNOUNCEMENT.md - GPU-focused launch messaging

Known Limitations

  • Single-GPU only (multi-GPU coming in Phase 2-3)
  • NVIDIA GPUs only (AMD/Intel support planned for later sprints)
  • GGUF format only (SafeTensors planned)
  • Tested primarily on GKE/EKS (other K8s distributions should work)

[0.1.0] - 2025-11-15

Added

  • Kubernetes Operator: Complete operator implementation with Kubebuilder
  • Model CRD: Define LLM models with source URLs, quantization, and hardware requirements
  • InferenceService CRD: Manage inference deployments with replicas and resources
  • Model Controller: Automatic model download from HuggingFace and other HTTP sources
    • GGUF format support
    • Size calculation and validation
    • Path management and status tracking
  • InferenceService Controller: Automatic deployment and service creation
    • Init containers for model downloading
    • Service creation (ClusterIP, NodePort, LoadBalancer)
    • OpenAI-compatible endpoint routing
  • CLI Tool: Basic CRUD operations
    • llmkube deploy - Deploy models
    • llmkube list - List models and services
    • llmkube status - Check deployment status
    • llmkube delete - Remove deployments
    • llmkube version - Version information
  • Inference Runtime: llama.cpp integration
    • Automatic model download via init containers
    • OpenAI-compatible API (/v1/chat/completions)
    • CPU inference support
    • Streaming and non-streaming responses

Performance

  • TinyLlama 1.1B Q4_K_M on GKE CPU nodes:
    • Model size: 637.8 MiB
    • Prompt processing: ~29 tokens/sec
    • Token generation: ~18.5 tokens/sec
    • Cold start (with download): ~5 seconds
    • Warm start: <1 second

Documentation

  • Initial README.md with installation and usage instructions
  • ROADMAP.md with development plan
  • API documentation for CRDs
  • Architecture overview in README

Release Links: