feat(controller): add InferenceService hardware-labels info metric (#1121)#1160
Merged
Merged
Conversation
Add a companion llmkube_inferenceservice_info gauge (value 1) carrying
{inferenceservice, namespace, accelerator, runtime} labels so operators
can group the fleet by Metal/AMD/NVIDIA and serving backend without
churning the phase metric cardinality.
The accelerator label is sourced from Model.Spec.Hardware.Accelerator
(defaulting to "cpu" when unset), and the runtime label from
InferenceService.Spec.Runtime (defaulting to "llamacpp").
Also add a unit test asserting the new metric is registered and emitted
with the hardware labels for a representative InferenceService.
Fixes defilantech#1121
Signed-off-by: Foreman Bot <chris@mahercode.io>
…changes The InferenceServiceInfo info gauge only called WithLabelValues(...).Set(1) for the current accelerator/runtime label-set and never cleared the previous one. When a Model's accelerator changed in place (e.g. cpu -> cuda) or an InferenceService's runtime changed, the old series lingered at 1 forever alongside the new one, causing double-counting on fleet dashboards grouping by accelerator. Fix by calling DeletePartialMatch on the info gauge before emitting the new series, ensuring exactly one series exists per InferenceService. Also fix the TestInferenceServiceInfo regression test which asserted the old 'cuda' series was still 1 after switching to 'rocm' — that encoded the bug. Change it to assert the old label-set is gone (0) and only the new label-set is 1. Fixes defilantech#1121 Signed-off-by: Foreman Bot <chris@mahercode.io>
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Merged
eleboucher
pushed a commit
to eleboucher/homelab
that referenced
this pull request
Jul 20, 2026
…7 ➔ 0.9.8) (#1660) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [ghcr.io/defilantech/charts/llmkube](https://github.com/defilantech/LLMKube) | patch | `0.9.7` → `0.9.8` | --- ### Release Notes <details> <summary>defilantech/LLMKube (ghcr.io/defilantech/charts/llmkube)</summary> ### [`v0.9.8`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#098-2026-07-19) [Compare Source](defilantech/LLMKube@v0.9.7...v0.9.8) ##### Features - **controller:** add InferenceService hardware-labels info metric ([#​1121](defilantech/LLMKube#1121)) ([#​1160](defilantech/LLMKube#1160)) ([039adaa](defilantech/LLMKube@039adaa)) - **controller:** add llamacpp-router runtime for multi-model serving ([#​1152](defilantech/LLMKube#1152)) ([fae5983](defilantech/LLMKube@fae5983)) - **controller:** explicit vllm serve entrypoint for image-agnostic vLLM ([#​1164](defilantech/LLMKube#1164)) ([#​1165](defilantech/LLMKube#1165)) ([3eeb580](defilantech/LLMKube@3eeb580)) - **controller:** pass staged model directory to vLLM/SGLang for multi-file models ([#​1157](defilantech/LLMKube#1157)) ([#​1159](defilantech/LLMKube#1159)) ([a24fa94](defilantech/LLMKube@a24fa94)) - **controller:** ROCm/HIP runtime tier for AMD nodes ([#​701](defilantech/LLMKube#701)) ([#​1154](defilantech/LLMKube#1154)) ([3704d47](defilantech/LLMKube@3704d47)) - **finalize:** gate foreman-finalize on the Foreman verify verdict ([#​1150](defilantech/LLMKube#1150)) ([#​1153](defilantech/LLMKube#1153)) ([5c7b2cd](defilantech/LLMKube@5c7b2cd)) - **foreman:** in-executor envtest gate feedback loop ([#​768](defilantech/LLMKube#768)) ([#​1135](defilantech/LLMKube#1135)) ([074544a](defilantech/LLMKube@074544a)) - **foreman:** provider-neutral CodeHost/WorkItems/ChangePolicy seams ([#​1158](defilantech/LLMKube#1158)) ([#​1161](defilantech/LLMKube#1161)) ([d3a8ad6](defilantech/LLMKube@d3a8ad6)) - **observability:** per-InferenceService SLO declaration via Pyrra ([#​415](defilantech/LLMKube#415)) ([#​1149](defilantech/LLMKube#1149)) ([4ae022c](defilantech/LLMKube@4ae022c)) ##### Bug Fixes - bump boilerplate copyright year 2025 -> 2026 ([#​1162](defilantech/LLMKube#1162)) ([#​1163](defilantech/LLMKube#1163)) ([30dfaf2](defilantech/LLMKube@30dfaf2)) ##### Documentation - document strict-taint model caches ([#​1132](defilantech/LLMKube#1132)) ([9250996](defilantech/LLMKube@9250996)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4xMDEuMSIsInVwZGF0ZWRJblZlciI6IjQzLjEwMS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJyZW5vdmF0ZS9jb250YWluZXIiLCJ0eXBlL3BhdGNoIl19--> Reviewed-on: https://git.erwanleboucher.dev/eleboucher/homelab/pulls/1660
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Add an
llmkube_inferenceservice_infogauge (value 1) carrying{inferenceservice, namespace, accelerator, runtime}so a fleet dashboard can group InferenceServices by hardware family (Metal / AMD / NVIDIA) and serving backend directly from the CR-state metric.Why
Fixes #1121. Today
llmkube_inferenceservice_phasecarries only{inferenceservice, namespace, phase}, so a fleet-overview dashboard can list models and phases but cannot answer "group my models by Metal vs AMD vs NVIDIA" without joining node-level metrics. This is the Phase-3 hardware-per-model follow-up to the fleet dashboard.How
InferenceServiceInfogauge ininternal/metrics/metrics.go, registered once.status_builder.gofromModel.Spec.Hardware.AcceleratorandInferenceService.Spec.Runtime(with the CRD defaultscpu/llamacppas fallbacks; empty values guarded). Bounded cardinality (both labels are closed enums).DeletePartialMatch({inferenceservice, namespace})runs before theSet(1)so that when a Model's accelerator or an ISvc's runtime changes in place, exactly one series exists per ISvc (no double-counting on a fleet dashboard).Testing
DeletePartialMatch).internal/controller+internal/metricssuites green;make lintclean.Checklist
make testpasses (controller envtest + metrics suites green)make lintpasses (0 issues)Fixes #1121AI-assisted (band-3): implemented by an LLMKube Foreman coder agent (qwopus 35B), then reviewed by a human who caught a stale-series leak; a second Foreman revision cycle applied the
DeletePartialMatchfix, re-verified before opening. Human owns the review conversation.