Skip to content

installation.md

Latest commit

 

History

History
82 lines (62 loc) · 2.64 KB

installation.md

File metadata and controls

82 lines (62 loc) · 2.64 KB
Raw

Installation

Prerequisites

  • Kubernetes 1.27+ cluster (any flavor — k3s, EKS, kind, Docker Desktop).
  • One or more inferenced daemons running on Apple Silicon hosts that the cluster can reach over HTTP. See the inferenced install docs.
  • kubectl and (for the easy path) helm 3.13+.
  • Network reachability from the operator's pod to each daemon's HTTP port. If your cluster runs in a Tailscale-in-VM environment, the operator pod will already have a Tailscale IP and can reach the macOS hosts directly.

Install with Helm

helm install inferenced-operator \
  oci://ghcr.io/dormlab/charts/inferenced-operator \
  --namespace inferenced \
  --create-namespace \
  --version 0.1.0

Useful overrides:

--set image.tag=0.1.0
--set logLevel=debug
--set installCRDs=false           # if you manage CRDs out of band (Argo, Flux)
--set replicaCount=1              # >1 requires leader election (v0.2)

See charts/inferenced-operator/values.yaml for everything tunable.

Install without Helm

# 1. CRDs.
kubectl apply -f https://raw.githubusercontent.com/dormlab/inferenced-operator/main/crds/inferencehost.yaml
kubectl apply -f https://raw.githubusercontent.com/dormlab/inferenced-operator/main/crds/inferencemodel.yaml

# 2. Namespace.
kubectl create namespace inferenced

# 3. ServiceAccount + RBAC + Deployment from the chart's rendered output.
helm template inferenced-operator \
  oci://ghcr.io/dormlab/charts/inferenced-operator \
  --namespace inferenced \
  --set installCRDs=false \
  | kubectl apply -f -

Build from source

git clone https://github.com/dormlab/inferenced-operator
cd inferenced-operator
cargo build --release
./target/release/inferenced-operator --print-crds | kubectl apply -f -
docker build -t ghcr.io/yourorg/inferenced-operator:dev .

First resources

After install, register your hosts and models:

kubectl apply -f https://raw.githubusercontent.com/dormlab/inferenced-operator/main/examples/host-amelia.yaml
kubectl apply -f https://raw.githubusercontent.com/dormlab/inferenced-operator/main/examples/model-qwen-3b.yaml

kubectl get inferencehost
kubectl get inferencemodel

Uninstall

helm uninstall inferenced-operator -n inferenced
kubectl delete crd inferencehosts.inference.dormlab.dev
kubectl delete crd inferencemodels.inference.dormlab.dev
kubectl delete namespace inferenced

CRD deletion will cascade-delete all InferenceHost / InferenceModel resources, which in turn (via OwnerReference) deletes the operator-managed Services and EndpointSlices.