Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 107 additions & 4 deletions public/kubernetes-guides/cni/deploying-cilium.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ talosctl gen config \

### Installation using Cilium CLI

> Note: It is recommended to template the cilium manifest using helm and use it as part of Talos machine config, but if you want to install Cilium using the Cilium CLI, you can follow the steps below.
> Note: It is recommended to template the Cilium manifest using helm and use it as part of Talos machine config, but if you want to install Cilium using the Cilium CLI, you can follow the steps below.

Install the [Cilium CLI](https://docs.cilium.io/en/stable/gettingstarted/k8s-install-default/#install-the-cilium-cli) following the steps here.

Expand Down Expand Up @@ -407,9 +407,112 @@ cluster:
- k8sServicePort=7445
```

It's also possible to run a job with the values inside a configmap.
This can be useful in situations where you have a lot of values configured.

``` yaml
cluster:
inlineManifests:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: cilium-install
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: cilium-install
namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: cilium-install
namespace: kube-system
---
apiVersion: v1
kind: ConfigMap
metadata:
name: cilium-values
namespace: kube-system
data:
values.yaml: |
<Cilium Values>
---
apiVersion: batch/v1
kind: Job
metadata:
name: cilium-install
namespace: kube-system
spec:
backoffLimit: 10
template:
metadata:
labels:
app: cilium-install
spec:
restartPolicy: OnFailure
tolerations:
- operator: Exists
- effect: NoSchedule
operator: Exists
- effect: NoExecute
operator: Exists
- effect: PreferNoSchedule
operator: Exists
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoExecute
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: PreferNoSchedule
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: node-role.kubernetes.io/control-plane
operator: Exists
serviceAccount: cilium-install
serviceAccountName: cilium-install
hostNetwork: true
containers:
- name: cilium-install
image: quay.io/cilium/cilium-cli:latest
env:
- name: KUBERNETES_SERVICE_HOST
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: status.podIP
- name: KUBERNETES_SERVICE_PORT
value: "6443"
command:
- cilium
- install
- -f
- /tmp/values.yaml
- --wait
- --wait-duration=600s
volumeMounts:
- name: config
mountPath: /tmp/values.yaml
subPath: values.yaml
readOnly: true
volumes:
- name: config
configMap:
name: cilium-values
```

Because there is no CNI present at installation time the kubernetes.default.svc cannot be used to install Cilium, to overcome this limitation we'll utilize the host network connection to connect back to itself with 'hostNetwork: true' in tandem with the environment variables KUBERNETES_SERVICE_PORT and KUBERNETES_SERVICE_HOST.

The job runs a container to install cilium to your liking, after the job is finished Cilium can be managed/operated like usual.
The job runs a container to install Cilium to your liking, after the job is finished Cilium can be managed/operated like usual.

The above can be combined exchanged with for example Method 3 to host arbitrary configurations externally but render/run them at bootstrap time.

Expand All @@ -418,10 +521,10 @@ The above can be combined exchanged with for example Method 3 to host arbitrary
- There are some gotchas when using Talos and Cilium on the Google cloud platform when using internal load balancers.
For more details: [GCP ILB support / support scope local routes to be configured](https://github.com/siderolabs/talos/issues/4109)

- When using Talos `forwardKubeDNSToHost=true` option (which is enabled by default) in combination with cilium `bpf.masquerade=true`.
- When using Talos `forwardKubeDNSToHost=true` option (which is enabled by default) in combination with Cilium `bpf.masquerade=true`.
There is a known issue that causes `CoreDNS` to not work correctly.
As a workaround, configuring `forwardKubeDNSToHost=false` resolves the issue.
For more details see [the discusssion here](https://github.com/siderolabs/talos/pull/9200)
For more details see [the discussion here](https://github.com/siderolabs/talos/pull/9200)

## Other things to know

Expand Down