Skip to content

Commit 90e65ca

Browse files
authored
Merge pull request #14 from cozystack/docs/operating
Document adoption, teardown and CRD upgrades for operators
2 parents 73bbdf3 + 6c809fc commit 90e65ca

2 files changed

Lines changed: 127 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -124,6 +124,9 @@ The controller image is published as
124124
`ghcr.io/cozystack/address-controller:main` (plus `main-<sha>` and semver tags)
125125
by the release workflow on every push to main.
126126

127+
Adopting addresses Services already hold, removing the controllers, and the
128+
CRD changes each release makes are in [docs/operating.md](docs/operating.md).
129+
127130

128131
## Admission policy
129132

‎docs/operating.md‎

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
# Operating address-controller
2+
3+
Procedures for cluster admins. The [design](design.md) explains why the
4+
controller behaves as described here.
5+
6+
## Adopting an address a Service already holds
7+
8+
A Service pinned by hand to an address of a class's range keeps it while
9+
it moves under the ledger, because the driver's pin equals the existing
10+
one:
11+
12+
1. Create an `IPAddressClaim` in the Service's namespace that names the
13+
class and, in `spec.addressName`, the `IPAddress` of step 2. The MetalLB
14+
driver does not provision for a claim that names an address, so the
15+
claim waits `Pending` for that one. Where the optional `spec.addressName` admission
16+
policy is enabled, this takes an allowlisted principal.
17+
18+
```yaml
19+
apiVersion: local.sdn.cozystack.io/v1alpha1
20+
kind: IPAddressClaim
21+
metadata:
22+
name: web
23+
namespace: tenant-a
24+
spec:
25+
className: public
26+
addressName: ip-203-0-113-7
27+
```
28+
29+
2. Create the `IPAddress` pre-bound to it:
30+
31+
```yaml
32+
apiVersion: local.sdn.cozystack.io/v1alpha1
33+
kind: IPAddress
34+
metadata:
35+
name: ip-203-0-113-7
36+
spec:
37+
address: 203.0.113.7
38+
className: public
39+
reclaimPolicy: Retain
40+
claimRef: {namespace: tenant-a, name: web}
41+
source: {fromClass: {}}
42+
```
43+
44+
The core accepts the pre-binding only if the class and family match the
45+
claim and the claim holds no address of that family yet; otherwise the
46+
claim records a `PreBindingRefused` event.
47+
3. Annotate the Service with `local.sdn.cozystack.io/ip-address-claim: web`.
48+
The driver records the association and pins the same address.
49+
50+
The hand-written pin stays on the Service. Where metallb-iad's optional
51+
admission policy is enabled, removing it later takes an allowlisted
52+
principal. Without that policy, removing it is not gated, and only the
53+
driver's conflict detection reports the address going to another Service.
54+
55+
Creating the `IPAddress` before the claim does not work: an address whose
56+
`claimRef` names a missing claim is reclaimed at once, `Released` under
57+
`Retain` and deleted under `Delete`.
58+
59+
## Removing the controllers
60+
61+
Claims, addresses and classes carry finalizers that the core removes
62+
(`claim-protection`, `address-protection`, `class-protection`), and the
63+
MetalLB driver adds `teardown` to the addresses it creates. While objects
64+
remain and a controller is not running, deleting them, or a namespace that
65+
holds claims, hangs. Tear down in this order:
66+
67+
1. Remove the claim annotation from Services, or delete the Services.
68+
2. Delete every `IPAddressClaim`. The core reclaims each address by its
69+
`reclaimPolicy`.
70+
3. Delete the `IPAddress` objects left behind (`Released`, `Available`).
71+
The driver releases each backend reservation.
72+
4. Delete the `IPAddressClass` objects. Each goes once no address of it is
73+
left.
74+
5. Uninstall the driver, then the core. `helm uninstall` leaves the CRDs.
75+
76+
If the controllers are already gone, clear the finalizers by hand, for
77+
example:
78+
79+
```sh
80+
kubectl get ipaddressclaims.local.sdn.cozystack.io -A \
81+
-o jsonpath='{range .items[*]}{.metadata.namespace} {.metadata.name}{"\n"}{end}' |
82+
while read -r ns name; do
83+
kubectl -n "$ns" patch ipaddressclaims.local.sdn.cozystack.io "$name" \
84+
--type=merge -p '{"metadata":{"finalizers":null}}'
85+
done
86+
for kind in ipaddresses ipaddressclasses; do
87+
kubectl get "$kind.local.sdn.cozystack.io" -o name |
88+
xargs -r -n1 kubectl patch --type=merge -p '{"metadata":{"finalizers":null}}'
89+
done
90+
```
91+
92+
The group suffix matters: `ipaddresses` alone resolves to the Kubernetes
93+
`networking.k8s.io` kind. Backend state the driver would have released,
94+
such as MetalLB placeholder Services, is then left for you to delete.
95+
96+
### Deleting a class
97+
98+
A class is held until no `IPAddress` of it is left, in any phase. Under
99+
`reclaimPolicy: Retain` a deleted claim leaves its addresses `Released`,
100+
and a class's `releaseDelay` returns them to `Available` rather than
101+
deleting them, so a class that has served claims keeps addresses that hold
102+
it. Delete those `IPAddress` objects, and the class goes. Until then the
103+
class reports a `ClassInUse` event naming them.
104+
105+
## Quarantining an address
106+
107+
`ipaddress.local.sdn.cozystack.io/held: "true"` on a `Released` address
108+
keeps its class's `releaseDelay` from returning it to the pool. The
109+
annotation pauses the return, not the delay: removing it from an address
110+
that has been `Released` for longer than `releaseDelay` returns the
111+
address on the next pass.
112+
113+
## Upgrading the CRDs
114+
115+
The API is `v1alpha1` with no conversion webhook. Helm installs the chart's
116+
`crds/` on first install only; `make install-crds` applies the current
117+
CRDs, and a distribution that replaces CRDs on upgrade does the same. A
118+
field removed or renamed in a later CRD invalidates stored objects that
119+
still set it, so every release states its schema changes:
120+
121+
| release | schema change |
122+
|---|---|
123+
| v0.1.0 | initial `IPAddress`, `IPAddressClaim`, `IPAddressClass` |
124+
| unreleased | `IPAddressClass.spec.releaseDelay` added; `IPAddress` gains a core-set `Bound` condition |

0 commit comments

Comments
 (0)