How can i use CSI with an existing LINSTOR Cluster, managed outside Kubernetes? #492
Comments
lomonosov77
changed the title
|
That will be difficult. Why are your host names different between kubernetes and the host OS? |
|
linstor is installed on a different server and is not managed by the k8s cluster. These are 2 different hardware. |
|
You still need at least a satellite running on the kubernetes workers. You can use the operator to connect to your existing cluster: https://github.com/piraeusdatastore/piraeus-operator/blob/v2/docs/how-to/external-controller.md |
|
I tried that. And how do I add linstor satellites that are outside of the Kubernetes cluster? |
|
Well, you can start with setting up the Controller and external Satellites, including registering the satellites with Then, you can use the Operator to connect to the existing cluster. The Operator will register the satellites it control in the Kubernetes cluster. |
|
The whole problem is that the satelite pod won't start. The problem is in drbd-shutdown-guard. _**
**_ |
|
Same issue with |
|
See #426 (comment) for a workaround, and the overall issue |
|
thanks, recipe from #426 (comment) helps, apiVersion: piraeus.io/v1
kind: LinstorCluster
metadata:
name: linstorcluster
spec:
externalController:
url: https://my-controller-ip:3371
apiTLS:
apiSecretName: linstor-api-tls
clientSecretName: linstor-client-tls
csiControllerSecretName: linstor-client-tls
csiNodeSecretName: linstor-client-tlswith secrets set according to guide https://github.com/piraeusdatastore/piraeus-operator/blob/v2/docs/how-to/api-tls.md#provision-keys-and-certificates-using-openssl but in logs of linstor-wait-node-online got lines like this: |
|
Can you verify that the |
|
I'd entered into # check env
env | grep "^LS"
LS_USER_CERTIFICATE=-----BEGIN CERTIFICATE-----
LS_CONTROLLERS=https://my-controller-ip:3371
LS_USER_KEY=-----BEGIN RSA PRIVATE KEY-----
LS_ROOT_CA=-----BEGIN CERTIFICATE-----
# check connectivity
apt install curl
echo "$LS_USER_CERTIFICATE" > client.crt
echo "$LS_USER_KEY" > client.key
echo "$LS_ROOT_CA" > ca.crt
curl -I --cacert ca.crt --cert client.crt --key client.key $LS_CONTROLLERS
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: origin, content-type, accept, authorization
Access-Control-Allow-Credentials: true
Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS, HEAD
Content-Length: 294
Content-Type: text/plain
# install linstor client utility
apt install gpg
curl -o /tmp/package-signing-pubkey.asc https://packages.linbit.com/package-signing-pubkey.asc
gpg --yes -o /etc/apt/trusted.gpg.d/linbit-keyring.gpg --dearmor /tmp/package-signing-pubkey.asc
PVERS=7 && echo "deb [signed-by=/etc/apt/trusted.gpg.d/linbit-keyring.gpg] \
http://packages.linbit.com/public/ proxmox-$PVERS drbd-9" | tee -a /etc/apt/sources.list.d/linbit.list
deb [signed-by=/etc/apt/trusted.gpg.d/linbit-keyring.gpg] http://packages.linbit.com/public/ proxmox-7 drbd-9
apt update && apt install linstor-client
# run
linstor n l
Error: Error reading response from https://my-controller-ip:3371: Remote end closed connection without response
Error: Unable to connect to linstor://localhost:3370: [Errno 111] Connection refused
# run with explicit args
linstor --certfile client.crt --key client.key --cafile ca.crt --controllers my-controller-ip:3371 n l
╭─────────────────────────────────────────────────────────────╮
┊ Node ┊ NodeType ┊ Addresses ┊ State ┊
╞═════════════════════════════════════════════════════════════╡
┊ stor01 ┊ SATELLITE ┊ x.x.x.32:3366 (PLAIN) ┊ Online ┊
┊ stor ┊ SATELLITE ┊ x.x.x.27:3366 (PLAIN) ┊ Online ┊
┊ worker01 ┊ SATELLITE ┊ x.x.x.225:3366 (PLAIN) ┊ Online ┊
┊ worker02 ┊ SATELLITE ┊ x.x.x.226:3366 (PLAIN) ┊ Online ┊
┊ worker03 ┊ SATELLITE ┊ x.x.x.227:3366 (PLAIN) ┊ Online ┊
┊ worker04 ┊ SATELLITE ┊ x.x.x.228:3366 (PLAIN) ┊ Online ┊
┊ worker05 ┊ SATELLITE ┊ x.x.x.229:3366 (PLAIN) ┊ EVICTED ┊
┊ stor02 ┊ SATELLITE ┊ x.x.x.26:3366 (PLAIN) ┊ Online ┊
┊ stor03 ┊ SATELLITE ┊ x.x.x.37:3366 (PLAIN) ┊ Online ┊
┊ stor04 ┊ SATELLITE ┊ x.x.x.34:3366 (PLAIN) ┊ Online ┊
╰─────────────────────────────────────────────────────────────╯ |
This method helps to start |
|
It does have a purpose, namely to improve DRBD handling during shut down: By default, DRBD volumes created by Piraeus will suspend IO if connection is lost. During node shutdown, the DRBD devices will remain configured, and mounted, unless the node was properly evicted in Kubernetes first. But during shutdown, the Pod network will stop working, at which point DRBD can no longer access the peers, so IO is suspended. Eventually systemd will come around and try to unmount all remaining mounts, including those mounts for containers using Piraeus Volumes. Then the unmount gets stuck, because DRBD is suspending IO. You would have to do a hard reset, as most systemd by default do not have a unmount time out in systemd. The shutdown-guard is there to run during node shutdown and force the DRBD device to report IO errors instead: then unmounting can continue. So while not strictly necessary, it's definitly "nice to have". |
I have the same problem. and linstor --certfile client.crt --key client.key --cafile ca.crt --controllers $LS_CONTROLLERS n l |
Hello! I have k8s cluster and i want to use Linstor CSI Plugin with an existing LINSTOR Cluster, managed outside Kubernetes.
I've used manifest - https://github.com/piraeusdatastore/linstor-csi/blob/master/examples/k8s/deploy/linstor-csi-1.19.yaml , but KUBE_NODE_NAME value equal name of k8s node and it's not the same in my case.
Who knows how to set it up, help me, please!
The text was updated successfully, but these errors were encountered: