orbiter issue with existing kubeconfig

[daveboe]

Hi Guys,
we have an issue with one of our clusters.
Orbiter log shows the following messages:

ts="2021-07-16T12:55:51Z" evt="Hostname desired" machine="master03" provider="abxnet"
ts="2021-07-16T12:55:51Z" evt="Hostname desired" machine="master01" provider="abxnet"
ts="2021-07-16T12:55:51Z" evt="Hostname desired" machine="worker03" provider="abxnet"
ts="2021-07-16T12:55:51Z" evt="Hostname desired" machine="worker01" provider="abxnet"
ts="2021-07-16T12:55:51Z" evt="Hostname desired" machine="master02" provider="abxnet"
ts="2021-07-16T12:55:52Z" evt="Hostname desired" machine="worker02" provider="abxnet"
ts="2021-07-16T12:55:54Z" msg="Scaling is not done yet" cluster="t08"
ts="2021-07-16T12:55:54Z" err="initializing a cluster is not supported when kubeconfig exists or the flag --recur is passed" src="/home/runner/work/orbos/orbos/internal/operator/orbiter/takeoff.go:165"
ts="2021-07-16T12:55:55Z" msg="Repository pushed" repository="ssh://[email protected]:2222/plattformen/aap/cluster-08/t08-cluster.git"
ts="2021-07-16T12:55:55Z" msg="Iteration done" took="6.167976841s"

In the current state, it is not possible to add new nodes or anything else.

What we have done so far:
• Orbiter Version was 3.0.5
• Kubeconfig was expired and we updated it using orbctl writesecret after doing so the above-mentioned message appeared in the log. Not sure if updating the kubeconfig is the cause
• We then tried to update orbiter version to v4.1.4 which updated the node-agents on the nodes but not the deployment, so we manually updated the image in the orbiter deployment

Unfortunately, this did not fix the issue and we still have the error in the orbiter log

Can you tell us why orbiter is trying to initialize an already initialized cluster?
How can we fix this issue?

If you need more information from us, do not hesitate to ask!

Information about the cluster
Kubernetes Version: v1.18.5
ORBOS Version: v4.1.4 (& v3.0.5)
Provider: Static Provider with CentOS 7

[eliobischof]

Hi @daveboe
Could you please doublecheck if you've written a fully functional kubeconfig using the orbctl readsecret command?

[daveboe]

Hi @eliobischof
There was a typo in the current-context of the kubeconfig. After fixing the kubeconfig and writing it again with orbctl writesecret it looks good.
Would it be possible to add some error details (original error message) to the log message in a future version?

[eliobischof]

Good to see that it works now.

Would it be possible to add some error details (original error message) to the log message in a future version?

I've opened an issue for better error messages #782