某个calico-node 的pod一直在restart

[peterbooy]

查看log结果:
`
[root@cl-net /]# kubectl describe pod calico-node-nhfkn -n kube-system
Name: calico-node-nhfkn
Namespace: kube-system
Priority: 2000001000
Priority Class Name: system-node-critical
Node: tx-node/172.21.0.13
Start Time: Tue, 28 Jun 2022 14:50:38 +0800
Labels: controller-revision-hash=55dddb9c8d
k8s-app=calico-node
pod-template-generation=1
Annotations: scheduler.alpha.kubernetes.io/critical-pod:
Status: Running
IP: 172.21.0.13
IPs:
IP: 172.21.0.13
Controlled By: DaemonSet/calico-node
Init Containers:
upgrade-ipam:
Container ID: docker://cba9fdad046b953b95a2eb9d9b1cc9c7b08720aa1e837f0fb3413935a54e2c9c
Image: calico/cni:v3.13.1
Image ID: docker-pullable://calico/cni@sha256:c699d5ec4d0799ca5785e9134cfb1f55a1376ebdbb607f5601394736fceef7c8
Port:
Host Port:
Command:
/opt/cni/bin/calico-ipam
-upgrade
State: Terminated
Reason: Completed
Exit Code: 0
Started: Tue, 28 Jun 2022 14:50:39 +0800
Finished: Tue, 28 Jun 2022 14:50:39 +0800
Ready: True
Restart Count: 0
Environment:
KUBERNETES_NODE_NAME: (v1:spec.nodeName)
CALICO_NETWORKING_BACKEND: <set to the key 'calico_backend' of config map 'calico-config'> Optional: false
Mounts:
/host/opt/cni/bin from cni-bin-dir (rw)
/var/lib/cni/networks from host-local-net-dir (rw)
/var/run/secrets/kubernetes.io/serviceaccount from calico-node-token-6rzvd (ro)
install-cni:
Container ID: docker://72bf3d6713495bef76842d4b7a79d790cab2d63a38a09a772063ee55db60b914
Image: calico/cni:v3.13.1
Image ID: docker-pullable://calico/cni@sha256:c699d5ec4d0799ca5785e9134cfb1f55a1376ebdbb607f5601394736fceef7c8
Port:
Host Port:
Command:
/install-cni.sh
State: Terminated
Reason: Completed
Exit Code: 0
Started: Tue, 28 Jun 2022 14:50:40 +0800
Finished: Tue, 28 Jun 2022 14:50:40 +0800
Ready: True
Restart Count: 0
Environment:
CNI_CONF_NAME: 10-calico.conflist
CNI_NETWORK_CONFIG: <set to the key 'cni_network_config' of config map 'calico-config'> Optional: false
KUBERNETES_NODE_NAME: (v1:spec.nodeName)
CNI_MTU: <set to the key 'veth_mtu' of config map 'calico-config'> Optional: false
SLEEP: false
Mounts:
/host/etc/cni/net.d from cni-net-dir (rw)
/host/opt/cni/bin from cni-bin-dir (rw)
/var/run/secrets/kubernetes.io/serviceaccount from calico-node-token-6rzvd (ro)
flexvol-driver:
Container ID: docker://6c4722cccfab9214ac28f9a42c75c77bd9c849c4c1ab131b9df05c4abec97866
Image: calico/pod2daemon-flexvol:v3.13.1
Image ID: docker-pullable://calico/pod2daemon-flexvol@sha256:762e1d75e4e031066fe4b67b6f386520923b48067354bb6e9e53215957a4c6b4
Port:
Host Port:
State: Terminated
Reason: Completed
Exit Code: 0
Started: Tue, 28 Jun 2022 14:50:41 +0800
Finished: Tue, 28 Jun 2022 14:50:41 +0800
Ready: True
Restart Count: 0
Environment:
Mounts:
/host/driver from flexvol-driver-host (rw)
/var/run/secrets/kubernetes.io/serviceaccount from calico-node-token-6rzvd (ro)
Containers:
calico-node:
Container ID: docker://fef1c042b8bb34cd7a3a14c45a13ceae9ddb150a2980a3d2cfa6900074102fca
Image: calico/node:v3.13.1
Image ID: docker-pullable://calico/node@sha256:f24c59e93881178bfae85ee1375889fe9399edf1e15b0026713b2870cef079be
Port:
Host Port:
State: Running
Started: Tue, 28 Jun 2022 15:39:48 +0800
Last State: Terminated
Reason: Error
Exit Code: 137
Started: Tue, 28 Jun 2022 15:38:45 +0800
Finished: Tue, 28 Jun 2022 15:39:47 +0800
Ready: False
Restart Count: 17
Requests:
cpu: 250m
Liveness: exec [/bin/calico-node -felix-live -bird-live] delay=10s timeout=1s period=10s #success=1 #failure=6
Readiness: exec [/bin/calico-node -felix-ready -bird-ready] delay=0s timeout=1s period=10s #success=1 #failure=3
Environment:
DATASTORE_TYPE: kubernetes
WAIT_FOR_DATASTORE: true
NODENAME: (v1:spec.nodeName)
CALICO_NETWORKING_BACKEND: <set to the key 'calico_backend' of config map 'calico-config'> Optional: false
CLUSTER_TYPE: k8s,bgp
IP: autodetect
CALICO_IPV4POOL_IPIP: Always
FELIX_IPINIPMTU: <set to the key 'veth_mtu' of config map 'calico-config'> Optional: false
CALICO_DISABLE_FILE_LOGGING: true
FELIX_DEFAULTENDPOINTTOHOSTACTION: ACCEPT
FELIX_IPV6SUPPORT: false
FELIX_LOGSEVERITYSCREEN: info
FELIX_HEALTHENABLED: true
Mounts:
/lib/modules from lib-modules (ro)
/run/xtables.lock from xtables-lock (rw)
/var/lib/calico from var-lib-calico (rw)
/var/run/calico from var-run-calico (rw)
/var/run/nodeagent from policysync (rw)
/var/run/secrets/kubernetes.io/serviceaccount from calico-node-token-6rzvd (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
lib-modules:
Type: HostPath (bare host directory volume)
Path: /lib/modules
HostPathType:
var-run-calico:
Type: HostPath (bare host directory volume)
Path: /var/run/calico
HostPathType:
var-lib-calico:
Type: HostPath (bare host directory volume)
Path: /var/lib/calico
HostPathType:
xtables-lock:
Type: HostPath (bare host directory volume)
Path: /run/xtables.lock
HostPathType: FileOrCreate
cni-bin-dir:
Type: HostPath (bare host directory volume)
Path: /opt/cni/bin
HostPathType:
cni-net-dir:
Type: HostPath (bare host directory volume)
Path: /etc/cni/net.d
HostPathType:
host-local-net-dir:
Type: HostPath (bare host directory volume)
Path: /var/lib/cni/networks
HostPathType:
policysync:
Type: HostPath (bare host directory volume)
Path: /var/run/nodeagent
HostPathType: DirectoryOrCreate
flexvol-driver-host:
Type: HostPath (bare host directory volume)
Path: /usr/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds
HostPathType: DirectoryOrCreate
calico-node-token-6rzvd:
Type: Secret (a volume populated by a Secret)
SecretName: calico-node-token-6rzvd
Optional: false
QoS Class: Burstable
Node-Selectors: kubernetes.io/os=linux
Tolerations: :NoScheduleop=Exists
:NoExecuteop=Exists
CriticalAddonsOnly op=Exists
node.kubernetes.io/disk-pressure:NoSchedule op=Exists
node.kubernetes.io/memory-pressure:NoSchedule op=Exists
node.kubernetes.io/network-unavailable:NoSchedule op=Exists
node.kubernetes.io/not-ready:NoExecute op=Exists
node.kubernetes.io/pid-pressure:NoSchedule op=Exists
node.kubernetes.io/unreachable:NoExecute op=Exists
node.kubernetes.io/unschedulable:NoSchedule op=Exists
Events:
Type Reason Age From Message

---

Normal Scheduled 49m default-scheduler Successfully assigned kube-system/calico-node-nhfkn to tx-node
Normal Pulled 49m kubelet Container image "calico/cni:v3.13.1" already present on machine
Normal Started 49m kubelet Started container upgrade-ipam
Normal Created 49m kubelet Created container upgrade-ipam
Normal Started 49m kubelet Started container install-cni
Normal Pulled 49m kubelet Container image "calico/cni:v3.13.1" already present on machine
Normal Created 49m kubelet Created container install-cni
Normal Started 49m kubelet Started container flexvol-driver
Normal Pulled 49m kubelet Container image "calico/pod2daemon-flexvol:v3.13.1" already present on machine
Normal Created 49m kubelet Created container flexvol-driver
Normal Pulled 49m kubelet Container image "calico/node:v3.13.1" already present on machine
Normal Created 49m kubelet Created container calico-node
Normal Started 49m kubelet Started container calico-node
Warning Unhealthy 48m (x6 over 49m) kubelet Readiness probe failed: calico/node is not ready: felix is not ready: Get http://localhost:9099/readiness: dial tcp 127.0.0.1:9099: connect: connection refused
Normal Killing 48m kubelet Container calico-node failed liveness probe, will be restarted
Warning Unhealthy 14m (x80 over 49m) kubelet Liveness probe failed: calico/node is not ready: Felix is not live: Get http://localhost:9099/liveness: dial tcp 127.0.0.1:9099: connect: connection refused
`

help me

[ihengshuai]

I didn‘t meet the problem,you can describe again