Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimization of kubeedge deployment documentation: Handling issues with insufficient edge node resources for normal deployment. #551

Open
wants to merge 22 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
c96e24a
kubeedge部署文档内容优化:边缘节点资源紧张无法正常部署的问题处理
hyp4293 May 22, 2024
571e55c
Optimization of kubeedge deployment documentation: Handling issues wi…
hyp4293 May 23, 2024
a294732
镜像预加载功能指导文档
hyp4293 May 24, 2024
b2f2d09
Merge remote-tracking branch 'upstream/master'
hyp4293 May 25, 2024
d75689d
New Blog for Release KubeEdge v1.11
1Shubham7 May 15, 2024
214001a
fixed the release date
1Shubham7 May 16, 2024
b512169
New Blog for Release KubeEdge v1.11
1Shubham7 May 16, 2024
cd9db44
New Blog for Release KubeEdge v1.14
1Shubham7 May 16, 2024
755bcee
added the release date
1Shubham7 May 16, 2024
11c7d68
update version for release 1.17
fisherxu May 20, 2024
18d7615
kubeedge部署文档内容优化:边缘节点资源紧张无法正常部署的问题处理
hyp4293 May 22, 2024
345a540
Optimization of kubeedge deployment documentation: Handling issues wi…
hyp4293 May 23, 2024
4af8dd1
镜像预加载功能指导文档
hyp4293 May 24, 2024
1712f6a
Replacing Twitter with X
1Shubham7 May 17, 2024
603928a
Merge branch 'master' of https://github.com/hyp4293/website
hyp4293 Jul 8, 2024
d59f6bc
KubeEdge Image PrePull Feature Guide Document
hyp4293 May 25, 2024
52ac1e3
KubeEdge Image PrePull Feature Guide Document
hyp4293 Jun 12, 2024
bf2a75e
6/7/2024
hyp4293 Jul 6, 2024
3bb5a84
add case study about raisecom tech
wbc6080 May 27, 2024
0c2e057
add case study about XingHai IoT
wbc6080 May 27, 2024
a5f98d9
Part 1: improving the install with keadm docs
1Shubham7 May 17, 2024
628e9d7
fix merge
hyp4293 Jul 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
# KubeEdge 镜像预加载功能指导文档
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please push a separate PR to add imapePrepull guide. And it belongs to docs but not blog. A English version is also needed.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have resubmitted as requested


KubeEdge 1.16版本引入了镜像预下载新特性,用户可以通过ImagePrePullJob的Kubernetes API提前在边缘节点上加载镜像,该特性支持在批量边缘节点或节点组中预下载多个镜像,帮助减少加载镜像在应用部署或更新过程,尤其是大规模场景中,带来的失败率高、效率低下等问题。

镜像预下载API示例:

```
apiVersion: operations.kubeedge.io/v1alpha1
kind: ImagePrePullJob
metadata:
name: imageprepull-example
labels:
description:ImagePrePullLabel
spec:
imagePrePullTemplate:
images:
- image1
- image2
nodes:
- edgenode1
- edgenode2
checkItems:
- "disk"
failureTolerate: "0.3"
concurrency: 2
timeoutSeconds: 180
retryTimes: 1

```


## 1. 准备工作

**选用示例:Nginx Demo**

nginx是一个轻量级镜像,用户无需任何环境即可进行此演示。nginx镜像将会提前上传到一个私有镜像仓库中。用户可以从云端调用预加载功能API,将私有镜像仓库中的nginx镜像,提前下发到边缘节点中。


**1)本示例要求KubeEdge版本必须是v1.16.0+,kubernetes版本是v1.27.0+,此次选择的版本是KubeEdge v1.16.0,Kubernetes版本是v1.27.3**

```
[root@ke-cloud ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
cloud.kubeedge Ready control-plane,master 3d v1.27.3
edge.kubeedge Ready agent,edge 2d v1.27.7-kubeedge-v1.16.0

说明:本文接下来的验证将使用边缘节点edge.kubeedge进行,如果你参考本文进行相关验证,后续边缘节点名称的配置需要根据你的实际情况进行更改。
```

**2)确保k8s apiserver开启了以下配置:**


```
taskManager:
enable: true // 由false修改为true
```
可以通过命令修改kubectl edit configmap cloudcore -n kubeedge文件,并重启k8s-apiserver组件的cloudcore来进行更改。

**3)准备示例代码:**

yaml文件示例代码
```
apiVersion: operations.kubeedge.io/v1alpha1
kind: ImagePrePullJob
metadata:
name: imageprepull-example
spec:
imagePrePullTemplate:
concurrency: 1
failureTolerate: '0.1'
images:
- test:nginx
nodeNames:
- edge.kubeedge
imageSecrets: default/secret
retryTimes: 1
timeoutSeconds: 120

```

## 2. 准备私有镜像仓的镜像和Secret
在这里准备了一个阿里云的私有镜像仓用作演示:registry.cn-hangzhou.aliyuncs.com/,使用的演示空间为jilimoxing。实际操作过程中可以依据真实情况进行修改

**1)推送nginx进入私有镜像仓**
```
[root@cloud ~]# docker tag nginx registry.cn-hangzhou.aliyuncs.com/jilimoxing/test:nginx
[root@cloud crds~]# docker push registry.cn-hangzhou.aliyuncs.com/jilimoxing/test:nginx
```

**2)在云端创建Secret**

使用Kubectl create secret docker-registry生成私有镜像仓库的secret,根据你的实际情况来进行修改

```
[root@cloud ~]# kubectl create secret docker-registry my-secret \
--docker-server=registry.cn-hangzhou.aliyuncs.com \
--docker-username=23021*****@qq.com \
--docker-password=Xy***** \
--docker-email=23021*****@qq.com

[root@cloud ~]# kubectl get secret -A
NAMESPACE NAME TYPE DATA AGE
default my-secret kubernetes.io/dockerconfigjson 1 31s

```

## 3. 创建Yaml文件

**1)修改代码**

在云端节点上创建yaml文件,需要修改对应的images信息以及imageSecrets信息,保持和所需要预加载的镜像仓库secret一致,如下所示:
```

[root@ke-cloud ~]# vim imageprepull.yaml

apiVersion: operations.kubeedge.io/v1alpha1
kind: ImagePrePullJob
metadata:
name: imageprepull-example
spec:
imagePrePullTemplate:
concurrency: 1
failureTolerate: '0.1'
images:
- registry.cn-hangzhou.aliyuncs.com/jilimoxing/test:nginx
nodeNames:
- edge.kubeedge
imageSecrets: default/my-secret
retryTimes: 1
timeoutSeconds: 120

```

**2)执行文件**


```
[root@ke-cloud ~]# kubectl apply -f imageprepull.yaml
```


## 4. 检查边缘节点镜像是否预加载成功

进入边缘端,使用命令ctr -n k8s.io i ls进行查看
```
[root@edge ~]# ctr -n k8s.io i ls
```
找到对应的镜像已被预加载成功
```
REF TYPE DIGEST SIZE PLATFORMS LABELS
registry.cn-hangzhou.aliyuncs.com/jilimoxing/test:nginx application/vnd.docker.distribution.manifest.v2+json sha256:73e957703f1266530db0aeac1fd6a3f87c1e59943f4c13eb340bb8521c6041d7 67.3 MiB linux/amd64
```

## 5. 其他

**1)更多的KubeEdge官方示例请参考 https://github.com/kubeedge/examples**
12 changes: 11 additions & 1 deletion docs/setup/install-with-keadm.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,16 @@ keadm init --set server.advertiseAddress="THE-EXPOSED-IP" --set server.nodeName=
If you are familiar with the helm chart installation, please refer to [KubeEdge Helm Charts](https://github.com/kubeedge/kubeedge/tree/master/manifests/charts).


**SPECIAL SCENARIO:**
In the case of insufficient qualifications for edge nodes, we need to label them to prevent some applications from extending to edge nodes. `Kube-proxy` and some others is not required at the edge.We can handle it accordingly.

```
kubectl get daemonset -n kube-system |grep -v NAME |awk '{print $1}' | xargs -n 1 kubectl patch daemonset -n kube-system --type='json' -p='[{"op": "replace","path": "/spec/template/spec/affinity","value":{"nodeAffinity":{"requiredDuringSchedulingIgnoredDuringExecution":{"nodeSelectorTerms":[{"matchExpressions":[{"key":"node-role.kubernetes.io/edge","operator":"DoesNotExist"}]}]}}}}]'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/kubeedge/website/pull/551/files#diff-58f39633768f7241615b445ab20cf0842acb0f0f38597ca61e8c4afceeeac56fR308 has a similar description, we need to let users know that kube-proxy and some others is not required at the edge

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will remind users that kube-proxy and some other components are not required at the edge. They can handle it accordingly.


```

To handle kube-proxy, you can refer to the [two methods](#anchor-name) mentioned in the " Enable `kubectl logs` Feature " section of this document.

### keadm manifest generate

You can also get the manifests with `keadm manifest generate`.
Expand Down Expand Up @@ -340,7 +350,7 @@ Before deploying metrics-server , `kubectl logs` feature must be activated:
```
If you fail to restart edgecore, check if that is because of `kube-proxy` and kill it. **kubeedge** reject it by default, we use a succedaneum called [edgemesh](https://github.com/kubeedge/kubeedge/blob/master/docs/proposals/edgemesh-design.md)

**Note:** the importance is to avoid `kube-proxy` being deployed on edgenode. There are two methods to solve it:
**Note:** the importance is to avoid `kube-proxy` being deployed on edgenode.<a name="anchor-name"> There are two methods to solve it: </a >

1. Add the following settings by calling `kubectl edit daemonsets.apps -n kube-system kube-proxy`:
``` yaml
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,17 @@ KubeEdge cloudcore is running, For logs visit: /var/log/kubeedge/cloudcore.log

当您看到以上信息,说明 KubeEdge 的云端组件 cloudcore 已经成功运行。

**特殊场景:**
边缘计算的硬件条件不好的情况,这里我们需要打上标签,让一些应用不扩展到edge节点上去。 kube-proxy和其他的一些应用不是必须部署在边缘端,所以我们可以对他们进行处理。


```
kubectl get daemonset -n kube-system |grep -v NAME |awk '{print $1}' | xargs -n 1 kubectl patch daemonset -n kube-system --type='json' -p='[{"op": "replace","path": "/spec/template/spec/affinity","value":{"nodeAffinity":{"requiredDuringSchedulingIgnoredDuringExecution":{"nodeSelectorTerms":[{"matchExpressions":[{"key":"node-role.kubernetes.io/edge","operator":"DoesNotExist"}]}]}}}}]'

```

如何处理kube-proxy,可以参考本文中在'启用 kubectl logs 功能'部分提到的 [2种方法](#anchor-name)

### keadm beta init

如果您想要使用容器化方式部署云端组件 cloudcore ,您可以使用 `keadm beta init` 进行云端组件安装。
Expand Down Expand Up @@ -287,7 +298,7 @@ KubeEdge edgecore is running, For logs visit: /var/log/kubeedge/edgecore.log
如果您无法重启 edgecore,请检查是否是由于 `kube-proxy` 的缘故,同时杀死这个进程。 **kubeedge**
默认不纳入该进程,我们使用 [edgemesh](https://github.com/kubeedge/kubeedge/blob/master/docs/proposals/edgemesh-design.md) 来进行替代

**注意:** 可以考虑避免 `kube-proxy` 部署在 edgenode 上。有两种解决方法:
**注意:** 可以考虑避免 `kube-proxy` 部署在 edgenode 上。<a name="anchor-name">有两种解决方法:</a >

1. 通过调用 `kubectl edit daemonsets.apps -n kube-system kube-proxy` 添加以下设置:

Expand Down