Skip to content

KubeEdge

Jump to bottom
Krijn Doekemeijer edited this page Nov 10, 2023 · 1 revision

How to run KubeEdge locally

Use KVM to run KubeEdge on your local machine

Setting up the machines

  1. Get the Ubuntu 20.04 server iso: https://releases.ubuntu.com/20.04/
  2. Install KVM: https://linuxize.com/post/how-to-install-kvm-on-ubuntu-20-04/

To manage VMs, either use the virsh command, the qemu-system-x86_64 command , or use the Virtual Machine Manager application (GUI).

  1. Create a VM called 'template' using the Ubuntu 20.04 iso. Install the OpenSSH server (this is an option during the installation). Keep this VM as is, we will clone it to create VMs we will actually use. In this way if a VM gets corrupted in some way and you need a new one, you can simply clone this template without having to install Ubuntu from scratch.
  2. Clone the template VM three times, call the first VM 'cloud', the second VM 'edge1' and the third 'endpoint1'. These will represent a single KubeEdge cloudcore, edgecore and endpoint system.
  3. Change the IP of the VMs: https://bobcares.com/blog/virsh-set-ip-address/ and https://serverfault.com/questions/627238/kvm-libvirt-how-to-configure-static-guest-ip-addresses-on-the-virtualisation-ho

To login via SSH, start the VM and do ssh username@ip with the username being the one you chose during installation, and the ip you can find via: https://www.cyberciti.biz/faq/find-ip-address-of-linux-kvm-guest-virtual-machine/

  1. Change the hostname of the VMs to 'cloud', 'edge1' and 'endpoint1': Either use sudo hostnamectl set-hostname 'name' or https://www.cyberciti.biz/faq/ubuntu-change-hostname-command/
  2. Check if the host can see both VMs and the VMs can see each other and the host by using the 'ping' command.

Setting up Kubernetes

Source: https://www.linuxtechi.com/install-kubernetes-k8s-on-ubuntu-20-04/
Do the following steps on the cloud and edge VM:

  1. Disable swap. First check if swap is enabled: swapon -s. If there is no output, swap is disabled. Otherwise, edit /etc/fstab by commenting out the line with 'swap' in it.
  2. Enable ipv4 forwarding: Uncomment the line 'net.ipv4.ip_forward=1' in /etc/sysctl.conf

Do the following step on the cloud, edge and endpoint VM:

  1. Install docker: sudo apt update and sudo apt install -y docker.io, then start docker sudo systemctl enable docker.service --now and check if it is correctly started systemctl status docker. If docker doesn't work for some reason, try to install version 19.03.

Do the following steps on the cloud VM only:

  1. Install kubernetes (version 1.19.0 for KubeEdge 1.6.1): sudo apt install -y apt-transport-https curl and curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add and sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main" and sudo apt update and finally sudo apt install -y kubelet=1.19.0-00 kubeadm=1.19.0-00 kubectl=1.19.0-00
  2. Initialize kubernetes: sudo kubeadm init (note: kubernetes now uses cgroup as the docker driver, not the recommended systemd)
  3. Start using the cluster: mkdir -p $HOME/.kube and sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config and sudo chown $(id -u):$(id -g) $HOME/.kube/config

Setting up KubeEdge

Source: https://kubeedge.io/en/docs/ and https://kubeedge.io/en/docs/setup/keadm/
Do the following step on the cloud and edge VM:

  1. Get keadm: wget https://github.com/kubeedge/kubeedge/releases/download/v1.6.1/keadm-v1.6.1-linux-amd64.tar.gz. Extract it: tar -xvf keadm-v1.6.1-linux-amd64.tar.gz. Make it an executable: cd keadm-v1.6.1-linux-amd64/keadm && chmod +x keadm && sudo mv keadm /usr/local/bin

Do the following steps on the cloud VM only:

  1. Create a symbolic link so keadm can find kubernetes: sudo ln -s ~/.kube /root
  2. Start keadm: sudo keadm init --advertise-address="cloud-ip-here" --kubeedge-version=1.6.1
  3. Get the token which is used to join the edge VM: sudo keadm gettoken. Don't forget this token.

Do the following steps on the edge VM only:

  1. Join the cloudcore and install mandatory software (such as mosquito): sudo keadm join --cloudcore-ipport=cloud_ip_here:10000 --token=token_from_previous_step

Do the following step on the cloud VM only:

  1. Check if the setup has succeeded: kubectl get nodes -o wide. The cloud VM should have status 'NotReady', while the edge VM should have status 'Ready' and as version kubeedge. kubectl describe nodes edge1 should show the resources of the edge VM that are available to the cloudcore.

Enable the Metrics-server

TODO: see https://github.com/kubeedge/kubeedge/blob/master/docs/setup/keadm.md

Stop the cluster

Do the following step on both the cloud and edge VM:

  1. Stop KubeEdge: sudo keadm reset. To start KubeEdge next time, repeat step 3, 4 and 5 from the "Setting up KubeEdge" section.

Do the following step on the cloud VM only:

  1. Delete the edge node from Kubernetes: kubectl delete node edge1
  2. (Optional) Stop Kubernetes: sudo kubeadm reset and rm -rf ~/.kube. To start Kubernetes next time, repeat step 5 and 6 from the "Setting up Kubernetes" section.

Run example application 1: Nginx webserver

Source: https://kubeedge.io/en/docs/advanced/edgemesh/
This example application will run an nginx webserver.
Do the following steps on the cloud VM:

  1. Deploy the webserver: kubectl apply -f https://raw.githubusercontent.com/kubeedge/kubeedge/master/build/deployment.yaml
  2. Check if the pod is running on the edge: kubectl get pods -o wide

Do the following step on the edge VM:

  1. Check if the webserver works: curl 172.17.0.2. You should get a welcome message

Do the following steps on the cloud VM:

  1. Create a service for it:
$ cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
  name: nginx-svc
  namespace: default
spec:
  clusterIP: None
  selector:
    app: nginx
  ports:
    - name: http-0
      port: 12345
      protocol: TCP
      targetPort: 80
EOF
  1. Check if the service is up kubectl get service and the endpoint is up kubectl get endpoints

Do the following step on the edge VM:

  1. Check if you can access the webserver via the service: curl http://nginx-svc.default.svc.cluster.local:12345

Do the following steps on the cloud VM:

  1. Stop the service: kubectl delete service nginx-svc
  2. Stop the deployment / pod: kubectl delete -f https://raw.githubusercontent.com/kubeedge/kubeedge/master/build/deployment.yaml

Run example application 2: Image classification

This example application will send images from an endpoint device to an edge device one by one over MQTT. The edge device runs TFLite to classify the incoming images.
Do the following step on the endpoint VM:

  1. Pull the publisher image: sudo docker pull redplanet00/kubeedge-tflite-mqtt:publisher

Do the following steps on the cloud VM:

  1. Deploy the subscriber application in the cloud, which will run it on the edge:
$ cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: img-class-sub-deployment
  labels:
    app: img-class-sub
spec:
  replicas: 1
  selector:
    matchLabels:
      app: img-class-sub
  template:
    metadata:
      labels:
        app: img-class-sub
    spec:
      containers:
      - name: img-class-sub
        image: redplanet00/kubeedge-tflite-mqtt:subscriber
        ports:
        - containerPort: 1883
EOF
  1. Check if the pod is running on the edge: kubectl get pods -o wide

Do the following step on the endpoint VM:

  1. Run the endpoint application which will send data to the edge over MQTT: sudo docker container run redplanet00/kubeedge-tflite-mqtt:publisher

Do the following steps on the cloud VM:

  1. Stop the deployment / pod: kubectl delete -f redplanet00/kubeedge-tflite-mqtt:subscriber

Common Kubernetes / KubeEdge commands

For the cloud VM only:

  • Get deployment plans / pods: kubectl get pods -o wide
  • Get all kubernetes connected nodes: kubectl get nodes -o wide
  • Delete pod / deployment: kubectl delete -f <path-to-config-yaml>
  • Delete node: kubectl delete node <nodename>
  • Get pod details: kubectl describe pods <podname>
  • Get logs: less /var/log/kubeedge/cloudcore.log

For the edge VM only:

  • Get logs: journalctl -u edgecore.service -b --since "2021-04-30 12:45"
  • Clear old logs: journalctl --vacuum-time=2d --vacuum-size=500M

Extra: How to use docker

Images

  1. List docker images sudo docker image ls
  2. Remove (referenced) docker image sudo docker image rm (-f) <image_id>

Containers

  1. List all (running and stopped) containers sudo docker container ls -a
  2. Remove a stopped (/ running) docker container sudo docker container rm (-f) <container_id>
  3. Remove all running and stopped containers sudo docker container rm -f $(docker ps -aq)

Building and running

  1. Build image in current directory sudo docker build -t myimage:1.0 .
  2. Retag image sudo docker tag this_image:1.0 that_image:2.0
  3. Run a container named 'x' based on an image myimage:1.0 sudo docker container run --name x myimage:1.0
  4. Stop running container sudo docker container stop x

Saving an image

  1. Save and zip sudo docker save myimage:version | gzip > myimage_latest.tar.gz
  2. Load saved image sudo docker load -i ./myimage_latest

Upload an image

  1. Login to docker (if it doesn't work, first do 'docker logout') docker login
  2. Push image to repository docker push user/repository:tagname
Clone this wiki locally