Update Multus version to v4.0.2

[manuelbuil]

Is your feature request related to a problem? Please describe.

Update Multus release to fix several CVEs

Describe the solution you'd like

New version of multus image and chart

Describe alternatives you've considered

Additional context

[VestigeJ]

This issue will be morphing out from underneath me - where the current release for v1.27.3 RKE2 with Multus will be this new version. So I'm comparing against the v1.27.2 release in this test case which used the same version as the v1.27.3 release

##Environment Details
VERSION=v1.27.2+rke2r1
COMMIT=356adec505d82c23b9c962bc3aff3beac276d8f3

Infrastructure

• Cloud

Node(s) CPU architecture, OS, and version:

Linux 5.14.21-150400.24.11-default x86_64 GNU/Linux 
PRETTY_NAME="SUSE Linux Enterprise Server 15 SP4"

Cluster Configuration:

NAME               STATUS   ROLES                       AGE     VERSION
ip-1-1-1-183       Ready    control-plane,etcd,master   3m49s   v1.27.2+rke2r1 

Config.yaml:

write-kubeconfig-mode: 644
debug: true
token: calciumcarbonatemicrospheres
cni: multus,calico
profile: cis-1.23
selinux: true

Existing Multus Version

$ curl https://get.rke2.io --output install-"rke2".sh
$ sudo chmod +x install-"rke2".sh
$ sudo groupadd --system etcd && sudo useradd -s /sbin/nologin --system -g etcd etcd
$ sudo modprobe ip_vs_rr
$ sudo modprobe ip_vs_wrr
$ sudo modprobe ip_vs_sh
$ sudo printf "on_oovm.panic_on_oom=0 \nvm.overcommit_memory=1 \nkernel.panic=10 \nkernel.panic_ps=1 \nkernel.panic_on_oops=1 \n" > ~/60-rke2-cis.conf or ~/90-kubelet.conf
$ sudo cp 60-rke2-cis.conf /etc/sysctl.d/ or 90-kubelet.conf
$ sudo systemctl restart systemd-sysctl
$ sudo INSTALL_RKE2_VERSION=v1.27.2+rke2r1 INSTALL_RKE2_EXEC=server ./install-rke2.sh 
$ go_rke2 //sudo systemctl enable rke2-server --now
$ set_kubefig //KUBECONFIG=/etc/rancher/rke2/rke2.yaml
$ kga //kubectl get all -A
$ kd -n kube-system pod/rke2-multus-ds-fldjq //kubectl describe...
$ get_report

Results:

$ kd -n kube-system pod/rke2-multus-ds-fldjq

containers:
  kube-rke2-multus:
    Container ID:  containerd://7667ffbb0fab3c3e3e0bea6dd48e325e92b4bab8c52526b4bd56abac3f2b0b1b
    Image:         rancher/hardened-multus-cni:v3.9.3-build20230109
    Image ID:      docker.io/rancher/hardened-multus-cni@sha256:02002dc4ff9aa0ec9887f21cd3a5d6bd30aa1e71b21e1061267ed7a77b9fad9a

Novel Multus Version

$ curl https://get.rke2.io --output install-"rke2".sh
$ sudo chmod +x install-"rke2".sh
$ sudo groupadd --system etcd && sudo useradd -s /sbin/nologin --system -g etcd etcd
$ sudo modprobe ip_vs_rr
$ sudo modprobe ip_vs_wrr
$ sudo modprobe ip_vs_sh
$ sudo printf "on_oovm.panic_on_oom=0 \nvm.overcommit_memory=1 \nkernel.panic=10 \nkernel.panic_ps=1 \nkernel.panic_on_oops=1 \n" > ~/60-rke2-cis.conf or ~/90-kubelet.conf
$ sudo cp 60-rke2-cis.conf /etc/sysctl.d/ or 90-kubelet.conf
$ sudo systemctl restart systemd-sysctl
$ sudo INSTALL_RKE2_COMMIT=356adec505d82c23b9c962bc3aff3beac276d8f3 INSTALL_RKE2_EXEC=server ./install-rke2.sh
$ go_rke2 //sudo systemctl enable rke2-server --now
$ set_kubefig //KUBECONFIG=/etc/rancher/rke2/rke2.yaml
$ kga //kubectl get all -A
$ kd -n kube-system pod/rke2-multus-ds-jtnmp //kubectl describe...
$ get_report //generate this template

Results:

$ kd -n kube-system pod/rke2-multus-ds-jtnmp

Containers:
  kube-rke2-multus:
    Container ID:  containerd://53664f40345f2e424d2ec6e192552a253ab83b8252c1fc76896a1fcd0a5d3c35
    Image:         rancher/hardened-multus-cni:v4.0.2-build20230707
    Image ID:      docker.io/rancher/hardened-multus-cni@sha256:5521b36d9e6588403902a827f74fe28b7763783c6340546ce36d4eba0e8424a0

Additional context / logs: