kubectl karmada init get error mkdir /etc/karmada: permission denied #6080

olariuromeo · 2025-01-23T22:52:24Z

What happened:

./install_karmada_kind.sh
golang 1.23.5 is already installed
Step: Installing kubectl...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   138  100   138    0     0    388      0 --:--:-- --:--:-- --:--:--   388
100 47.4M  100 47.4M    0     0  3207k      0  0:00:15  0:00:15 --:--:-- 4016k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   138  100   138    0     0    466      0 --:--:-- --:--:-- --:--:--   467
100    64  100    64    0     0    135      0 --:--:-- --:--:-- --:--:--   135
kubectl: OK
Client Version: v1.29.0
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Step: Installing Kind...
Step: Adding GOPATH/bin to PATH...
Step: Verifying Kind installation...
Kind installed successfully!
Step: Installing Krew...
Adding "default" plugin index from https://github.com/kubernetes-sigs/krew-index.git.
Updated the local copy of plugin index.
Installing plugin: krew
Installed plugin: krew
\
 | Use this plugin:
 | 	kubectl krew
 | Documentation:
 | 	https://krew.sigs.k8s.io/
 | Caveats:
 | \
 |  | krew is now installed! To start using kubectl plugins, you need to add
 |  | krew's installation directory to your PATH:
 |  | 
 |  |   * macOS/Linux:
 |  |     - Add the following to your ~/.bashrc or ~/.zshrc:
 |  |         export PATH="${KREW_ROOT:-$HOME/.krew}/bin:$PATH"
 |  |     - Restart your shell.
 |  | 
 |  |   * Windows: Add %USERPROFILE%\.krew\bin to your PATH environment variable
 |  | 
 |  | To list krew commands and to get help, run:
 |  |   $ kubectl krew
 |  | For a full list of available plugins, run:
 |  |   $ kubectl krew search
 |  | 
 |  | You can find documentation at
 |  |   https://krew.sigs.k8s.io/docs/user-guide/quickstart/.
 | /
/
Step: Adding Krew to PATH...
Step: Verifying Krew installation...
OPTION            VALUE
GitTag            v0.4.4
GitCommit         343e657
IndexURI          https://github.com/kubernetes-sigs/krew-index.git
BasePath          /home/user/.krew
IndexPath         /home/user/.krew/index/default
InstallPath       /home/user/.krew/store
BinPath           /home/user/.krew/bin
DetectedPlatform  linux/amd64
Step: Creating Kind cluster named 'host'...
Creating cluster "host" ...
Context "kind-host" renamed to "host".
Cluster "kind-host" set.
cluster "host" is created successfully!
You can now use your cluster with:
kubectl cluster-info --context host --kubeconfig /home/user/.kube/host.config
Step: Installing Karmada plugin via Krew...
Updated the local copy of plugin index.
Installing plugin: karmada
Installed plugin: karmada
\
 | Use this plugin:
 | 	kubectl karmada
 | Documentation:
 | 	https://github.com/karmada-io/karmada
/
WARNING: You installed plugin "karmada" from the krew-index plugin repository.
   These plugins are not audited for security by the Krew maintainers.
   Run them at your own risk.
Step: Verifying Karmada plugin installation...
Karmada plugin installed successfully!
Step: Initializing Karmada...
I0123 22:36:35.320413 1102040 deploy.go:266] kubeconfig file: /home/user/.kube/host.config, kubernetes: https://172.20.0.2:6443
I0123 22:36:35.378838 1102040 deploy.go:286] karmada apiserver ip: [172.20.0.2]
error: failed to create directory: /etc/karmada, error: mkdir /etc/karmada: permission denied

What you expected to happen:

init karmada cluster The official documentation states that it is possible to execute it without sudo. However, I’m currently trying to start Karmada locally without using sudo, but it refuses to do so. I’m confused as to why it attempts to access the /etc/ directory when the command clearly indicates that it shouldn’t require sudo permissions: https://karmada.io/docs/installation/#install-karmada-in-kind-cluster kubectl works perfectly without sudo I don't understand why karmada would need it, it should just make a virtual cluster in kind

How to reproduce it (as minimally and precisely as possible):

#!/bin/bash

set -e

# Define the absolute path to the install directory
INSTALL_DIR="$(dirname "$(realpath "$0")")"

# Install tool
asdf install golang 1.23.5
asdf local golang 1.23.5

# Step 1: Install kubectl
echo "Step: Installing kubectl..."

# Download kubectl binary
curl -LO https://dl.k8s.io/release/v1.29.0/bin/linux/amd64/kubectl

# Download sha256 checksum file for kubectl
curl -LO "https://dl.k8s.io/release/v1.29.0/bin/linux/amd64/kubectl.sha256"

# Verify the checksum
echo "$(cat kubectl.sha256)  kubectl" | sha256sum --check

# Make kubectl executable
chmod +x kubectl

# Create directory for user-specific binaries if it doesn't exist
mkdir -p ~/.local/bin

# Move kubectl to ~/.local/bin
mv kubectl ~/.local/bin/kubectl

# Add ~/.local/bin to PATH if it's not already included
if ! grep -q "$HOME/.local/bin" ~/.bashrc; then
    echo 'export PATH=$HOME/.local/bin:$PATH' >> ~/.bashrc
fi

# Reload bashrc to apply changes (this applies only for future shell sessions)
source ~/.bashrc

# Directly check if kubectl is now available in PATH
if ! command -v kubectl &>/dev/null; then
    echo "kubectl is not available in PATH, trying again with direct path"
    export PATH="$HOME/.local/bin:$PATH"
fi

# Verify kubectl installation
kubectl version --client


# Step Install Kind
echo "Step: Installing Kind..."
go install sigs.k8s.io/[email protected]

# Step Add GOPATH/bin to PATH
echo "Step: Adding GOPATH/bin to PATH..."
export PATH=$PATH:$(go env GOPATH)/bin

# Verify Kind installation
echo "Step: Verifying Kind installation..."
if ! command -v kind &>/dev/null; then
    echo "Kind is not installed correctly! Please check the installation."
    exit 1
fi
echo "Kind installed successfully!"

# Step 6: Install Krew
# Check system architecture
echo "Step: Installing Krew..."
OS=$(uname | tr '[:upper:]' '[:lower:]')
ARCH=$(uname -m)
[ "$ARCH" == "x86_64" ] && ARCH="amd64" || true

# Define Krew name based on the system architecture
KREW="krew-${OS}_${ARCH}"

# Download and install Krew
curl -fsSL "https://github.com/kubernetes-sigs/krew/releases/latest/download/${KREW}.tar.gz" | tar zx
./${KREW} install krew

# Step 7: Add Krew to PATH
echo "Step: Adding Krew to PATH..."
if ! grep -q 'export PATH="${KREW_ROOT:-$HOME/.krew}/bin:$PATH"' ~/.bashrc; then
    echo 'export PATH="${KREW_ROOT:-$HOME/.krew}/bin:$PATH"' >> ~/.bashrc
fi
export PATH="${KREW_ROOT:-$HOME/.krew}/bin:$PATH"
source ~/.bashrc

# Step 8: Verify Krew installation
echo "Step: Verifying Krew installation..."
kubectl krew version

# Step 9: Create Kind cluster
echo "Step: Creating Kind cluster named 'host'..."
./hack/create-cluster.sh host $HOME/.kube/host.config

# Step 10: Install Karmada plugin via Krew
echo "Step: Installing Karmada plugin via Krew..."
kubectl krew install karmada

# Step 11: Verify Karmada plugin installation
echo "Step: Verifying Karmada plugin installation..."
if ! command -v kubectl-karmada &>/dev/null; then
    echo "Karmada plugin is not installed correctly!"
    exit 1
fi
echo "Karmada plugin installed successfully!"

# Step 12: Initialize Karmada
echo "Step: Initializing Karmada..."

kubectl karmada init --crds https://github.com/karmada-io/karmada/releases/download/v1.2.0/crds.tar.gz --kubeconfig=$HOME/.kube/host.config

echo "Karmada plugin installed and initialized successfully!"

Anything else we need to know?:

Environment:

ubuntu 22.04

Karmada version:
kubectl-karmada or karmadactl version (the result of kubectl-karmada version or karmadactl version):
Others:

The text was updated successfully, but these errors were encountered:

olariuromeo · 2025-01-24T00:33:04Z

Instead ./hack/local-up-karmada.sh starts the installation but never finishes. Even though I have 64 GB of RAM available and the CPU doesn't exceed 5%

./hack/local-up-karmada.sh
Preparing: 'kind' existence check - passed
Preparing: 'kubectl' existence check - passed
Preparing kindClusterConfig in path: /tmp/tmp.4fzQRjdozW
Deleted all necessary clusters and the log file is in /tmp/karmada/delete-necessary-resources.log
Creating cluster karmada-host and the log file is in /tmp/karmada/karmada-host.log
Creating cluster member1 and the log file is in /tmp/karmada/member1.log
Creating cluster member2 and the log file is in /tmp/karmada/member2.log
Creating cluster member3 and the log file is in /tmp/karmada/member3.log
make: Entering directory '/home/user/data/dev/apps/karmada'
set -e;\
target=$(echo karmada-aggregated-apiserver);\
make $target GOOS=linux;\
VERSION=latest REGISTRY=docker.io/karmada BUILD_PLATFORMS=linux/amd64 hack/docker.sh $target
make[1]: Entering directory '/home/user/data/dev/apps/karmada'
BUILD_PLATFORMS=linux/amd64 hack/build.sh karmada-aggregated-apiserver
!!! Building karmada-aggregated-apiserver for linux/amd64:
+ CGO_ENABLED=0
+ GOOS=linux
+ GOARCH=amd64
+ go build -ldflags '-X github.com/karmada-io/karmada/pkg/version.gitVersion=v1.13.0-alpha.2-15-g07747ed3f-dirty -X github.com/karmada-io/karmada/pkg/version.gitCommit=07747ed3f7a7c1a9e116156e8c751cca3b6e5d14 -X github.com/karmada-io/karmada/pkg/version.gitTreeState=dirty -X github.com/karmada-io/karmada/pkg/version.buildDate=2025-01-24T00:01:20Z ' -o _output/bin/linux/amd64/karmada-aggregated-apiserver github.com/karmada-io/karmada/cmd/aggregated-apiserver
+ set +x
make[1]: Leaving directory '/home/user/data/dev/apps/karmada'
Building image for linux/amd64: docker.io/karmada/karmada-aggregated-apiserver:latest
+ docker build --build-arg BINARY=karmada-aggregated-apiserver --label=image.karmada.io=May_be_pruned_in_local_up_environment --tag docker.io/karmada/karmada-aggregated-apiserver:latest --file hack/../cluster/images/Dockerfile hack/../_output/bin/linux/amd64
[+] Building 9.1s (10/10) FINISHED                                                                                                                       docker:default
 => [internal] load build definition from Dockerfile                                                                                                               0.0s
 => => transferring dockerfile: 835B                                                                                                                               0.0s
 => [internal] load metadata for docker.io/library/alpine:3.21.2                                                                                                   1.6s
 => [auth] library/alpine:pull token for registry-1.docker.io                                                                                                      0.0s
 => [internal] load .dockerignore                                                                                                                                  0.1s
 => => transferring context: 2B                                                                                                                                    0.0s
 => [1/4] FROM docker.io/library/alpine:3.21.2@sha256:56fa17d2a7e7f168a043a2712e63aed1f8543aeafdcee47c58dcffe38ed51099                                             0.0s
 => [internal] load build context                                                                                                                                  0.6s
 => => transferring context: 98.20MB                                                                                                                               0.5s
 => CACHED [2/4] RUN apk add --no-cache ca-certificates                                                                                                            0.0s
 => CACHED [3/4] RUN apk add --no-cache tzdata                                                                                                                     0.0s
 => [4/4] COPY karmada-aggregated-apiserver /bin/karmada-aggregated-apiserver                                                                                      5.4s
 => exporting to image                                                                                                                                             0.7s
 => => exporting layers                                                                                                                                            0.6s
 => => writing image sha256:ec4c7ffdf0175adf1d1696b689a34e104f400028d3128ab3155302fcea0d48b5                                                                       0.0s
 => => naming to docker.io/karmada/karmada-aggregated-apiserver:latest                                                                                             0.0s
+ set +x
set -e;\
target=$(echo karmada-controller-manager);\
make $target GOOS=linux;\
VERSION=latest REGISTRY=docker.io/karmada BUILD_PLATFORMS=linux/amd64 hack/docker.sh $target
make[1]: Entering directory '/home/user/data/dev/apps/karmada'
BUILD_PLATFORMS=linux/amd64 hack/build.sh karmada-controller-manager
!!! Building karmada-controller-manager for linux/amd64:
+ CGO_ENABLED=0
+ GOOS=linux
+ GOARCH=amd64
+ go build -ldflags '-X github.com/karmada-io/karmada/pkg/version.gitVersion=v1.13.0-alpha.2-15-g07747ed3f-dirty -X github.com/karmada-io/karmada/pkg/version.gitCommit=07747ed3f7a7c1a9e116156e8c751cca3b6e5d14 -X github.com/karmada-io/karmada/pkg/version.gitTreeState=dirty -X github.com/karmada-io/karmada/pkg/version.buildDate=2025-01-24T00:01:54Z ' -o _output/bin/linux/amd64/karmada-controller-manager github.com/karmada-io/karmada/cmd/controller-manager
+ set +x
make[1]: Leaving directory '/home/user/data/dev/apps/karmada'
Building image for linux/amd64: docker.io/karmada/karmada-controller-manager:latest
+ docker build --build-arg BINARY=karmada-controller-manager --label=image.karmada.io=May_be_pruned_in_local_up_environment --tag docker.io/karmada/karmada-controller-manager:latest --file hack/../cluster/images/Dockerfile hack/../_output/bin/linux/amd64
[+] Building 4.4s (9/9) FINISHED                                                                                                                         docker:default
 => [internal] load build definition from Dockerfile                                                                                                               0.0s
 => => transferring dockerfile: 835B                                                                                                                               0.0s
 => [internal] load metadata for docker.io/library/alpine:3.21.2                                                                                                   0.9s
 => [internal] load .dockerignore                                                                                                                                  0.2s
 => => transferring context: 2B                                                                                                                                    0.0s
 => [internal] load build context                                                                                                                                  0.6s
 => => transferring context: 95.51MB                                                                                                                               0.5s
 => [1/4] FROM docker.io/library/alpine:3.21.2@sha256:56fa17d2a7e7f168a043a2712e63aed1f8543aeafdcee47c58dcffe38ed51099                                             0.0s
 => CACHED [2/4] RUN apk add --no-cache ca-certificates                                                                                                            0.0s
 => CACHED [3/4] RUN apk add --no-cache tzdata                                                                                                                     0.0s
 => [4/4] COPY karmada-controller-manager /bin/karmada-controller-manager                                                                                          1.1s
 => exporting to image                                                                                                                                             1.2s
 => => exporting layers                                                                                                                                            1.1s
 => => writing image sha256:3252043c11793a9801411d50de4d650dee80d15ef26f5ed5c6bcebb407afba37                                                                       0.0s
 => => naming to docker.io/karmada/karmada-controller-manager:latest                                                                                               0.0s
+ set +x
set -e;\
target=$(echo karmada-scheduler);\
make $target GOOS=linux;\
VERSION=latest REGISTRY=docker.io/karmada BUILD_PLATFORMS=linux/amd64 hack/docker.sh $target
make[1]: Entering directory '/home/user/data/dev/apps/karmada'
BUILD_PLATFORMS=linux/amd64 hack/build.sh karmada-scheduler
!!! Building karmada-scheduler for linux/amd64:
+ CGO_ENABLED=0
+ GOOS=linux
+ GOARCH=amd64
+ go build -ldflags '-X github.com/karmada-io/karmada/pkg/version.gitVersion=v1.13.0-alpha.2-15-g07747ed3f-dirty -X github.com/karmada-io/karmada/pkg/version.gitCommit=07747ed3f7a7c1a9e116156e8c751cca3b6e5d14 -X github.com/karmada-io/karmada/pkg/version.gitTreeState=dirty -X github.com/karmada-io/karmada/pkg/version.buildDate=2025-01-24T00:02:06Z ' -o _output/bin/linux/amd64/karmada-scheduler github.com/karmada-io/karmada/cmd/scheduler
+ set +x
make[1]: Leaving directory '/home/user/data/dev/apps/karmada'
Building image for linux/amd64: docker.io/karmada/karmada-scheduler:latest
+ docker build --build-arg BINARY=karmada-scheduler --label=image.karmada.io=May_be_pruned_in_local_up_environment --tag docker.io/karmada/karmada-scheduler:latest --file hack/../cluster/images/Dockerfile hack/../_output/bin/linux/amd64
[+] Building 2.2s (9/9) FINISHED                                                                                                                         docker:default
 => [internal] load build definition from Dockerfile                                                                                                               0.0s
 => => transferring dockerfile: 835B                                                                                                                               0.0s
 => [internal] load metadata for docker.io/library/alpine:3.21.2                                                                                                   0.5s
 => [internal] load .dockerignore                                                                                                                                  0.0s
 => => transferring context: 2B                                                                                                                                    0.0s
 => [1/4] FROM docker.io/library/alpine:3.21.2@sha256:56fa17d2a7e7f168a043a2712e63aed1f8543aeafdcee47c58dcffe38ed51099                                             0.0s
 => [internal] load build context                                                                                                                                  0.4s
 => => transferring context: 74.43MB                                                                                                                               0.3s
 => CACHED [2/4] RUN apk add --no-cache ca-certificates                                                                                                            0.0s
 => CACHED [3/4] RUN apk add --no-cache tzdata                                                                                                                     0.0s
 => [4/4] COPY karmada-scheduler /bin/karmada-scheduler                                                                                                            0.7s
 => exporting to image                                                                                                                                             0.3s
 => => exporting layers                                                                                                                                            0.3s
 => => writing image sha256:ec87e54ae1e0809255493b22fb02d9f0995319efce4cf1c9e901fd2985bb418a                                                                       0.0s
 => => naming to docker.io/karmada/karmada-scheduler:latest                                                                                                        0.0s
+ set +x
set -e;\
target=$(echo karmada-descheduler);\
make $target GOOS=linux;\
VERSION=latest REGISTRY=docker.io/karmada BUILD_PLATFORMS=linux/amd64 hack/docker.sh $target
make[1]: Entering directory '/home/user/data/dev/apps/karmada'
BUILD_PLATFORMS=linux/amd64 hack/build.sh karmada-descheduler
!!! Building karmada-descheduler for linux/amd64:
+ CGO_ENABLED=0
+ GOOS=linux
+ GOARCH=amd64
+ go build -ldflags '-X github.com/karmada-io/karmada/pkg/version.gitVersion=v1.13.0-alpha.2-15-g07747ed3f-dirty -X github.com/karmada-io/karmada/pkg/version.gitCommit=07747ed3f7a7c1a9e116156e8c751cca3b6e5d14 -X github.com/karmada-io/karmada/pkg/version.gitTreeState=dirty -X github.com/karmada-io/karmada/pkg/version.buildDate=2025-01-24T00:02:13Z ' -o _output/bin/linux/amd64/karmada-descheduler github.com/karmada-io/karmada/cmd/descheduler
+ set +x
make[1]: Leaving directory '/home/user/data/dev/apps/karmada'
Building image for linux/amd64: docker.io/karmada/karmada-descheduler:latest
+ docker build --build-arg BINARY=karmada-descheduler --label=image.karmada.io=May_be_pruned_in_local_up_environment --tag docker.io/karmada/karmada-descheduler:latest --file hack/../cluster/images/Dockerfile hack/../_output/bin/linux/amd64
[+] Building 3.4s (9/9) FINISHED                                                                                                                         docker:default
 => [internal] load build definition from Dockerfile                                                                                                               0.0s
 => => transferring dockerfile: 835B                                                                                                                               0.0s
 => [internal] load metadata for docker.io/library/alpine:3.21.2                                                                                                   0.3s
 => [internal] load .dockerignore                                                                                                                                  0.0s
 => => transferring context: 2B                                                                                                                                    0.0s
 => [1/4] FROM docker.io/library/alpine:3.21.2@sha256:56fa17d2a7e7f168a043a2712e63aed1f8543aeafdcee47c58dcffe38ed51099                                             0.0s
 => [internal] load build context                                                                                                                                  0.4s
 => => transferring context: 73.69MB                                                                                                                               0.4s
 => CACHED [2/4] RUN apk add --no-cache ca-certificates                                                                                                            0.0s
 => CACHED [3/4] RUN apk add --no-cache tzdata                                                                                                                     0.0s
 => [4/4] COPY karmada-descheduler /bin/karmada-descheduler                                                                                                        1.4s
 => exporting to image                                                                                                                                             0.5s
 => => exporting layers                                                                                                                                            0.4s
 => => writing image sha256:50496a8966bf5489624abd47b2aa2392f4094946ca1832c346d779cf84704c65                                                                       0.0s
 => => naming to docker.io/karmada/karmada-descheduler:latest                                                                                                      0.0s
+ set +x
set -e;\
target=$(echo karmada-webhook);\
make $target GOOS=linux;\
VERSION=latest REGISTRY=docker.io/karmada BUILD_PLATFORMS=linux/amd64 hack/docker.sh $target
make[1]: Entering directory '/home/user/data/dev/apps/karmada'
BUILD_PLATFORMS=linux/amd64 hack/build.sh karmada-webhook
!!! Building karmada-webhook for linux/amd64:
+ CGO_ENABLED=0
+ GOOS=linux
+ GOARCH=amd64
+ go build -ldflags '-X github.com/karmada-io/karmada/pkg/version.gitVersion=v1.13.0-alpha.2-15-g07747ed3f-dirty -X github.com/karmada-io/karmada/pkg/version.gitCommit=07747ed3f7a7c1a9e116156e8c751cca3b6e5d14 -X github.com/karmada-io/karmada/pkg/version.gitTreeState=dirty -X github.com/karmada-io/karmada/pkg/version.buildDate=2025-01-24T00:02:20Z ' -o _output/bin/linux/amd64/karmada-webhook github.com/karmada-io/karmada/cmd/webhook
+ set +x
make[1]: Leaving directory '/home/user/data/dev/apps/karmada'
Building image for linux/amd64: docker.io/karmada/karmada-webhook:latest
+ docker build --build-arg BINARY=karmada-webhook --label=image.karmada.io=May_be_pruned_in_local_up_environment --tag docker.io/karmada/karmada-webhook:latest --file hack/../cluster/images/Dockerfile hack/../_output/bin/linux/amd64
[+] Building 3.5s (9/9) FINISHED                                                                                                                         docker:default
 => [internal] load build definition from Dockerfile                                                                                                               0.1s
 => => transferring dockerfile: 835B                                                                                                                               0.0s
 => [internal] load metadata for docker.io/library/alpine:3.21.2                                                                                                   0.6s
 => [internal] load .dockerignore                                                                                                                                  0.1s
 => => transferring context: 2B                                                                                                                                    0.0s
 => [1/4] FROM docker.io/library/alpine:3.21.2@sha256:56fa17d2a7e7f168a043a2712e63aed1f8543aeafdcee47c58dcffe38ed51099                                             0.0s
 => [internal] load build context                                                                                                                                  0.8s
 => => transferring context: 71.10MB                                                                                                                               0.4s
 => CACHED [2/4] RUN apk add --no-cache ca-certificates                                                                                                            0.0s
 => CACHED [3/4] RUN apk add --no-cache tzdata                                                                                                                     0.0s
 => [4/4] COPY karmada-webhook /bin/karmada-webhook                                                                                                                0.9s
 => exporting to image                                                                                                                                             0.5s
 => => exporting layers                                                                                                                                            0.4s
 => => writing image sha256:175f74eb61f0b74d50eeda0c45e11f554b7756ad6fa75098fc95b50e94330256                                                                       0.0s
 => => naming to docker.io/karmada/karmada-webhook:latest                                                                                                          0.0s
+ set +x
set -e;\
target=$(echo karmada-agent);\
make $target GOOS=linux;\
VERSION=latest REGISTRY=docker.io/karmada BUILD_PLATFORMS=linux/amd64 hack/docker.sh $target
make[1]: Entering directory '/home/user/data/dev/apps/karmada'
BUILD_PLATFORMS=linux/amd64 hack/build.sh karmada-agent
!!! Building karmada-agent for linux/amd64:
+ CGO_ENABLED=0
+ GOOS=linux
+ GOARCH=amd64
+ go build -ldflags '-X github.com/karmada-io/karmada/pkg/version.gitVersion=v1.13.0-alpha.2-15-g07747ed3f-dirty -X github.com/karmada-io/karmada/pkg/version.gitCommit=07747ed3f7a7c1a9e116156e8c751cca3b6e5d14 -X github.com/karmada-io/karmada/pkg/version.gitTreeState=dirty -X github.com/karmada-io/karmada/pkg/version.buildDate=2025-01-24T00:02:29Z ' -o _output/bin/linux/amd64/karmada-agent github.com/karmada-io/karmada/cmd/agent
+ set +x
make[1]: Leaving directory '/home/user/data/dev/apps/karmada'
Building image for linux/amd64: docker.io/karmada/karmada-agent:latest
+ docker build --build-arg BINARY=karmada-agent --label=image.karmada.io=May_be_pruned_in_local_up_environment --tag docker.io/karmada/karmada-agent:latest --file hack/../cluster/images/Dockerfile hack/../_output/bin/linux/amd64
[+] Building 2.3s (9/9) FINISHED                                                                                                                         docker:default
 => [internal] load build definition from Dockerfile                                                                                                               0.0s
 => => transferring dockerfile: 835B                                                                                                                               0.0s
 => [internal] load metadata for docker.io/library/alpine:3.21.2                                                                                                   0.5s
 => [internal] load .dockerignore                                                                                                                                  0.0s
 => => transferring context: 2B                                                                                                                                    0.0s
 => [1/4] FROM docker.io/library/alpine:3.21.2@sha256:56fa17d2a7e7f168a043a2712e63aed1f8543aeafdcee47c58dcffe38ed51099                                             0.0s
 => [internal] load build context                                                                                                                                  0.4s
 => => transferring context: 84.72MB                                                                                                                               0.4s
 => CACHED [2/4] RUN apk add --no-cache ca-certificates                                                                                                            0.0s
 => CACHED [3/4] RUN apk add --no-cache tzdata                                                                                                                     0.0s
 => [4/4] COPY karmada-agent /bin/karmada-agent                                                                                                                    0.7s
 => exporting to image                                                                                                                                             0.4s
 => => exporting layers                                                                                                                                            0.3s
 => => writing image sha256:50cee6b2892054c7858abc62a11cd3c2317775eeb75c3b4b1d785bdbc2a19f91                                                                       0.0s
 => => naming to docker.io/karmada/karmada-agent:latest                                                                                                            0.0s
+ set +x
set -e;\
target=$(echo karmada-scheduler-estimator);\
make $target GOOS=linux;\
VERSION=latest REGISTRY=docker.io/karmada BUILD_PLATFORMS=linux/amd64 hack/docker.sh $target
make[1]: Entering directory '/home/user/data/dev/apps/karmada'
BUILD_PLATFORMS=linux/amd64 hack/build.sh karmada-scheduler-estimator
!!! Building karmada-scheduler-estimator for linux/amd64:
+ CGO_ENABLED=0
+ GOOS=linux
+ GOARCH=amd64
+ go build -ldflags '-X github.com/karmada-io/karmada/pkg/version.gitVersion=v1.13.0-alpha.2-15-g07747ed3f-dirty -X github.com/karmada-io/karmada/pkg/version.gitCommit=07747ed3f7a7c1a9e116156e8c751cca3b6e5d14 -X github.com/karmada-io/karmada/pkg/version.gitTreeState=dirty -X github.com/karmada-io/karmada/pkg/version.buildDate=2025-01-24T00:02:34Z ' -o _output/bin/linux/amd64/karmada-scheduler-estimator github.com/karmada-io/karmada/cmd/scheduler-estimator
+ set +x
make[1]: Leaving directory '/home/user/data/dev/apps/karmada'
Building image for linux/amd64: docker.io/karmada/karmada-scheduler-estimator:latest
+ docker build --build-arg BINARY=karmada-scheduler-estimator --label=image.karmada.io=May_be_pruned_in_local_up_environment --tag docker.io/karmada/karmada-scheduler-estimator:latest --file hack/../cluster/images/Dockerfile hack/../_output/bin/linux/amd64
[+] Building 2.3s (9/9) FINISHED                                                                                                                         docker:default
 => [internal] load build definition from Dockerfile                                                                                                               0.0s
 => => transferring dockerfile: 835B                                                                                                                               0.0s
 => [internal] load metadata for docker.io/library/alpine:3.21.2                                                                                                   0.5s
 => [internal] load .dockerignore                                                                                                                                  0.0s
 => => transferring context: 2B                                                                                                                                    0.0s
 => [1/4] FROM docker.io/library/alpine:3.21.2@sha256:56fa17d2a7e7f168a043a2712e63aed1f8543aeafdcee47c58dcffe38ed51099                                             0.0s
 => [internal] load build context                                                                                                                                  0.4s
 => => transferring context: 75.38MB                                                                                                                               0.4s
 => CACHED [2/4] RUN apk add --no-cache ca-certificates                                                                                                            0.0s
 => CACHED [3/4] RUN apk add --no-cache tzdata                                                                                                                     0.0s
 => [4/4] COPY karmada-scheduler-estimator /bin/karmada-scheduler-estimator                                                                                        0.7s
 => exporting to image                                                                                                                                             0.4s
 => => exporting layers                                                                                                                                            0.3s
 => => writing image sha256:f5a6f428ccd3679a45695e7dced140103fb3396351d4b8a64115870d9c613d78                                                                       0.0s
 => => naming to docker.io/karmada/karmada-scheduler-estimator:latest                                                                                              0.0s
+ set +x
set -e;\
target=$(echo karmada-interpreter-webhook-example);\
make $target GOOS=linux;\
VERSION=latest REGISTRY=docker.io/karmada BUILD_PLATFORMS=linux/amd64 hack/docker.sh $target
make[1]: Entering directory '/home/user/data/dev/apps/karmada'
BUILD_PLATFORMS=linux/amd64 hack/build.sh karmada-interpreter-webhook-example
!!! Building karmada-interpreter-webhook-example for linux/amd64:
+ CGO_ENABLED=0
+ GOOS=linux
+ GOARCH=amd64
+ go build -ldflags '-X github.com/karmada-io/karmada/pkg/version.gitVersion=v1.13.0-alpha.2-15-g07747ed3f-dirty -X github.com/karmada-io/karmada/pkg/version.gitCommit=07747ed3f7a7c1a9e116156e8c751cca3b6e5d14 -X github.com/karmada-io/karmada/pkg/version.gitTreeState=dirty -X github.com/karmada-io/karmada/pkg/version.buildDate=2025-01-24T00:02:39Z ' -o _output/bin/linux/amd64/karmada-interpreter-webhook-example github.com/karmada-io/karmada/examples/customresourceinterpreter/webhook
+ set +x
make[1]: Leaving directory '/home/user/data/dev/apps/karmada'
Building image for linux/amd64: docker.io/karmada/karmada-interpreter-webhook-example:latest
+ docker build --build-arg BINARY=karmada-interpreter-webhook-example --label=image.karmada.io=May_be_pruned_in_local_up_environment --tag docker.io/karmada/karmada-interpreter-webhook-example:latest --file hack/../cluster/images/Dockerfile hack/../_output/bin/linux/amd64
[+] Building 1.7s (9/9) FINISHED                                                                                                                         docker:default
 => [internal] load build definition from Dockerfile                                                                                                               0.0s
 => => transferring dockerfile: 835B                                                                                                                               0.0s
 => [internal] load metadata for docker.io/library/alpine:3.21.2                                                                                                   0.3s
 => [internal] load .dockerignore                                                                                                                                  0.0s
 => => transferring context: 2B                                                                                                                                    0.0s
 => [1/4] FROM docker.io/library/alpine:3.21.2@sha256:56fa17d2a7e7f168a043a2712e63aed1f8543aeafdcee47c58dcffe38ed51099                                             0.0s
 => [internal] load build context                                                                                                                                  0.3s
 => => transferring context: 60.50MB                                                                                                                               0.3s
 => CACHED [2/4] RUN apk add --no-cache ca-certificates                                                                                                            0.0s
 => CACHED [3/4] RUN apk add --no-cache tzdata                                                                                                                     0.0s
 => [4/4] COPY karmada-interpreter-webhook-example /bin/karmada-interpreter-webhook-example                                                                        0.6s
 => exporting to image                                                                                                                                             0.3s
 => => exporting layers                                                                                                                                            0.2s
 => => writing image sha256:193c0630b2ee29320c884e1eb136b212cd8aaf5d5e32a7fe398c50a5b5e7c116                                                                       0.0s
 => => naming to docker.io/karmada/karmada-interpreter-webhook-example:latest                                                                                      0.0s
+ set +x
set -e;\
target=$(echo karmada-search);\
make $target GOOS=linux;\
VERSION=latest REGISTRY=docker.io/karmada BUILD_PLATFORMS=linux/amd64 hack/docker.sh $target
make[1]: Entering directory '/home/user/data/dev/apps/karmada'
BUILD_PLATFORMS=linux/amd64 hack/build.sh karmada-search
!!! Building karmada-search for linux/amd64:
+ CGO_ENABLED=0
+ GOOS=linux
+ GOARCH=amd64
+ go build -ldflags '-X github.com/karmada-io/karmada/pkg/version.gitVersion=v1.13.0-alpha.2-15-g07747ed3f-dirty -X github.com/karmada-io/karmada/pkg/version.gitCommit=07747ed3f7a7c1a9e116156e8c751cca3b6e5d14 -X github.com/karmada-io/karmada/pkg/version.gitTreeState=dirty -X github.com/karmada-io/karmada/pkg/version.buildDate=2025-01-24T00:02:44Z ' -o _output/bin/linux/amd64/karmada-search github.com/karmada-io/karmada/cmd/karmada-search
+ set +x
make[1]: Leaving directory '/home/user/data/dev/apps/karmada'
Building image for linux/amd64: docker.io/karmada/karmada-search:latest
+ docker build --build-arg BINARY=karmada-search --label=image.karmada.io=May_be_pruned_in_local_up_environment --tag docker.io/karmada/karmada-search:latest --file hack/../cluster/images/Dockerfile hack/../_output/bin/linux/amd64
[+] Building 2.8s (9/9) FINISHED                                                                                                                         docker:default
 => [internal] load build definition from Dockerfile                                                                                                               0.0s
 => => transferring dockerfile: 835B                                                                                                                               0.0s
 => [internal] load metadata for docker.io/library/alpine:3.21.2                                                                                                   0.5s
 => [internal] load .dockerignore                                                                                                                                  0.0s
 => => transferring context: 2B                                                                                                                                    0.0s
 => [1/4] FROM docker.io/library/alpine:3.21.2@sha256:56fa17d2a7e7f168a043a2712e63aed1f8543aeafdcee47c58dcffe38ed51099                                             0.0s
 => [internal] load build context                                                                                                                                  0.5s
 => => transferring context: 108.48MB                                                                                                                              0.5s
 => CACHED [2/4] RUN apk add --no-cache ca-certificates                                                                                                            0.0s
 => CACHED [3/4] RUN apk add --no-cache tzdata                                                                                                                     0.0s
 => [4/4] COPY karmada-search /bin/karmada-search                                                                                                                  1.0s
 => exporting to image                                                                                                                                             0.5s
 => => exporting layers                                                                                                                                            0.4s
 => => writing image sha256:c5e2ba38fdce5829d22c645474450384977e84246c7c8797e57b551a2aad0ae6                                                                       0.0s
 => => naming to docker.io/karmada/karmada-search:latest                                                                                                           0.0s
+ set +x
set -e;\
target=$(echo karmada-operator);\
make $target GOOS=linux;\
VERSION=latest REGISTRY=docker.io/karmada BUILD_PLATFORMS=linux/amd64 hack/docker.sh $target
make[1]: Entering directory '/home/user/data/dev/apps/karmada'
BUILD_PLATFORMS=linux/amd64 hack/build.sh karmada-operator
!!! Building karmada-operator for linux/amd64:
+ CGO_ENABLED=0
+ GOOS=linux
+ GOARCH=amd64
+ go build -ldflags '-X github.com/karmada-io/karmada/pkg/version.gitVersion=v1.13.0-alpha.2-15-g07747ed3f-dirty -X github.com/karmada-io/karmada/pkg/version.gitCommit=07747ed3f7a7c1a9e116156e8c751cca3b6e5d14 -X github.com/karmada-io/karmada/pkg/version.gitTreeState=dirty -X github.com/karmada-io/karmada/pkg/version.buildDate=2025-01-24T00:02:50Z ' -o _output/bin/linux/amd64/karmada-operator github.com/karmada-io/karmada/operator/cmd/operator
+ set +x
make[1]: Leaving directory '/home/user/data/dev/apps/karmada'
Building image for linux/amd64: docker.io/karmada/karmada-operator:latest
+ docker build --build-arg BINARY=karmada-operator --label=image.karmada.io=May_be_pruned_in_local_up_environment --tag docker.io/karmada/karmada-operator:latest --file hack/../cluster/images/Dockerfile hack/../_output/bin/linux/amd64
[+] Building 2.4s (9/9) FINISHED                                                                                                                         docker:default
 => [internal] load build definition from Dockerfile                                                                                                               0.1s
 => => transferring dockerfile: 835B                                                                                                                               0.0s
 => [internal] load metadata for docker.io/library/alpine:3.21.2                                                                                                   0.5s
 => [internal] load .dockerignore                                                                                                                                  0.0s
 => => transferring context: 2B                                                                                                                                    0.0s
 => [1/4] FROM docker.io/library/alpine:3.21.2@sha256:56fa17d2a7e7f168a043a2712e63aed1f8543aeafdcee47c58dcffe38ed51099                                             0.0s
 => [internal] load build context                                                                                                                                  0.4s
 => => transferring context: 73.28MB                                                                                                                               0.3s
 => CACHED [2/4] RUN apk add --no-cache ca-certificates                                                                                                            0.0s
 => CACHED [3/4] RUN apk add --no-cache tzdata                                                                                                                     0.0s
 => [4/4] COPY karmada-operator /bin/karmada-operator                                                                                                              0.7s
 => exporting to image                                                                                                                                             0.5s
 => => exporting layers                                                                                                                                            0.4s
 => => writing image sha256:aa29b9af5f656290716f631456fbda06b0fcf534d206ad532343c2dd5c3eae1d                                                                       0.0s
 => => naming to docker.io/karmada/karmada-operator:latest                                                                                                         0.0s
+ set +x
set -e;\
target=$(echo karmada-metrics-adapter);\
make $target GOOS=linux;\
VERSION=latest REGISTRY=docker.io/karmada BUILD_PLATFORMS=linux/amd64 hack/docker.sh $target
make[1]: Entering directory '/home/user/data/dev/apps/karmada'
BUILD_PLATFORMS=linux/amd64 hack/build.sh karmada-metrics-adapter
!!! Building karmada-metrics-adapter for linux/amd64:
+ CGO_ENABLED=0
+ GOOS=linux
+ GOARCH=amd64
+ go build -ldflags '-X github.com/karmada-io/karmada/pkg/version.gitVersion=v1.13.0-alpha.2-15-g07747ed3f-dirty -X github.com/karmada-io/karmada/pkg/version.gitCommit=07747ed3f7a7c1a9e116156e8c751cca3b6e5d14 -X github.com/karmada-io/karmada/pkg/version.gitTreeState=dirty -X github.com/karmada-io/karmada/pkg/version.buildDate=2025-01-24T00:02:56Z ' -o _output/bin/linux/amd64/karmada-metrics-adapter github.com/karmada-io/karmada/cmd/metrics-adapter
+ set +x
make[1]: Leaving directory '/home/user/data/dev/apps/karmada'
Building image for linux/amd64: docker.io/karmada/karmada-metrics-adapter:latest
+ docker build --build-arg BINARY=karmada-metrics-adapter --label=image.karmada.io=May_be_pruned_in_local_up_environment --tag docker.io/karmada/karmada-metrics-adapter:latest --file hack/../cluster/images/Dockerfile hack/../_output/bin/linux/amd64
[+] Building 2.4s (9/9) FINISHED                                                                                                                         docker:default
 => [internal] load build definition from Dockerfile                                                                                                               0.0s
 => => transferring dockerfile: 835B                                                                                                                               0.0s
 => [internal] load metadata for docker.io/library/alpine:3.21.2                                                                                                   0.5s
 => [internal] load .dockerignore                                                                                                                                  0.0s
 => => transferring context: 2B                                                                                                                                    0.0s
 => [1/4] FROM docker.io/library/alpine:3.21.2@sha256:56fa17d2a7e7f168a043a2712e63aed1f8543aeafdcee47c58dcffe38ed51099                                             0.0s
 => [internal] load build context                                                                                                                                  0.5s
 => => transferring context: 101.29MB                                                                                                                              0.4s
 => CACHED [2/4] RUN apk add --no-cache ca-certificates                                                                                                            0.0s
 => CACHED [3/4] RUN apk add --no-cache tzdata                                                                                                                     0.0s
 => [4/4] COPY karmada-metrics-adapter /bin/karmada-metrics-adapter                                                                                                0.8s
 => exporting to image                                                                                                                                             0.4s
 => => exporting layers                                                                                                                                            0.4s
 => => writing image sha256:7841176e6ccc2fb08219db60546a4c05cb3cacf7cdc07b4208c086d6c669d8f5                                                                       0.0s
 => => naming to docker.io/karmada/karmada-metrics-adapter:latest                                                                                                  0.0s
+ set +x
make: Leaving directory '/home/user/data/dev/apps/karmada'
Deleted Images:
deleted: sha256:dafcb6a3e0797fea4b0e2796c3e985c797627c3af4be37d82af3f7fe0d7fcae4
deleted: sha256:6a3131b28c2ac5b3c287f434e8473d9f9821ecbc9ae79d879459a54c44725db7
deleted: sha256:657f3b7cb55399dfde51f0e939659fe83a24c840bc4fa27dde87684e1b4f668b
deleted: sha256:5e2e6756263b964aca62eb060cd843651143aaf5897ae2eb22940c8d5527a65c

Total reclaimed space: 0B
Waiting for the clusters to be ready...
Waiting for kubeconfig file /home/user/.kube/karmada.config and clusters karmada-host to be ready...
Context "kind-karmada-host" renamed to "karmada-host".
Cluster "kind-karmada-host" set.
Waiting for kubeconfig file /home/user/.kube/member-tmp-member1.config and clusters member1 to be ready...
Context "kind-member1" renamed to "member1".
Cluster "kind-member1" set.
Waiting for kubeconfig file /home/user/.kube/member-tmp-member2.config and clusters member2 to be ready...
Context "kind-member2" renamed to "member2".
Cluster "kind-member2" set.
Waiting for kubeconfig file /home/user/.kube/member-tmp-member3.config and clusters member3 to be ready...
Context "kind-member3" renamed to "member3".
Cluster "kind-member3" set.
Image: "docker.io/karmada/karmada-controller-manager:latest" with ID "sha256:3252043c11793a9801411d50de4d650dee80d15ef26f5ed5c6bcebb407afba37" not yet present on node "karmada-host-control-plane", loading...
Image: "docker.io/karmada/karmada-scheduler:latest" with ID "sha256:ec87e54ae1e0809255493b22fb02d9f0995319efce4cf1c9e901fd2985bb418a" not yet present on node "karmada-host-control-plane", loading...
Image: "docker.io/karmada/karmada-descheduler:latest" with ID "sha256:50496a8966bf5489624abd47b2aa2392f4094946ca1832c346d779cf84704c65" not yet present on node "karmada-host-control-plane", loading...
Image: "docker.io/karmada/karmada-webhook:latest" with ID "sha256:175f74eb61f0b74d50eeda0c45e11f554b7756ad6fa75098fc95b50e94330256" not yet present on node "karmada-host-control-plane", loading...
Image: "docker.io/karmada/karmada-scheduler-estimator:latest" with ID "sha256:f5a6f428ccd3679a45695e7dced140103fb3396351d4b8a64115870d9c613d78" not yet present on node "karmada-host-control-plane", loading...
Image: "docker.io/karmada/karmada-aggregated-apiserver:latest" with ID "sha256:ec4c7ffdf0175adf1d1696b689a34e104f400028d3128ab3155302fcea0d48b5" not yet present on node "karmada-host-control-plane", loading...
Image: "docker.io/karmada/karmada-search:latest" with ID "sha256:c5e2ba38fdce5829d22c645474450384977e84246c7c8797e57b551a2aad0ae6" not yet present on node "karmada-host-control-plane", loading...
Image: "docker.io/karmada/karmada-metrics-adapter:latest" with ID "sha256:7841176e6ccc2fb08219db60546a4c05cb3cacf7cdc07b4208c086d6c669d8f5" not yet present on node "karmada-host-control-plane", loading...
Image: "docker.io/karmada/karmada-agent:latest" with ID "sha256:50cee6b2892054c7858abc62a11cd3c2317775eeb75c3b4b1d785bdbc2a19f91" not yet present on node "member3-control-plane", loading...
connecting cluster networks...
Connecting cluster member1 to /home/user/.kube/member-tmp-member2.config
exec cmd in docker member1-control-plane ip route add 10.12.0.0/24 via 172.20.0.4
Connecting cluster member2 to /home/user/.kube/member-tmp-member1.config
exec cmd in docker member2-control-plane ip route add 10.10.0.0/24 via 172.20.0.6
Connecting cluster karmada-host to /home/user/.kube/member-tmp-member1.config
exec cmd in docker karmada-host-control-plane ip route add 10.10.0.0/24 via 172.20.0.6
Connecting cluster member1 to /home/user/.kube/karmada.config
exec cmd in docker member1-control-plane ip route add 10.244.0.0/24 via 172.20.0.5
Connecting cluster karmada-host to /home/user/.kube/member-tmp-member2.config
exec cmd in docker karmada-host-control-plane ip route add 10.12.0.0/24 via 172.20.0.4
Connecting cluster member2 to /home/user/.kube/karmada.config
exec cmd in docker member2-control-plane ip route add 10.244.0.0/24 via 172.20.0.5
cluster networks connected
go install github.com/cloudflare/cfssl/cmd/[email protected]
go: downloading github.com/cloudflare/cfssl v1.6.5
go: downloading github.com/go-sql-driver/mysql v1.7.1
go: downloading github.com/mattn/go-sqlite3 v1.14.22
go: downloading github.com/lib/pq v1.10.9
go: downloading github.com/prometheus/client_golang v1.19.0
go: downloading github.com/zmap/zlint/v3 v3.5.0
go: downloading github.com/jmoiron/sqlx v1.3.5
go: downloading github.com/kisielk/sqlstruct v0.0.0-20201105191214-5f3e10d3ab46
go: downloading github.com/google/certificate-transparency-go v1.1.7
go: downloading golang.org/x/crypto v0.19.0
go: downloading github.com/cloudflare/redoctober v0.0.0-20211013234631-6a74ccc611f6
go: downloading github.com/zmap/zcrypto v0.0.0-20230310154051-c8b263fd8300
go: downloading github.com/jmhodges/clock v1.2.0
go: downloading github.com/prometheus/common v0.48.0
go: downloading github.com/prometheus/client_model v0.5.0
go: downloading github.com/beorn7/perks v1.0.1
go: downloading github.com/cespare/xxhash/v2 v2.2.0
go: downloading github.com/prometheus/procfs v0.12.0
go: downloading google.golang.org/protobuf v1.32.0
go: downloading golang.org/x/net v0.20.0
go: downloading k8s.io/klog/v2 v2.100.1
go: downloading github.com/getsentry/sentry-go v0.11.0
go: downloading golang.org/x/sys v0.17.0
go: downloading github.com/go-logr/logr v1.2.4
go: downloading github.com/weppos/publicsuffix-go v0.30.0
go: downloading github.com/pelletier/go-toml v1.9.3
go: downloading golang.org/x/text v0.14.0
.+.+...........+....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*....+.........+..........+...+..+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..............+....+.................+......+...+.+......+..+......+....+...........+....+...+.........+...+.....+...+...+....+......+.....+.+.........+......+........+............+......+....+...+..+....+...........+.+..............+......+....+.........+......+...............+.....+.+........+......+...............+.+........+....+..+...+...............+.+.........+.....+.+......+.....+................+.................+..........+........+...+......+......+....+...........+...+...................+...+..+.+............+..+.+...........+..........+...+........+.......+..+.............+..................+..+......+.......+.........+....................+..........+.....+.+...+..+.........+..........+...........+...+..........+.........+...+........+.+.....+..........+........+................................................+...+................+........+.+.....+......................+.............................+....+..+.........+...+................+............+..+.....................+....+..+.+.........+...+.................+....+..+............+.+.........+.....+.............+...+............+.....+...+............+...+..........+...........+......+.......+...............+......+......+.....+.+........+..........+.........+.....+......+.......+..............+......+....+...+..+....+......+.........+..+.........+..........+.....+...+...+....+......+.....+...+.+.........+.....+.......+...+..+.......+....................+.............+..+......+.+.......................+.......+...+.................+.............+..+.........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
..+......+..+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+.......+............+.....+.......+......+.....+.......+......+......+.....+......+.......+..+..........+.....+....+..+...+...........................+....+...+............+...+...+.........+...........+...+.......+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----
2025/01/24 00:04:54 [INFO] generate received request
2025/01/24 00:04:54 [INFO] received CSR
2025/01/24 00:04:54 [INFO] generating key: rsa-3072
2025/01/24 00:04:55 [INFO] encoded CSR
2025/01/24 00:04:55 [INFO] signed certificate with serial number 399757365533017803196698675805111630451566367967
2025/01/24 00:04:55 [INFO] generate received request
2025/01/24 00:04:55 [INFO] received CSR
2025/01/24 00:04:55 [INFO] generating key: rsa-3072
2025/01/24 00:04:55 [INFO] encoded CSR
2025/01/24 00:04:55 [INFO] signed certificate with serial number 579340700356190127597689300489457173329722577615
2025/01/24 00:04:55 [INFO] generate received request
2025/01/24 00:04:55 [INFO] received CSR
2025/01/24 00:04:55 [INFO] generating key: rsa-3072
2025/01/24 00:04:56 [INFO] encoded CSR
2025/01/24 00:04:56 [INFO] signed certificate with serial number 391562778338020287262629856713772296213960600462
2025/01/24 00:04:56 [INFO] generate received request
2025/01/24 00:04:56 [INFO] received CSR
2025/01/24 00:04:56 [INFO] generating key: rsa-3072
2025/01/24 00:04:57 [INFO] encoded CSR
2025/01/24 00:04:57 [INFO] signed certificate with serial number 221739118562535443927053683716700220585501051072
2025/01/24 00:04:58 [INFO] generate received request
2025/01/24 00:04:58 [INFO] received CSR
2025/01/24 00:04:58 [INFO] generating key: rsa-3072
2025/01/24 00:04:58 [INFO] encoded CSR
2025/01/24 00:04:58 [INFO] signed certificate with serial number 689247321437652550900107462834689080900048869855
writing RSA key
namespace/karmada-system created
/tmp/tmp.lr2IFQyeUm
secret/karmada-apiserver-cert created
secret/karmada-aggregated-apiserver-cert created
secret/karmada-metrics-adapter-cert created
secret/karmada-search-cert created
secret/karmada-webhook-cert created
secret/karmada-interpreter-webhook-example-cert created
secret/etcd-cert created
secret/karmada-apiserver-etcd-client-cert created
secret/karmada-apiserver-front-proxy-client-cert created
secret/karmada-aggregated-apiserver-etcd-client-cert created
secret/karmada-search-etcd-client-cert created
secret/etcd-etcd-client-cert created
secret/karmada-scheduler-scheduler-estimator-client-cert created
secret/karmada-descheduler-scheduler-estimator-client-cert created
secret/kube-controller-manager-ca-cert created
secret/kube-controller-manager-service-account-key-pair created
secret/karmada-apiserver-service-account-key-pair created
secret/karmada-aggregated-apiserver-config created
secret/karmada-controller-manager-config created
secret/kube-controller-manager-config created
secret/karmada-scheduler-config created
secret/karmada-descheduler-config created
secret/karmada-metrics-adapter-config created
secret/karmada-search-config created
secret/karmada-webhook-config created
secret/karmada-interpreter-webhook-example-config created
statefulset.apps/etcd created
service/etcd-client created
service/etcd created
wait the etcd ready...
pod/etcd-0 condition met

Apply dynamic rendered apiserver service in /tmp/tmp.FwcYFAMynC/karmada-apiserver.yaml.
deployment.apps/karmada-apiserver created
service/karmada-apiserver created
wait the karmada-apiserver ready...
error: timed out waiting for the condition on pods/karmada-apiserver-77bd48668f-v4ltb
kubectl --context=karmada-host wait --for=condition=Ready --timeout=30s pods -l app=karmada-apiserver -n karmada-system failed, retrying(1 times)
error: timed out waiting for the condition on pods/karmada-apiserver-77bd48668f-v4ltb
kubectl --context=karmada-host wait --for=condition=Ready --timeout=30s pods -l app=karmada-apiserver -n karmada-system failed, retrying(2 times)
error: timed out waiting for the condition on pods/karmada-apiserver-77bd48668f-v4ltb
kubectl --context=karmada-host wait --for=condition=Ready --timeout=30s pods -l app=karmada-apiserver -n karmada-system failed, retrying(3 times)
error: timed out waiting for the condition on pods/karmada-apiserver-77bd48668f-v4ltb
kubectl --context=karmada-host wait --for=condition=Ready --timeout=30s pods -l app=karmada-apiserver -n karmada-system failed, retrying(4 times)
error: timed out waiting for the condition on pods/karmada-apiserver-77bd48668f-v4ltb
kubectl --context=karmada-host wait --for=condition=Ready --timeout=30s pods -l app=karmada-apiserver -n karmada-system failed, retrying(5 times)
error: timed out waiting for the condition on pods/karmada-apiserver-77bd48668f-v4ltb
kubectl --context=karmada-host wait --for=condition=Ready --timeout=30s pods -l app=karmada-apiserver -n karmada-system failed, retrying(6 times)
error: timed out waiting for the condition on pods/karmada-apiserver-77bd48668f-v4ltb
kubectl --context=karmada-host wait --for=condition=Ready --timeout=30s pods -l app=karmada-apiserver -n karmada-system failed, retrying(7 times)
error: timed out waiting for the condition on pods/karmada-apiserver-77bd48668f-v4ltb
kubectl --context=karmada-host wait --for=condition=Ready --timeout=30s pods -l app=karmada-apiserver -n karmada-system failed, retrying(8 times)
error: timed out waiting for the condition on pods/karmada-apiserver-77bd48668f-v4ltb
kubectl --context=karmada-host wait --for=condition=Ready --timeout=30s pods -l app=karmada-apiserver -n karmada-system failed, retrying(9 times)
error: timed out waiting for the condition on pods/karmada-apiserver-77bd48668f-v4ltb
kubectl --context=karmada-host wait --for=condition=Ready --timeout=30s pods -l app=karmada-apiserver -n karmada-system failed, retrying(10 times)
kubectl --context=karmada-host wait --for=condition=Ready --timeout=30s pods -l app=karmada-apiserver -n karmada-system failed
error: timed out waiting for the condition on pods/karmada-apiserver-77bd48668f-v4ltb
kubectl describe info:
Name:                 karmada-apiserver-77bd48668f-v4ltb
Namespace:            karmada-system
Priority:             2000001000
Priority Class Name:  system-node-critical
Service Account:      default
Node:                 karmada-host-control-plane/172.20.0.5
Start Time:           Fri, 24 Jan 2025 00:05:18 +0000
Labels:               app=karmada-apiserver
                      pod-template-hash=77bd48668f
Annotations:          <none>
Status:               Running
SeccompProfile:       RuntimeDefault
IP:                   172.20.0.5
IPs:
  IP:           172.20.0.5
Controlled By:  ReplicaSet/karmada-apiserver-77bd48668f
Containers:
  karmada-apiserver:
    Container ID:  containerd://2f09e734edbdcb21c64f43b12f7fa3ec46478e57f7824095e907214566fe1410
    Image:         registry.k8s.io/kube-apiserver:v1.31.3
    Image ID:      registry.k8s.io/kube-apiserver@sha256:c040161602d7b5c0f899ef41d0e53b28fe007120819e91440f89d6a4b15fbcee
    Port:          <none>
    Host Port:     <none>
    Command:
      kube-apiserver
      --allow-privileged=true
      --authorization-mode=Node,RBAC
      --enable-bootstrap-token-auth=true
      --etcd-cafile=/etc/karmada/pki/etcd-client/ca.crt
      --etcd-certfile=/etc/karmada/pki/etcd-client/tls.crt
      --etcd-keyfile=/etc/karmada/pki/etcd-client/tls.key
      --etcd-servers=https://etcd-client.karmada-system.svc.cluster.local:2379
      --bind-address=0.0.0.0
      --disable-admission-plugins=StorageObjectInUseProtection,ServiceAccount
      --runtime-config=
      --secure-port=5443
      --service-account-issuer=https://kubernetes.default.svc.cluster.local
      --service-account-key-file=/etc/karmada/pki/service-account-key-pair/sa.pub
      --service-account-signing-key-file=/etc/karmada/pki/service-account-key-pair/sa.key
      --service-cluster-ip-range=10.96.0.0/12
      --proxy-client-cert-file=/etc/karmada/pki/front-proxy-client/tls.crt
      --proxy-client-key-file=/etc/karmada/pki/front-proxy-client/tls.key
      --requestheader-client-ca-file=/etc/karmada/pki/front-proxy-client/ca.crt
      --requestheader-allowed-names=front-proxy-client
      --requestheader-extra-headers-prefix=X-Remote-Extra-
      --requestheader-group-headers=X-Remote-Group
      --requestheader-username-headers=X-Remote-User
      --tls-cert-file=/etc/karmada/pki/server/tls.crt
      --tls-private-key-file=/etc/karmada/pki/server/tls.key
      --client-ca-file=/etc/karmada/pki/server/ca.crt
      --tls-min-version=VersionTLS13
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    255
      Started:      Fri, 24 Jan 2025 00:10:06 +0000
      Finished:     Fri, 24 Jan 2025 00:10:26 +0000
    Ready:          False
    Restart Count:  5
    Requests:
      cpu:        250m
    Liveness:     http-get https://:5443/livez delay=10s timeout=15s period=10s #success=1 #failure=8
    Readiness:    http-get https://:5443/readyz delay=0s timeout=15s period=1s #success=1 #failure=3
    Environment:  <none>
    Mounts:
      /etc/karmada/pki/etcd-client from etcd-client-cert (ro)
      /etc/karmada/pki/front-proxy-client from front-proxy-client-cert (ro)
      /etc/karmada/pki/server from server-cert (ro)
      /etc/karmada/pki/service-account-key-pair from service-account-key-pair (ro)
Conditions:
  Type                        Status
  PodReadyToStartContainers   True 
  Initialized                 True 
  Ready                       False 
  ContainersReady             False 
  PodScheduled                True 
Volumes:
  server-cert:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  karmada-apiserver-cert
    Optional:    false
  etcd-client-cert:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  karmada-apiserver-etcd-client-cert
    Optional:    false
  front-proxy-client-cert:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  karmada-apiserver-front-proxy-client-cert
    Optional:    false
  service-account-key-pair:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  karmada-apiserver-service-account-key-pair
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     :NoExecute op=Exists
Events:
  Type     Reason     Age                    From               Message
  ----     ------     ----                   ----               -------
  Normal   Scheduled  5m41s                  default-scheduler  Successfully assigned karmada-system/karmada-apiserver-77bd48668f-v4ltb to karmada-host-control-plane
  Normal   Pulling    5m40s                  kubelet            Pulling image "registry.k8s.io/kube-apiserver:v1.31.3"
  Normal   Pulled     5m33s                  kubelet            Successfully pulled image "registry.k8s.io/kube-apiserver:v1.31.3" in 7.185s (7.185s including waiting). Image size: 27972563 bytes.
  Warning  Unhealthy  5m12s                  kubelet            Liveness probe failed: Get "https://172.20.0.5:5443/livez": read tcp 172.20.0.5:50584->172.20.0.5:5443: read: connection reset by peer
  Warning  Unhealthy  5m12s                  kubelet            Readiness probe failed: Get "https://172.20.0.5:5443/readyz": read tcp 172.20.0.5:50574->172.20.0.5:5443: read: connection reset by peer
  Warning  Unhealthy  4m51s                  kubelet            Liveness probe failed: Get "https://172.20.0.5:5443/livez": read tcp 172.20.0.5:50950->172.20.0.5:5443: read: connection reset by peer
  Warning  Unhealthy  4m51s                  kubelet            Readiness probe failed: Get "https://172.20.0.5:5443/readyz": read tcp 172.20.0.5:50934->172.20.0.5:5443: read: connection reset by peer
  Warning  Unhealthy  4m50s                  kubelet            Liveness probe failed: Get "https://172.20.0.5:5443/livez": dial tcp 172.20.0.5:5443: connect: connection refused
  Warning  Unhealthy  4m50s (x4 over 5m12s)  kubelet            Readiness probe failed: Get "https://172.20.0.5:5443/readyz": dial tcp 172.20.0.5:5443: connect: connection refused
  Warning  BackOff    4m50s                  kubelet            Back-off restarting failed container karmada-apiserver in pod karmada-apiserver-77bd48668f-v4ltb_karmada-system(14465634-aa29-44ad-a35d-0376d7e42cfe)
  Normal   Pulled     4m40s (x2 over 5m12s)  kubelet            Container image "registry.k8s.io/kube-apiserver:v1.31.3" already present on machine
  Normal   Started    4m40s (x3 over 5m32s)  kubelet            Started container karmada-apiserver
  Normal   Created    4m40s (x3 over 5m32s)  kubelet            Created container karmada-apiserver
  Warning  Unhealthy  4m20s (x4 over 5m22s)  kubelet            Readiness probe failed: Get "https://172.20.0.5:5443/readyz": net/http: TLS handshake timeout
  Warning  Unhealthy  4m20s                  kubelet            Liveness probe failed: Get "https://172.20.0.5:5443/livez": net/http: TLS handshake timeout
  Warning  Unhealthy  33s (x8 over 4m20s)    kubelet            (combined from similar events): Liveness probe failed: Get "https://172.20.0.5:5443/livez": read tcp 172.20.0.5:52116->172.20.0.5:5443: read: connection reset by peer
kubectl logs info:
I0124 00:10:06.526967       1 server.go:144] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
I0124 00:10:06.713777       1 shared_informer.go:313] Waiting for caches to sync for node_authorizer
I0124 00:10:06.717550       1 shared_informer.go:313] Waiting for caches to sync for *generic.policySource[*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicy,*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicyBinding,k8s.io/apiserver/pkg/admission/plugin/policy/validating.Validator]
I0124 00:10:06.720210       1 plugins.go:157] Loaded 9 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook.
I0124 00:10:06.720234       1 plugins.go:160] Loaded 12 validating admission controller(s) successfully in the following order: LimitRanger,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,ClusterTrustBundleAttest,CertificateSubjectRestriction,ValidatingAdmissionPolicy,ValidatingAdmissionWebhook,ResourceQuota.
I0124 00:10:06.720427       1 instance.go:232] Using reconciler: lease
W0124 00:10:26.713962       1 logging.go:55] [core] [Channel #1 SubChannel #3]grpc: addrConn.createTransport failed to connect to {Addr: "etcd-client.karmada-system.svc.cluster.local:2379", ServerName: "etcd-client.karmada-system.svc.cluster.local:2379", }. Err: connection error: desc = "transport: Error while dialing: dial tcp: lookup etcd-client.karmada-system.svc.cluster.local: operation was canceled"
W0124 00:10:26.713968       1 logging.go:55] [core] [Channel #2 SubChannel #4]grpc: addrConn.createTransport failed to connect to {Addr: "etcd-client.karmada-system.svc.cluster.local:2379", ServerName: "etcd-client.karmada-system.svc.cluster.local:2379", }. Err: connection error: desc = "transport: Error while dialing: dial tcp: lookup etcd-client.karmada-system.svc.cluster.local: operation was canceled"
W0124 00:10:26.721496       1 logging.go:55] [core] [Channel #5 SubChannel #6]grpc: addrConn.createTransport failed to connect to {Addr: "etcd-client.karmada-system.svc.cluster.local:2379", ServerName: "etcd-client.karmada-system.svc.cluster.local:2379", }. Err: connection error: desc = "transport: Error while dialing: dial tcp: lookup etcd-client.karmada-system.svc.cluster.local: i/o timeout"
F0124 00:10:26.721508       1 instance.go:225] Error creating leases: error creating storage factory: context deadline exceeded

RainbowMango · 2025-01-24T01:37:28Z

error: failed to create directory: /etc/karmada, error: mkdir /etc/karmada: permission denied

I remember we fixed this issue recently, @zhzhuang-zju is it you sent the PR?

zhzhuang-zju · 2025-01-24T01:57:15Z

I remember we fixed this issue recently, @zhzhuang-zju is it you sent the PR?

The command I fixed is kubectl karmada register. Unlike this command, kubectl karmada init requires local files to store certificates and other files, and the paths to these files can be customized.

	flags.StringVarP(&opts.KarmadaDataPath, "karmada-data", "d", "/etc/karmada", "Karmada data path. kubeconfig cert and crds files")
	flags.StringVarP(&opts.KarmadaPkiPath, "karmada-pki", "", "/etc/karmada/pki", "Karmada pki path. Karmada cert files")

@olariuromeo You can set these two flags to customize the file paths, thereby avoiding issues related to insufficient permissions, for example,

$ kubectl karmada init --crds https://github.com/karmada-io/karmada/releases/download/v1.2.0/crds.tar.gz --kubeconfig=$HOME/.kube/host.config --karmada-data=${HOME}/karmada --karmada-pki=${HOME}/karmada/pki

olariuromeo · 2025-01-24T02:02:23Z

I remember we fixed this issue recently, @zhzhuang-zju is it you sent the PR?

The command I fixed is kubectl karmada register. Unlike this command, kubectl karmada init requires local files to store certificates and other files, and the paths to these files can be customized.

flags.StringVarP(&opts.KarmadaDataPath, "karmada-data", "d", "/etc/karmada", "Karmada data path. kubeconfig cert and crds files")
flags.StringVarP(&opts.KarmadaPkiPath, "karmada-pki", "", "/etc/karmada/pki", "Karmada pki path. Karmada cert files")
@olariuromeo You can set these two flags to customize the file paths, thereby avoiding issues related to insufficient permissions, for example,

$ kubectl karmada init --crds https://github.com/karmada-io/karmada/releases/download/v1.2.0/crds.tar.gz --kubeconfig=$HOME/.kube/host.config --karmada-data=${HOME}/karmada --karmada-pki=${HOME}/karmada/pki

Thanks, I'll try that, in this case the documentation should be modified: https://karmada.io/docs/installation/#install-karmada-in-kind-cluster
Regards

zhzhuang-zju · 2025-01-24T02:09:27Z

in this case the documentation should be modified: https://karmada.io/docs/installation/#install-karmada-in-kind-cluster

Yes, the previous documentation did not provide detailed instructions for operations by non-root users. It would be valuable to update the documentation to include this information.
@olariuromeo Are you interested in submitting a PR to update it?

olariuromeo · 2025-01-24T02:10:13Z

Ok now it started correctly. So who wants to start automatically in 2 minutes here is the script

#!/bin/bash

set -e

source ~/.bashrc
# Define the absolute path to the install directory
INSTALL_DIR="$(dirname "$(realpath "$0")")"

# Step 1: Install or verify asdf version manager
echo "Step 1: Verifying asdf installation..."
if ! command -v asdf &>/dev/null; then
    echo "asdf not found! Installing asdf..."
    git clone https://github.com/asdf-vm/asdf.git ~/.asdf --branch v0.10.0
    echo "asdf installed successfully!"
    source ~/.bashrc
fi

# Step 2: Install Go plugin for asdf if not installed
echo "Step 2: Verifying Go plugin for asdf..."
if ! asdf plugin-list | grep -q "golang"; then
    echo "Go plugin not found! Installing Go plugin for asdf..."
    asdf plugin-add golang https://github.com/kennyp/asdf-golang.git
    echo "Go plugin installed successfully!"
fi

# Step 3: Install Golang version 1.23.5 using asdf
echo "Step 3: Installing Golang version 1.23.5..."
asdf install golang 1.23.5
asdf local golang 1.23.5
echo "Golang 1.23.5 installed successfully!"

# Step 4: Install kubectl
echo "Step: Installing kubectl..."

# Download kubectl binary
curl -LO https://dl.k8s.io/release/v1.29.0/bin/linux/amd64/kubectl

# Download sha256 checksum file for kubectl
curl -LO "https://dl.k8s.io/release/v1.29.0/bin/linux/amd64/kubectl.sha256"

# Verify the checksum
echo "$(cat kubectl.sha256)  kubectl" | sha256sum --check

# Make kubectl executable
chmod +x kubectl

# Create directory for user-specific binaries if it doesn't exist
mkdir -p ~/.local/bin

# Move kubectl to ~/.local/bin
mv kubectl ~/.local/bin/kubectl

# Add ~/.local/bin to PATH if it's not already included
if ! grep -q "$HOME/.local/bin" ~/.bashrc; then
    echo 'export PATH=$HOME/.local/bin:$PATH' >> ~/.bashrc
fi

# Reload bashrc to apply changes (this applies only for future shell sessions)
source ~/.bashrc

# Directly check if kubectl is now available in PATH
if ! command -v kubectl &>/dev/null; then
    echo "kubectl is not available in PATH, trying again with direct path"
    export PATH="$HOME/.local/bin:$PATH"
fi

# Verify kubectl installation
kubectl version --client


# Step 5: Install Kind
echo "Step: Installing Kind..."
go install sigs.k8s.io/[email protected]

# Add GOPATH/bin to PATH
echo "Step: Adding GOPATH/bin to PATH..."
export PATH=$PATH:$(go env GOPATH)/bin

# Verify Kind installation
echo "Step: Verifying Kind installation..."
if ! command -v kind &>/dev/null; then
    echo "Kind is not installed correctly! Please check the installation."
    exit 1
fi

# Verify Kind version
kind_version=$(kind version)
echo "Kind installed successfully! Version: $kind_version"

# Add GOPATH/bin to shell PATH permanently (for future sessions)
if ! grep -q "$(go env GOPATH)/bin" ~/.bashrc; then
    echo "Adding GOPATH/bin to ~/.bashrc..."
    echo "export PATH=\$PATH:$(go env GOPATH)/bin" >> ~/.bashrc
    source ~/.bashrc
    echo "GOPATH/bin added to ~/.bashrc for future sessions."
fi

# Step 6: Install Krew
# Check system architecture
echo "Step: Installing Krew..."
OS=$(uname | tr '[:upper:]' '[:lower:]')
ARCH=$(uname -m)
[ "$ARCH" == "x86_64" ] && ARCH="amd64" || true

# Define Krew name based on the system architecture
KREW="krew-${OS}_${ARCH}"

# Download and install Krew
curl -fsSL "https://github.com/kubernetes-sigs/krew/releases/latest/download/${KREW}.tar.gz" | tar zx
./${KREW} install krew

# Step 7: Add Krew to PATH
echo "Step: Adding Krew to PATH..."
if ! grep -q 'export PATH="${KREW_ROOT:-$HOME/.krew}/bin:$PATH"' ~/.bashrc; then
    echo 'export PATH="${KREW_ROOT:-$HOME/.krew}/bin:$PATH"' >> ~/.bashrc
fi
export PATH="${KREW_ROOT:-$HOME/.krew}/bin:$PATH"
source ~/.bashrc

# Step 8: Verify Krew installation
echo "Step: Verifying Krew installation..."
kubectl krew version

# Step 9: Create Kind cluster
echo "Step: Creating Kind cluster named 'host'..."
./hack/create-cluster.sh host $HOME/.kube/host.config

# Step 9.1: Verify the Kind cluster is running (with retries)
echo "Step: Verifying Kind cluster is running..."
max_retries=10
retry_count=0
while ! kubectl cluster-info --kubeconfig=$HOME/.kube/host.config &>/dev/null; do
    if [ $retry_count -ge $max_retries ]; then
        echo "Kind cluster is not running after $max_retries attempts. Exiting..."
        exit 1
    fi
    echo "Kind cluster is not running, retrying in 10 seconds... Attempt $((retry_count + 1))/$max_retries"
    ((retry_count++))
    sleep 10
done

# Add a small delay to ensure that the cluster is fully initialized
echo "Waiting an additional 10 seconds for cluster to stabilize..."
sleep 10
echo "Kind cluster is running correctly!"

# Step 10: Install Karmada plugin via Krew
echo "Step: Installing Karmada plugin via Krew..."
kubectl krew install karmada

# Step 11: Verify Karmada plugin installation
echo "Step: Verifying Karmada plugin installation..."
if ! command -v kubectl-karmada &>/dev/null; then
    echo "Karmada plugin is not installed correctly!"
    exit 1
fi
echo "Karmada plugin installed successfully!"

# Step 12: Initialize Karmada
echo "Step: Initializing Karmada..."
kubectl karmada init --crds https://github.com/karmada-io/karmada/releases/download/v1.2.0/crds.tar.gz --kubeconfig=$HOME/.kube/host.config --karmada-data=${HOME}/karmada --karmada-pki=${HOME}/karmada/pki
echo "Karmada plugin installed and initialized successfully!"

regards

------------------------------------------------------------------------------------------------------
 █████   ████   █████████   ███████████   ██████   ██████   █████████   ██████████     █████████
░░███   ███░   ███░░░░░███ ░░███░░░░░███ ░░██████ ██████   ███░░░░░███ ░░███░░░░███   ███░░░░░███
 ░███  ███    ░███    ░███  ░███    ░███  ░███░█████░███  ░███    ░███  ░███   ░░███ ░███    ░███
 ░███████     ░███████████  ░██████████   ░███░░███ ░███  ░███████████  ░███    ░███ ░███████████
 ░███░░███    ░███░░░░░███  ░███░░░░░███  ░███ ░░░  ░███  ░███░░░░░███  ░███    ░███ ░███░░░░░███
 ░███ ░░███   ░███    ░███  ░███    ░███  ░███      ░███  ░███    ░███  ░███    ███  ░███    ░███
 █████ ░░████ █████   █████ █████   █████ █████     █████ █████   █████ ██████████   █████   █████
░░░░░   ░░░░ ░░░░░   ░░░░░ ░░░░░   ░░░░░ ░░░░░     ░░░░░ ░░░░░   ░░░░░ ░░░░░░░░░░   ░░░░░   ░░░░░
------------------------------------------------------------------------------------------------------
Karmada is installed successfully.

Register Kubernetes cluster to Karmada control plane.

Register cluster with 'Push' mode

Step 1: Use "kubectl karmada join" command to register the cluster to Karmada control plane. --cluster-kubeconfig is kubeconfig of the member cluster.
(In karmada)~# MEMBER_CLUSTER_NAME=$(cat ~/.kube/config  | grep current-context | sed 's/: /\n/g'| sed '1d')
(In karmada)~# kubectl karmada --kubeconfig /home/user/karmada/karmada-apiserver.config  join ${MEMBER_CLUSTER_NAME} --cluster-kubeconfig=$HOME/.kube/config

Step 2: Show members of karmada
(In karmada)~# kubectl --kubeconfig /home/user/karmada/karmada-apiserver.config get clusters


Register cluster with 'Pull' mode

Step 1: Create bootstrap token and generate the 'kubectl karmada register' command which will be used later.
~# kubectl karmada token create --print-register-command --kubeconfig=/home/user/karmada/karmada-apiserver.config
This command will generate a registration command similar to:

kubectl karmada register 172.18.0.5:5443 --token t8xfio.640u9gp9obc72v5d --discovery-token-ca-cert-hash sha256:9cfa542ff48f43793d1816b1dd0a78ad574e349d8f6e005e6e32e8ab528e4244

Step 2: Use the output from Step 1 to register the cluster to the Karmada control plane. 
You need to specify the target member cluster by flag '--kubeconfig'
~# kubectl karmada register 172.18.0.5:5443 --token t8xfio.640u9gp9obc72v5d --discovery-token-ca-cert-hash sha256:9cfa542ff48f43793d1816b1dd0a78ad574e349d8f6e005e6e32e8ab528e4244 --kubeconfig=<path-to-member-cluster-kubeconfig>

Step 3: Show members of Karmada.
~# kubectl karmada --kubeconfig=/home/user/karmada/karmada-apiserver.config get clusters

The kubectl karmada register command has several optional parameters for setting the properties of the member cluster. For more details, run:

~# kubectl karmada register --help

olariuromeo · 2025-01-24T02:15:16Z

in this case the documentation should be modified: https://karmada.io/docs/installation/#install-karmada-in-kind-cluster

Yes, the previous documentation did not provide detailed instructions for operations by non-root users. It would be valuable to update the documentation to include this information. @olariuromeo Are you interested in submitting a PR to update it?

Yes, I will do that, thank you.

zhzhuang-zju · 2025-01-24T02:24:37Z

/kind question
/remove-bug
/assign @olariuromeo

zhzhuang-zju · 2025-01-24T02:27:46Z

/remove-kind bug

zhzhuang-zju · 2025-01-24T08:53:19Z

@olariuromeo Nice work, I saw you pr and it did fixed this issue

olariuromeo added the kind/bug Categorizes issue or PR as related to a bug. label Jan 23, 2025

github-project-automation bot added this to Karmada Overall Backlog Jan 23, 2025

karmada-bot assigned olariuromeo Jan 24, 2025

karmada-bot added the kind/question Indicates an issue that is a support question. label Jan 24, 2025

karmada-bot removed the kind/bug Categorizes issue or PR as related to a bug. label Jan 24, 2025

olariuromeo mentioned this issue Jan 24, 2025

Updated Karmada installation documentation for non-root users karmada-io/website#788

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

kubectl karmada init get error mkdir /etc/karmada: permission denied #6080

kubectl karmada init get error mkdir /etc/karmada: permission denied #6080

olariuromeo commented Jan 23, 2025 •

edited

Loading

olariuromeo commented Jan 24, 2025

RainbowMango commented Jan 24, 2025

zhzhuang-zju commented Jan 24, 2025

olariuromeo commented Jan 24, 2025

zhzhuang-zju commented Jan 24, 2025

olariuromeo commented Jan 24, 2025 •

edited

Loading

olariuromeo commented Jan 24, 2025

zhzhuang-zju commented Jan 24, 2025

zhzhuang-zju commented Jan 24, 2025

zhzhuang-zju commented Jan 24, 2025

kubectl karmada init get error mkdir /etc/karmada: permission denied #6080

kubectl karmada init get error mkdir /etc/karmada: permission denied #6080

Comments

olariuromeo commented Jan 23, 2025 • edited Loading

olariuromeo commented Jan 24, 2025

RainbowMango commented Jan 24, 2025

zhzhuang-zju commented Jan 24, 2025

olariuromeo commented Jan 24, 2025

zhzhuang-zju commented Jan 24, 2025

olariuromeo commented Jan 24, 2025 • edited Loading

olariuromeo commented Jan 24, 2025

zhzhuang-zju commented Jan 24, 2025

zhzhuang-zju commented Jan 24, 2025

zhzhuang-zju commented Jan 24, 2025

olariuromeo commented Jan 23, 2025 •

edited

Loading

olariuromeo commented Jan 24, 2025 •

edited

Loading