Skip to content

Commit

Permalink
Korifi installer for kind clusters
Browse files Browse the repository at this point in the history
fixes #2895

Co-authored-by: Georgi Sabev <[email protected]>
Co-authored-by: Danail Branekov <[email protected]>
  • Loading branch information
danail-branekov and georgethebeatle committed Oct 30, 2023
1 parent 6f457fc commit 5a29281
Show file tree
Hide file tree
Showing 4 changed files with 208 additions and 40 deletions.
97 changes: 59 additions & 38 deletions INSTALL.kind.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,29 +3,25 @@
# Install Korifi on kind

This document integrates our [install instructions](./INSTALL.md) with specific tips to install Korifi locally using [kind](https://kind.sigs.k8s.io/).
In order to install korifi on kind effortlessly we have prepared an installation job definition that you simply apply to your kind cluster. It will install korifi with reasonable defautls using a local docker registry (also running on your kind cluster).

## Initial setup

Export the following environment variables:

```sh
ROOT_NAMESPACE="cf"
KORIFI_NAMESPACE="korifi-system"
ADMIN_USERNAME="kubernetes-admin"
BASE_DOMAIN="apps-127-0-0-1.nip.io"
```

`apps-127-0-0-1.nip.io` will conveniently resolve to `127.0.0.1` using [nip.io](https://nip.io/), which is exactly what we need.

### Cluster creation
## Cluster creation

In order to access the Korifi API, we'll need to [expose the cluster ingress locally](https://kind.sigs.k8s.io/docs/user/ingress/). To do it, create your kind cluster using a command like this:

```sh
cat <<EOF | kind create cluster --name korifi --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
containerdConfigPatches:
- |-
[plugins."io.containerd.grpc.v1.cri".registry]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."localregistry-docker-registry.default.svc.cluster.local:30050"]
endpoint = ["http://127.0.0.1:30050"]
[plugins."io.containerd.grpc.v1.cri".registry.configs]
[plugins."io.containerd.grpc.v1.cri".registry.configs."127.0.0.1:30050".tls]
insecure_skip_verify = true
nodes:
- role: control-plane
extraPortMappings:
Expand All @@ -35,46 +31,71 @@ nodes:
- containerPort: 443
hostPort: 443
protocol: TCP
- containerPort: 30050
hostPort: 30050
protocol: TCP
EOF
```

### Container registry
## Install Korifi

We recommend you use [DockerHub](https://hub.docker.com/) as your container registry.
- Run the installer job:

## Dependencies
```sh
kubectl apply -f https://github.com/cloudfoundry/korifi/releases/latest/download/install-korifi-kind.yaml
```

Follow the [common instructions](./INSTALL.md#dependencies), with the exception of Metrics Server.
- If you want track the job progress, run:

### Metrics Server
```sh
kubectl -n korifi-installer logs --follow job/install-korifi
```

Make sure you pass the following flags to the Metrics Server container (see [_Configuration_](https://github.com/kubernetes-sigs/metrics-server#configuration)):
- **Optional** After the job is complete you can delete the `korifi-installer` namespace

- `--kubelet-insecure-tls`
- `--kubelet-preferred-address-types=InternalIP`
```sh
kubectl delete namespace korifi-installer
```

## Pre-install configuration
## Test Korifi

No changes here, follow the [common instructions](./INSTALL.md#pre-install-configuration).
For the container registry credentials `Secret`, we recommend you [create an access token](https://hub.docker.com/settings/security?generateToken=true) on DockerHub.
- Target the api:

## Install Korifi
```sh
cf api https://localhost --skip-ssl-validation
```

- Authenticate as the cf admin user:

```sh
cf auth kind-korifi
```

No changes here, follow the [common instructions](./INSTALL.md#install-korifi).
If using DockerHub as recommended above, set the following values:
- Create and target an org and a space

```sh
cf create-org org && cf create-space -o org space && cf target -o org
```

- `kpackImageBuilder.builderRepository`: `index.docker.io/<username>/kpack-builder`;
- `containerRepositoryPrefix`: `index.docker.io/<username>/`;
- Push a buildpack app and access it:

Remember to set `generateIngressCertificates` to `true` if you want to use self-signed TLS certificates.
```sh
make build-dorifi
cf push dorifi -p tests/assets/dorifi
curl -k https://dorifi.apps-127-0-0-1.nip.io
```

If `$KORIFI_NAMESPACE` doesn't exist yet, you can add the `--create-namespace` flag to the `helm` invocation.
- Push a docker app and access it:

## Post-install Configuration
```sh
cf push nginx --docker-image nginxinc/nginx-unprivileged:1.23.2
curl -k https://nginx.apps-127-0-0-1.nip.io
```

Yon can skip this section.
## Cleanup

## Test Korifi
When you no longer need korifi you can delete the whole kind cluster via:

No changes here, follow the [common instructions](./INSTALL.md#test-korifi).
When running `cf login`, make sure you select the entry associated to your kind cluster (`kind-korifi` in our case).
```sh
kind delete cluster --name korifi
```
4 changes: 2 additions & 2 deletions INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ This document was tested on:

- [EKS](https://aws.amazon.com/eks/), using AWS' [Elastic Container Registry (ECR)](https://aws.amazon.com/ecr/) (see [_Install Korifi on EKS_](./INSTALL.EKS.md));
- [GKE](https://cloud.google.com/kubernetes-engine), using GCP's [Artifact Registry](https://cloud.google.com/artifact-registry);
- [kind](https://kind.sigs.k8s.io/), using [DockerHub](https://hub.docker.com/) (see [_Install Korifi on kind_](./INSTALL.kind.md)).
- [kind](https://kind.sigs.k8s.io/): see [_Install Korifi on kind_](./INSTALL.kind.md).

## Initial setup

Expand Down Expand Up @@ -222,4 +222,4 @@ cf create-space -o org1 space1
cf target -o org1
cd <directory of a test cf app>
cf push test-app
```
```
34 changes: 34 additions & 0 deletions scripts/installer/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
FROM ubuntu

ARG HELM_CHART_SOURCE

COPY scripts/install-dependencies.sh scripts/install-dependencies.sh
COPY scripts/create-new-user.sh scripts/create-new-user.sh
COPY tests/dependencies tests/dependencies
COPY tests/vendor tests/vendor
COPY ${HELM_CHART_SOURCE} helm/korifi

RUN apt-get update \
&& apt-get install --yes \
--no-install-recommends \
apt-transport-https \
ca-certificates \
conntrack \
gnupg2 \
curl \
git \
sudo \
&& rm -rf /var/lib/apt/lists/*

# helm
RUN echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/helm.gpg] https://baltocdn.com/helm/stable/debian/ all main" >/etc/apt/sources.list.d/helm-stable-debian.list \
&& curl -fsSL https://baltocdn.com/helm/signing.asc | gpg --dearmor >/usr/share/keyrings/helm.gpg

RUN apt-get update \
&& apt-get install --yes \
helm \
&& rm -rf /var/lib/apt/lists/*

# kubectl
RUN curl -fsSLo /usr/bin/kubectl "https://dl.k8s.io/release/$(curl -fsSL https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" \
&& chmod +x /usr/bin/kubectl
113 changes: 113 additions & 0 deletions scripts/installer/install-korifi-kind.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
---
apiVersion: v1
kind: Namespace
metadata:
name: korifi-installer

---
apiVersion: v1
kind: Namespace
metadata:
name: cf
labels:
pod-security.kubernetes.io/audit: restricted
pod-security.kubernetes.io/enforce: restricted

---
apiVersion: v1
kind: Namespace
metadata:
name: korifi
labels:
pod-security.kubernetes.io/audit: restricted
pod-security.kubernetes.io/enforce: restricted

---
apiVersion: v1
data:
.dockerconfigjson: eyJhdXRocyI6eyJsb2NhbHJlZ2lzdHJ5LWRvY2tlci1yZWdpc3RyeS5kZWZhdWx0LnN2Yy5jbHVzdGVyLmxvY2FsOjMwMDUwIjp7InVzZXJuYW1lIjoidXNlciIsInBhc3N3b3JkIjoicGFzc3dvcmQiLCJhdXRoIjoiZFhObGNqcHdZWE56ZDI5eVpBPT0ifX19
kind: Secret
metadata:
name: image-registry-credentials
namespace: cf
type: kubernetes.io/dockerconfigjson

---
apiVersion: v1
kind: ServiceAccount
metadata:
name: korifi-installer
namespace: korifi-installer

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: korifi-installer
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: korifi-installer
namespace: korifi-installer

---
apiVersion: batch/v1
kind: Job
metadata:
name: install-korifi
namespace: korifi-installer
spec:
template:
metadata:
name: install-korifi
spec:
serviceAccountName: korifi-installer
restartPolicy: Never
containers:
- name: install-korifi
image: cloudfoundry/korifi-installer
command:
- bash
- -c
- |
scripts/install-dependencies.sh --insecure-tls-metrics-server
helm repo add twuni https://helm.twun.io
# the htpasswd value below is username: user, password: password encoded using `htpasswd` binary
# e.g. `docker run --entrypoint htpasswd httpd:2 -Bbn user password`
#
helm upgrade --install localregistry twuni/docker-registry \
--namespace default \
--set service.type=NodePort,service.nodePort=30050,service.port=30050 \
--set persistence.enabled=true \
--set persistence.deleteEnabled=true \
--set secrets.htpasswd='user:$2y$05$Ue5dboOfmqk6Say31Sin9uVbHWTl8J1Sgq9QyAEmFQRnq1TPfP1n2'
registry_status_code=""
while [[ "$registry_status_code" != "200" ]]; do
echo Waiting for the local docker registry to start...
registry_status_code=$(curl -o /dev/null -w "%{http_code}" --user user:password http://localregistry-docker-registry.default.svc.cluster.local:30050/v2/_catalog 2>/dev/null)
sleep 1
done
helm upgrade --install korifi helm/korifi \
--namespace korifi \
--set=adminUserName="kubernetes-admin" \
--set=defaultAppDomainName="apps-127-0-0-1.nip.io" \
--set=generateIngressCertificates="true" \
--set=logLevel="debug" \
--set=debug="false" \
--set=stagingRequirements.buildCacheMB="1024" \
--set=api.apiServer.url="localhost" \
--set=controllers.taskTTL="5s" \
--set=jobTaskRunner.jobTTL="5s" \
--set=containerRepositoryPrefix="localregistry-docker-registry.default.svc.cluster.local:30050/" \
--set=kpackImageBuilder.clusterStackBuildImage="paketobuildpacks/build-jammy-base" \
--set=kpackImageBuilder.clusterStackRunImage="paketobuildpacks/run-jammy-base" \
--set=kpackImageBuilder.builderRepository="localregistry-docker-registry.default.svc.cluster.local:30050/kpack-builder" \
--wait
kubectl wait --for=condition=ready clusterbuilder --all=true --timeout=15m

0 comments on commit 5a29281

Please sign in to comment.