Skip to content

Latest commit

 

History

History
336 lines (281 loc) · 17.5 KB

README.md

File metadata and controls

336 lines (281 loc) · 17.5 KB
Raw

Build E2E Docker GitHub release (latest SemVer) Dependabot

fluentd-sidecar-injector

fluentd-sidecar-injector is a webhook server for kubernetes admission webhook. This server inject fluentd or fluent-bit container as sidecar for specified Pod using mutation webhook. The feature is

  • Automatically sidecar injection
  • You can control injection using Pod's annotations
  • You can change fluentd or fluent-bit docker image to be injected

Install

With cert-manager (Recommended)

This controller creates a webhook server, and you can manage the certificate of the webhook with cert-manager. Make sure you have already installed cert-manager before you install.

You can install this controller and webhook server using helm. By default, the helm chart installs this controller with --use-cert-manager.

$ helm repo add h3poteto-stable https://h3poteto.github.io/charts/stable
$ helm install my-injector --namespace kube-system h3poteto-stable/fluentd-sidecar-injector

Please refer helm repository for parameters.

Without cert-manager

If you don't use cert-manager, please choose this way. But the certificate will not be renewed when it expires. So please re-install this controller when it expires.

$ helm repo add h3poteto-stable https://h3poteto.github.io/charts/stable
$ helm install my-injector --namespace kube-system h3poteto-stable/fluentd-sidecar-injector --set useCertManager=false

Confirm

After install it, custom resources and controller will be installed.

$ kubectl get sidecarinjectors -n kube-system
NAME                  AGE
my-injector-fluentd   1m56s

$ kubectl get pods -n kube-system -l operator.h3poteto.dev=control-plane
NAME                                   READY   STATUS    RESTARTS   AGE
my-injector-manager-6d7f6bcd55-z5jcv   1/1     Running   0          2m17s

And it creates admission webhook for the sidecar injector.

$ kubectl get mutatingwebhookconfigurations
NAME                                           WEBHOOKS   AGE
sidecar-injector-webhook-my-injector-fluentd   1          5m15s

$ kubectl get svc -n kube-system -l sidecarinjectors.operator.h3poteto.dev=webhook-service
NAME                                   TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
sidecar-injector-my-injector-fluentd   ClusterIP   100.69.147.98   <none>        443/TCP   4m2s

$ kubectl get pods -n kube-system -l sidecarinjectors.operator.h3poteto.dev=webhook-pod
NAME                                           READY   STATUS    RESTARTS   AGE
my-injector-fluentd-handler-5969df9695-ftklp   1/1     Running   0          4m51s
my-injector-fluentd-handler-5969df9695-x5n5r   1/1     Running   0          4m51s

Usage

After you install this webhook server, fluentd sidecar containers are automatically injected, if you specify the annotation fluentd-sidecar-injector.h3poteto.dev/injection: 'enabled' to the pods.

For example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-test
  labels:
    app: nginx-test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-test
  template:
    metadata:
      annotations:
        fluentd-sidecar-injector.h3poteto.dev/injection: 'enabled'
        fluentd-sidecar-injector.h3poteto.dev/application-log-dir: '/var/log/nginx'
        fluentd-sidecar-injector.h3poteto.dev/collector: 'fluentd'
      labels:
        app: nginx-test
    spec:
      containers:
        - name: nginx
          image: nginx:latest

FluentD is injected for this Pod.

$ kubectl get pod
NAME                          READY   STATUS    RESTARTS   AGE
nginx-test-6cbf4485f8-kq8ws   2/2     Running   0          9s
$ kubectl describe pod nginx-test-6cbf4485f8-kq8ws
Name:           nginx-test-6cbf4485f8-kq8ws
Namespace:      default
Containers:
  nginx:
    Container ID:   docker://ce74393381205786668a1fe2a4bc83ba058d380714b8a7ddca23966c8c7f0eb0
    Image:          nginx:latest
    Image ID:       docker-pullable://nginx@sha256:ad5552c786f128e389a0263104ae39f3d3c7895579d45ae716f528185b36bc6f
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Fri, 14 Feb 2020 13:49:21 +0900
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/log/nginx from fluentd-sidecar-injector-logs (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-8rcns (ro)
  fluentd-sidecar:
    Container ID:   docker://49503c3836fa5ebc40c55db3717f16f21fbdbfaae8859a8ed8a366d04a2b6d9b
    Image:          ghcr.io/h3poteto/fluentd-forward:latest
    Image ID:       docker-pullable://ghcr.io/h3poteto/fluentd-forward@sha256:5d93af333ad9fefbfcb8013d20834fd89c2bbd3fe8b9b9bfa620ded29d7b3205
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Fri, 14 Feb 2020 13:49:23 +0900
    Ready:          True
    Restart Count:  0
    Limits:
      memory:  1000Mi
    Requests:
      cpu:     100m
      memory:  200Mi
    Environment:
      AGGREGATOR_HOST:      127.0.0.1
      APPLICATION_LOG_DIR:  /var/log/nginx
      TAG_PREFIX:           prod
      TIME_KEY:             time
    Mounts:
      /var/log/nginx from fluentd-sidecar-injector-logs (rw)

Custom fluent.conf

If you need to use your own fluent.conf, use config-volume option. The following yaml has fluent-conf configmap. It will be mounted on /fluentd/etc/fluent/fluent.conf.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-test
  labels:
    app: nginx-test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-test
  template:
    metadata:
      annotations:
        fluentd-sidecar-injector.h3poteto.dev/injection: 'enabled'
        fluentd-sidecar-injector.h3poteto.dev/collector: 'fluentd'
        fluentd-sidecar-injector.h3poteto.dev/docker-image: 'fluent/fluentd:latest'
        fluentd-sidecar-injector.h3poteto.dev/application-log-dir: '/var/log/nginx'
        fluentd-sidecar-injector.h3poteto.dev/aggregator-host: 'fluentd.example.com'
        fluentd-sidecar-injector.h3poteto.dev/config-volume: 'fluent-conf'
      labels:
        app: nginx-test
    spec:
      containers:
        - name: nginx
          image: nginx:latest
    volumes:
      - name: fluent-conf
        configMap:
          name: fluent-conf
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: fluent-conf
  labels:
    app: fluent-conf
data:
  fluent.conf: |-
    <source>
      @type tail
      path "#{ENV['APPLICATION_LOG_DIR']}/*.access.log"
      pos_file /var/tmp/application.log.pos
      tag "app.*"
      <parse>
        @type ltsv
      </parse>
    </source>

    <filter app.*>
      @type record_transformer
      <record>
        hostname "#{Socket.gethostname}"
      </record>
    </filter>

    <match app.*>
      @type forward

      <server>
        host "#{ENV['AGGREGATOR_HOST']}"
        port "#{ENV['AGGREGATOR_PORT']} || 24224"
      </server>
    </match>

Annotations

Please specify these annotations to your pods like this.

Name Required Default
fluentd-sidecar-injector.h3poteto.dev/injection optional ""
fluentd-sidecar-injector.h3poteto.dev/docker-image optional ghcr.io/h3poteto/fluentd-forward:latest
fluentd-sidecar-injector.h3poteto.dev/collector optional fluentd
fluentd-sidecar-injector.h3poteto.dev/aggregator-host required ""
fluentd-sidecar-injector.h3poteto.dev/aggregator-port optional 24224
fluentd-sidecar-injector.h3poteto.dev/application-log-dir required ""
fluentd-sidecar-injector.h3poteto.dev/tag-prefix optional ""
fluentd-sidecar-injector.h3poteto.dev/custom-env optional ""
fluentd-sidecar-injector.h3poteto.dev/expose-port optional ""
fluentd-sidecar-injector.h3poteto.dev/config-volume optional ""
fluentd-sidecar-injector.h3poteto.dev/memory-request optional 200Mi
fluentd-sidecar-injector.h3poteto.dev/memory-limit optional 1000Mi
fluentd-sidecar-injector.h3poteto.dev/cpu-request optional 100m
fluentd-sidecar-injector.h3poteto.dev/cpu-limit optional ""

These annotations are used when collector is fluentd.

Name Required Default
fluentd-sidecar-injector.h3poteto.dev/send-timeout optional 60s
fluentd-sidecar-injector.h3poteto.dev/recover-wait optional 10s
fluentd-sidecar-injector.h3poteto.dev/hard-timeout optional 120s
fluentd-sidecar-injector.h3poteto.dev/time-key optional time
fluentd-sidecar-injector.h3poteto.dev/time-format optional %Y-%m-%dT%H:%M:%S%z
fluentd-sidecar-injector.h3poteto.dev/log-format optional json

These annotations are used when collector is fluent-bit.

Name Required Default
fluentd-sidecar-injector.h3poteto.dev/refresh-interval optional 60
fluentd-sidecar-injector.h3poteto.dev/rotate-wait optional 5

Fixed environment variables

The following values ​​will be set for each fluentd-sidecar. You can use this value in your fluent.conf with config-volume option.

Name Default
NODE_NAME spec.nodeName
POD_NAME metadata.name
POD_NAMESPACE metadata.namespace
POD_IP status.podIP
POD_SERVICE_ACCOUNT spec.serviceAccountName
CPU_RESOURCE requests.cpu
CPU_LIMIT limits.cpu
MEM_RESOURCE requests.memory
MEM_LIMIT limits.memory

You can find out more about the values on The Downward API.

Development

Please prepare a Kubernetes cluster to install this, and export KUBECONFIG.

$ export KUBECONFIG=$HOME/.kube/config

At first, build docker and push it.

$ docker build -t my-docker-registry/fluentd-sidecar-injector:experimental .
$ docker push my-docker-registry/fluentd-sidecar-injector:experimental

Next, install CRDs.

$ make install

Next, please run controller in local.

$ export WEBHOOK_CONTAINER_IMAGE=my-docker-registry/fluentd-sidecar-injector:experimental
$ export POD_NAMESPACE=my-namespace
$ make run

License

The package is available as open source under the terms of the MIT License.