documentation for running event-gateway on k8s (#498)

* adding more changes to README * interim commit for more README updates * more updates to README * adding in fedora section * moving README one level higher * adding in ports description * breaking up lines a bit * fixing the event url * adding in dev flag for running * adding example function register * updating data * updating for typo * cleaning up k8s content * updating based on feedback * interim commit for k8s * cleaning up helm files * moving to template variables * initial commit minikube readme * updating README * updating number * updating formatting * more formatting * updating based upon feedback * updating with trigger * updating the TOC * updating based upon further feedback * updating with more feedback * updating to query all event types * updating the README for the contrib level folder * updating to builleted items * removing extraneous reference * updating README with feedback * cleaning up doublenote
serverless · Aug 16, 2018 · 02e8f5f · 02e8f5f
1 parent 8b31577
commit 02e8f5f
Showing 10 changed files with 451 additions and 2,634 deletions.
diff --git a/README.md b/README.md
@@ -73,7 +73,7 @@ If you don't want to run the Event Gateway yourself, you can use the hosted vers
 There is a [official Docker image](https://hub.docker.com/r/serverless/event-gateway/).
 
 ```bash
-docker run -p 4000:4000 -p 4001:4001 serverless/event-gateway -dev
+docker run -p 4000:4000 -p 4001:4001 serverless/event-gateway --dev
 ```
 
 ### Binary
@@ -89,9 +89,14 @@ On Windows download [binary](https://github.com/serverless/event-gateway/release
 Then run the binary in development mode with:
 
 ```bash
-$ event-gateway -dev
+$ event-gateway --dev
 ```
 
+### Kubernetes
+
+The repo contains `helm` [charts](contrib/helm/README.md) for a quick deploy to an existing cluster using native nginx Ingress. To deploy
+a development cluster you can follow the [minikube](contrib/helm/MINIKUBE.md) instructions.
+
 ---
 
 If you want more detailed information on running and developing with the Event Gateway,

diff --git a/contrib/README.md b/contrib/README.md
@@ -0,0 +1,6 @@
+# Event Gateway Contributions 
+
+This folder contains services and contributions which complement the Event Gateway.
+
++ Install Event Gateway on Kubernetes using [helm](helm/README.md)
++ Provision the Event Gateway on AWS ECS Fargate using [terraform](terraform/README.md)
diff --git a/contrib/helm/MINIKUBE.md b/contrib/helm/MINIKUBE.md
@@ -0,0 +1,66 @@
+# Install Minikube for Local Development 
+
+To develop and deploy the `event-gateway` and all related elements locally, the easiest method includes using 
+the [minikube](https://github.com/kubernetes/minikube) toolset. To get started, set up your local cluster with the 
+following instructions...
+
+## Contents
+1. [Fedora/RHEL/CentOS](#fedora-rhel-centos)
+1. [Debian/Ubuntu](#debian-ubuntu)
+1. [MacOS](#macos)
+
+### Fedora/RHEL/CentOS
++ Install the prerequisite packages:
+  ```bash
+  sudo dnf install kubernetes libvirt-daemon-kvm qemu-kvm nodejs docker
+  ```
+
++ Ensure your user is added to the `libvirt` group for VM access. You can verify with `getent group libvirt` once done.
+  ```bash
+  sudo usermod -a -G libvirt $(whoami)
+  ```
+
++ Next, add the `libvirt` group to your current user grouplist. Verify by running `id` once done.
+  ```bash
+  newgrp libvirt
+  ```
+
++ Add the [docker-machine](https://github.com/docker/machine) binary to your system 
+  ```bash
+  curl -L https://github.com/docker/machine/releases/download/v0.15.0/docker-machine-$(uname -s)-$(uname -m) >/tmp/docker-machine && \
+  chmod +x /tmp/docker-machine && \
+  sudo cp /tmp/docker-machine /usr/local/bin/docker-machine
+  ```
+
++ Add the CentOS `docker-machine` kvm driver. It's ok if you're not using CentOS as the driver should **still work**&trade;
+  ```bash
+  sudo curl -L https://github.com/dhiltgen/docker-machine-kvm/releases/download/v0.10.0/docker-machine-driver-kvm-centos7 > /tmp/docker-machine-driver-kvm && \
+  sudo chmod +x /tmp/docker-machine-driver-kvm && \
+  sudo mv /tmp/docker-machine-driver-kvm /usr/local/bin/docker-machine-driver-kvm
+  ```
+
++ Download the minikube instance for your system 
+  ```bash
+  curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 && \
+  sudo chmod +x minikube && \
+  sudo mv minikube /usr/local/bin/
+  ```
+
++ Finally, start up your minikube service! **NOTE:** the instructions recommend using `kvm2` but please use the version that matches your system install
+  ```bash
+  minikube start --vm-driver kvm2
+  ```
+
++ Once everything is running you should be able to view your running cluster status
+  ```bash
+  minikube status
+  minikube service kubernetes-dashboard --namespace kube-system
+  ```
+
+### Debian/Ubuntu
+
+PENDING
+
+### MacOS
+
+PENDING
diff --git a/contrib/helm/README.md b/contrib/helm/README.md
@@ -1,36 +1,284 @@
-# Event Gateway Helm chart
+# Event Gateway on Kubernetes 
 
-This chart deploys the Event Gateway with etcd onto a Kubernetes cluster.
+This chart deploys the Event Gateway with etcd onto a Kubernetes cluster. Please note, the default instructions expect
+an existing kubernetes cluster that supports ingress, such as GKE. If your environment doesn't have ingress support
+set up, please follow the [minikube](MINIKUBE.md) instructions to set this up for your development environment.  
 
-## Installation
+## Contents
 
-Make sure you have helm installed on you machine and run `helm init` on your K8s cluster.
+1. [Quickstart](#quickstart)
+1. [Examples](#examples)
+    1. [Register a function](#register-a-function)
+    1. [Query all function](#query-all-function)
+    1. [Register an event type](#register-an-event-type)
+    1. [Query all event types](#query-all-event-types)
+    1. [Register a subscription](#register-a-subscription)
+    1. [Query all subscriptions](#query-all-subscriptions)
+    1. [Trigger an event](#trigger-an-event)
+1. [Configuration](#configuration)
+1. [Cleanup](#cleanup)
 
-From the `event-gateway/contrib/helm` folder:
+## Quickstart
 
-First, install etcd operator:
+Make sure you have helm installed on your machine and run `helm init` on your k8s cluster. This will set up the
+`helm` and `tiller` functions required for easy deployment of config files to your cluster. You can follow
+instructions [here](https://docs.helm.sh/using_helm/#quickstart) if you have not set this up previously.
+
+Once installed, navigate to the `event-gateway/contrib/helm` folder and install the following components:
+
+**etcd-operator**
+```bash
+helm install stable/etcd-operator --name ego [--namespace <namespace>]
 ```
-helm install stable/etcd-operator --name ego
+
+**event-gateway**
+```bash
+helm install event-gateway --name eg [--namespace <namespace>]
 ```
 
-Then, install the Event Gateway:
+This will install each of the `etcd-operator` and `event-gateway` into the `default` namespace in kubernetes. Please note,
+this namespace has no bearing on your Event Gateway `spaces` as outlined in the [docs](https://github.com/serverless/event-gateway/blob/master/README.md). If you'd like
+to install `etcd-operator` and `event-gateway` in another namespace, add the `--namespace <namespace>` option to
+ both `helm install` commands above.
+
+Next we'll need to collect the Event Gateway IP to use on the CLI. Since the Ingress was configured to receive all 
+connections from the `eventgateway.minikube` host, you'll need to pass this as a header value to the request.
+
+   ```bash
+   export EVENT_GATEWAY_URL=$(kubectl get ingress event-gateway-ingress -o jsonpath={.status.loadBalancer.ingress[0].ip})
+   curl --request GET \
+        --url http://{EVENT_GATEWAY_URL}/v1/metrics \
+        --header 'content-type: application/json' \
+        --header 'host: eventgateway.minikube'
+   ```
+
+With your environment set up, you can now jump to the [examples](#examples) section to put your `event-gateway` to use!
+
+## Examples
+
+Once you've set the `EVENT_GATEWAY_URL` environment variable, you're set to start interacting with the `event-gateway`! 
+
+**NOTE:** the events and configuration API ports are abstracted away from us via the kubernetes Ingress. The path-based 
+routing will ensure the request goes to the proper service managed by the cluster. 
+
+**DOUBLENOTE:** the examples below all assume the `default` namespace for the `event-gateway`. If you've updated or changed
+this on your end, please don't forget to update the queries accordingly.
+
+#### Register a function
+
+Define the function registration payload, using **AWS** as an example:
+
+```bash
+cat > function.json <<EOF
+{
+    "functionId": "echo",
+    "type": "awslambda",
+    "provider": {
+        "arn": "arn:aws:lambda:us-east-1:123456789012:function:event-gateway-tests-dev-echo",
+        "region": "us-east-1",
+        "awsAccessKeyID": "AAAAAAAAAAAAAAAAAAAA",
+        "awsSecretAccessKey": "AAAAaBcDeFgHiJqLmNoPqRsTuVwXyz0123456789"
+    }
+}
+EOF
 ```
-helm install event-gateway --name eg
+
+Then call the registration endpoint with your json payload:
+
+```bash
+curl --request POST \
+  --url http://${EVENT_GATEWAY_URL}/v1/spaces/default/functions \
+  --header 'content-type: application/json' \
+  --header 'host: eventgateway.minikube' \
+  --data @function.json
 ```
 
-Note: to deploy the stack to a namespace other than the default, add `--namespace` option to both `helm install` commands.
+And the corresponsing reply (if successful) should read something like the following:
 
-To get the Event Gateway load balancer IP:
+```bash
+{
+	"space": "default",
+	"functionId": "echo",
+	"type": "awslambda",
+	"provider": {
+		"arn": "arn:aws:lambda:us-east-1:123456789012:function:event-gateway-tests-dev-echo",
+		"region": "us-east-1",
+		"awsAccessKeyId": "AAAAAAAAAAAAAAAAAAAA",
+		"awsSecretAccessKey": "AAAAaBcDeFgHiJkLmNoPqRsTuVwXyZ0123456789"
+	}
+}
 ```
-kubectl get svc
+
+**NOTE:** if you try to overwrite an existing function, you will receive an error! To replace an existing function
+you will have to delete it first, then register the function once more. For example, trying to re-register the `echo`
+function will yield:
+
+```bash
+curl --request POST \
+   --url http://${EVENT_GATEWAY_URL}/v1/spaces/default/functions \
+   --header 'content-type: application/json' \
+   --header 'host: eventgateway.minikube' \
+   --data @function.json
+
+{
+    "errors": [{
+        "message": "Function \"echo\" already registered."
+    }]
+}
+```
+
+#### Query all functions
+
+To check for registered functions, query the `config` API with the `GET` request:
+
+```bash
+curl --request GET \
+  --url http://${EVENT_GATEWAY_URL}/v1/spaces/default/functions \
+  --header 'content-type: application/json' \
+  --header 'host: eventgateway.minikube'  | jq
+```
+
+You should see the functions list return your defined set of functions across all vendors.
+
+```bash
+{
+  "functions": [
+    {
+      "space": "default",
+      "functionId": "echo",
+      "type": "awslambda",
+      "provider": {
+        "arn": "arn:aws:lambda:us-east-1:123456789012:function:event-gateway-tests-dev-echo",
+        "region": "us-east-1",
+        "awsAccessKeyId": "AAAAAAAAAAAAAAAAAAAA",
+        "awsSecretAccessKey": "AAAAaBcDeFgHiJkLmNoPqRsTuVwXyZ0123456789"
+      }
+    }
+  ]
+}
 ```
 
-To delete the Event Gatway and etcd:
+#### Register an event type
+
+To register an event, make sure to `POST` the event name to the `event-gateway`.
+
+```bash
+curl --request POST \
+  --url http://${EVENT_GATEWAY_URL}/v1/spaces/default/eventtypes \
+  --header 'content-type: application/json' \
+  --header 'host: eventgateway.minikube' \
+  --data '{ "name": "eventgateway.function.invoked" }'
 ```
-helm delete eg
-helm delete ego
+
+The reply should look something like the following:
+
+```bash
+{ 
+  "space": "default",
+  "name": "eventgateway.function.invoked"
+}
 ```
 
+#### Query all event types
+
+```bash
+curl --request GET \
+  --url http://${EVENT_GATEWAY_URL}/v1/spaces/default/eventtypes \
+  --header 'content-type: application/json' \
+  --header 'host: eventgateway.minikube'
+```
+
+Your registered events reply should look as follows:
+
+```bash
+{
+  "eventTypes": [
+    {
+      "space": "default",
+      "name": "eventgateway.function.invoked"
+    }
+  ]
+}
+```
+
+#### Register a subscription
+
+To register subscriptions to one of your registered event types, make sure to specify the `eventType` in 
+the JSON POST payload.
+
+```bash
+curl --request POST \
+  --url http://${EVENT_GATEWAY_URL}/v1/spaces/default/subscriptions \
+  --header 'content-type: application/json' \
+  --header 'host: eventgateway.minikube' \
+  --data '{
+    "type": "async",
+    "eventType": "eventgateway.function.invoked",
+    "functionId": "echo",
+    "path": "/",
+    "method": "POST"
+}'
+```
+
+Your reply payload should include the `subscriptionId` for your new subscription:
+
+```bash
+{
+  "space": "default",
+  "subscriptionId": "YXN5bmMsZXZlbnRnYXRld2F5LmZ1bmN0aW9uLmludm9rZWQsZWNobywlMkYsUE9TVA",
+  "type": "async",
+  "eventType": "eventgateway.function.invoked",
+  "functionId": "echo",
+  "path": "/",
+  "method": "POST"
+}
+```
+
+#### Query all subscriptions
+
+To list our your current subscrptions, you can do the following:
+
+```bash
+curl --request GET \
+  --url http://${EVENT_GATEWAY_URL}/v1/spaces/default/subscriptions \
+  --header 'content-type: application/json' \
+  --header 'host: eventgateway.minikube'
+```
+
+The output should list each of the registered subscriptions:
+
+```bash
+{
+  "subscriptions": [
+    {
+      "space": "default",
+      "subscriptionId": "YXN5bmMsZXZlbnRnYXRld2F5LmZ1bmN0aW9uLmludm9rZWQsZWNobywlMkYsUE9TVA",
+      "type": "async",
+      "eventType": "eventgateway.function.invoked",
+      "functionId": "echo",
+      "path": "/",
+      "method": "POST"
+    }
+  ]
+}
+```
+
+#### Trigger an event
+
+In order to trigger a registered function, call the `event-gateway` URL with the proper `functionId`. Following
+our example from earlier, we've registered a `GET` function with `functionId` set to `echo`. To trigger this function,
+we would:
+
+```bash
+curl --request GET \
+  --url http://${EVENT_GATEWAY_URL}/echo \
+  --header 'content-type: application/json' \
+  --header 'host: eventgateway.minikube'
+```
+
+**NOTE**: as mentioned earlier, the `events` service is handled by the path-routing service of the kubernetes Ingress. Any path
+that's prepended with `/v1` will ultimately route to the `config` service, while other paths default to the `events` service. 
+
 ## Configuration
 
 | Parameter                   | Description                                  | Default                    |
@@ -46,8 +294,8 @@ helm delete ego
 | `resources.limits.memory`   | Memory resource limits                       | `256Mi`                    |
 | `resources.requests.cpu`    | CPU resource requests                        | `200m`                     |
 | `resources.requests.memory` | Memory resource requests                     | `256Mi`                    |
-| `command`                   | Options to pass to `event-gateway` command   | `[-db-hosts=eg-etcd-cluster-client:2379, -log-level=debug]`|
-| `etcd_cluster_name`         | Name of the etcd cluster. Passed to the `-db-hosts` option as `<etcd-cluster-name>-client`  | `eg-etcd-cluster`|
+| `command`                   | Options to pass to `event-gateway` command   | `[--db-hosts=eg-etcd-cluster-client:2379, --log-level=debug]`|
+| `etcd_cluster_name`         | Name of the etcd cluster. Passed to the `--db-hosts` option as `<etcd-cluster-name>-client`  | `eg-etcd-cluster`|
 
 The service annotations can be used to set any annotations required by your platform, for example, if
 you update your values.yml with:
@@ -70,3 +318,12 @@ metadata:
     service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0
     foo: bar
 ```
+
+## Cleanup
+
+When you'd like to clean up the deployments, it's easy to remove services using helm: 
+
+```bash
+helm delete --purge eg
+helm delete --purge ego
+```
diff --git a/contrib/helm/event-gateway/templates/etcd-cluster.yaml b/contrib/helm/event-gateway/templates/etcd-cluster.yaml
@@ -4,4 +4,4 @@ metadata:
   name: {{ .Values.etcd_cluster_name }}
 spec:
   size: 3
-  version: "3.2.13"
+  version: "3.2.13"
diff --git a/contrib/helm/event-gateway/templates/event-gateway-ingress.yaml b/contrib/helm/event-gateway/templates/event-gateway-ingress.yaml
@@ -0,0 +1,18 @@
+apiVersion: extensions/v1beta1
+kind: Ingress
+metadata:
+  name: event-gateway-ingress
+  annotations:
+spec:
+  rules:
+    - host: eventgateway.minikube
+      http:
+        paths:
+          - path: /v1/
+            backend:
+              serviceName: {{ template "event-gateway.fullname" . }}
+              servicePort: config
+          - path: /
+            backend:
+              serviceName: {{ template "event-gateway.fullname" . }}
+              servicePort: events
diff --git a/contrib/helm/event-gateway/templates/event-gateway.yaml b/contrib/helm/event-gateway/templates/event-gateway.yaml
@@ -24,7 +24,7 @@ spec:
           image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
           imagePullPolicy: {{ .Values.image.pullPolicy }}
           args:
-            - "-db-hosts={{ .Values.etcd_cluster_name }}-client:{{ .Values.etcd_cluster_port }}"
+            - "--db-hosts={{ .Values.etcd_cluster_name }}-client:{{ .Values.etcd_cluster_port }}"
             {{- range .Values.command }}
             - {{ . }}
             {{- end }}
@@ -36,6 +36,7 @@ spec:
             httpGet:
               path: /v1/status
               port: {{ .Values.service.config.port }}
+
 ---
 apiVersion: v1
 kind: Service
@@ -46,13 +47,14 @@ metadata:
     {{ . }}
     {{- end }}
   labels:
-    chart: "{{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}"
+    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
 spec:
-  type: {{ .Values.service.type }}
   ports:
   - name: config
     port: {{ .Values.service.config.port }}
+    targetPort: {{ .Values.service.config.port }}
   - name: events
     port: {{ .Values.service.events.port }}
+    targetPort: {{ .Values.service.events.port }}
   selector:
     app: {{ template "event-gateway.name" . }}
diff --git a/contrib/helm/event-gateway/values.yaml b/contrib/helm/event-gateway/values.yaml
@@ -17,6 +17,6 @@ resources:
   requests:
    cpu: 200m
    memory: 256Mi
-command: ["-log-level=debug"]
+command: ["--log-level=debug"]
 etcd_cluster_name: eg-etcd-cluster
 etcd_cluster_port: 2379
diff --git a/examples/aws-golang-simple-http-endpoint/package-lock.json b/examples/aws-golang-simple-http-endpoint/package-lock.json
diff --git a/examples/aws-golang-simple-http-endpoint/package.json b/examples/aws-golang-simple-http-endpoint/package.json
@@ -8,7 +8,6 @@
     "@serverless/serverless-event-gateway-plugin": "^0.7.3"
   },
   "dependencies": {
-    "@serverless/event-gateway-sdk": "^0.6.0",
-    "serverless": "^1.28.0"
+    "@serverless/event-gateway-sdk": "^0.6.0"
   }
 }