Create KeptnTask to send bizevents not entity-focused events

[agardnerIT]

Existing repo sends events that attach to entities (SERVICES, APPLICATIONS and SYNTHETIC_TEST).

This creates bloat so rather just move to biz events.

openfeature/argocd/notifications-cm.yaml

Line 25 in cdddbae

service.webhook.dt_application: |

[agardnerIT]

OAuth token flow:

curl --request POST 'https://sso.dynatrace.com/sso/oauth2/token' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'grant_type=client_credentials' \
--data-urlencode 'client_id={your-client-id}' \
--data-urlencode 'client_secret={your-client-secret}' \
--data-urlencode 'scope=app-engine:apps:run storage:logs:read'

A bizevent is ingested via (with you being in control of the body - a cloudevent is shown below):

curl -X POST "https://abc12345.live.dynatrace.com/api/v2/bizevents/ingest" \
-H "accept: application/json; charset=utf-8" \
-H "Content-Type: application/cloudevent+json" \
-d "{\"specversion\":\"1.0\",\"id\":\"1\",\"source\":\"custom.source\",\"type\":\"com.mycompany.shop.checkout\",\"dtcontext\":\"dt.session_id=\\\"234234234324235345345\\\",dt.entity.rum_application=\\\"APPLICATION-53453458340758\\\",host.name=\\\"123.123.123.123\\\"\",\"dataschema\":\"http://dynatrace.com/schema/bizevents/generic/1.0\",\"traceparent\":\"00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-00\",\"data\":{\"id\":\"OR-838475\",\"paymentType\":\"paypal\",\"plannedDeliveryDate\":\"01.01.2021\",\"total\":234}}"

1. Create an OAuth client
   

2. Store creds as secret

apiVersion: v1
kind: Secret
metadata:
  name: argocd-notifications-secret
stringData:
  dt-env: "https://abc12345.live.dynatrace.com"
  dt-bizevents-write-client-id: "*****"
  dt-bizevents-write-client-secret: "********"
  dt-bizevents-write-account-urn: "******"
type: Opaque

Apply secret: kubectl -n argocd apply -f secret.yaml

Open question: Can / how does argo handle "workflows" for notifications.
For example, we need to make a call to DT, store the returned OAuth token, then POST the payload

TODO: Research

The argo notification payload will look like this:

data:
  service.webhook.dt_bizevent: |
    url: "$dt-env/api/v2/events/ingest"
    headers:
    - name: "accept"
      value: "application/json; charset=utf-8"
    - name: "Content-Type"
      value: "application/cloudevent+json"
...
  template.outofsync: |
    webhook:
      dt_bizevent: |
        method: POST
        body: |
          {
            "specversion": "1.0",
            "source": "argocd",
            "type": "com.dynatrace.outofsync",
            "dataschema": "http://dynatrace.com/schema/bizevents/generic/1.0",
            "data": {
              "app": "{{ .app.metadata.name }}",
              "action": "out-of-sync",
              "revision": "{{.app.status.sync.revision}}",
              "dt.event.deployment.name": "{{ .app.metadata.name }}",
              "dt.event.description": "{{ .app.metadata.name }} out of sync",
              "dt.event.is_rootcause_relevant": false,
              "history": "GITHUB_REPO_URL/commit/{{.app.status.sync.revision}}"
            }
          }
...
  template.insync: |
    webhook:
      dt_bizevent: |
        method: POST
        body: |
          {
            "specversion": "1.0",
            "source": "argocd",
            "type": "com.dynatrace.insync",
            "dataschema": "http://dynatrace.com/schema/bizevents/generic/1.0",
            "data": {
              "app": "{{ .app.metadata.name }}",
              "action": "in-sync",
              "revision": "{{.app.status.sync.revision}}",
              "dt.event.deployment.name": "{{ .app.metadata.name }}",
              "dt.event.description": "{{ .app.metadata.name }} out of sync",
              "dt.event.is_rootcause_relevant": true,
              "history": "GITHUB_REPO_URL/commit/{{.app.status.sync.revision}}"
            }
          }

Edit: This may not be necessary. Seems we can pull from the event log when DT is monitoring the cluster:

k8s.namespace.name = argocd
dt.kubernetes.event.involved_object.kind = Application
dt.kubernetes.event.involved_object.name = fibonacci     OR
dt.kubernetes.event.involved_object.name = fib3r            OR
content = Updated sync status: Synced                             OR
content = Updated sync status: OutOfSync -> Synced