Merge pull request #24 from opendatadiscovery/feature/quicklaunch_on_…

…aws_marketplace

Create helm chart that combines odd-platform and postgres

charts/odd-quicklaunch/Chart.yaml

@@ -0,0 +1,10 @@
+apiVersion: v2
+name: odd-quicklaunch
+description: A Helm chart for deploying odd-platform and its PostgreSQL database
+version: 1.0.13
+appVersion: "1.0"
+
+dependencies:
+  - name: postgresql
+    version: "13.2.9"
+    repository: "https://charts.bitnami.com/bitnami"

charts/odd-quicklaunch/README.md

@@ -0,0 +1,80 @@
+# Helm Chart for AWS Marketplace QuickLaunch
+
+For AWS Marketplace with product offering with Helm chart delivery method and support of QuickLaunch we need to prepare a parent helm chart that has to child charts: odd-platform and postgresql database for it.
+This approach would benefit from utilizing (QuickLaunch)[https://docs.aws.amazon.com/marketplace/latest/buyerguide/buyer-configuring-a-product.html#buyer-launch-container-quicklaunch]: first AWS Marketplace creates a new EKS cluster with its built-in CloudFormation for EKS and in the same stack it deploys mentioned Helm chart that should be stored at AWS ECR managed by AWS Marketplace that was created during new product registration.
+
+There are some differences to the base helm chart of odd-platform that should reflect new approach of AWS Marketplace QuickLaunch.
+
+1. configmap.yaml template for odd-platform has been changed 
+```
+...
+data:
+  application.yml: |-
+    spring:
+      datasource:
+        username: "postgres"
+        password: "{{ .Values.global.postgresql.auth.postgresPassword }}"
+        url: "jdbc:postgresql://odd-quicklaunch-postgresql:5432/odd-platform"
+...
+```
+That has been done to get value for password from value.yaml file of parent chart
+
+2. service.yaml template for odd-platform has been changed
+2.1. service.type: 
+```
+...
+spec:
+  type: {{ .Values.global.platformServiceType }}
+...
+```
+That has been done as AWS Marketplace QuickLaunch does not accept Override parameter key with '-' that we have in the name of chart. 
+So we have to use another name.
+
+2.2 annotations: 
+```
+...
+metadata:
+  name: {{ include "odd-platform.fullname" . }}
+  annotations:
+    "service.beta.kubernetes.io/load-balancer-source-ranges": "{{ .Values.global.loadBalancerSourceRanges }}"
+...
+```
+There is a restriction of 50 characters to be propagated with AWS Marketplace QuickLaunch Override parameter key. 
+So we have to use new shorter one.
+
+
+# Steps to update version
+Change tags for odd-platform and chart version according to the actual state. In this example odd-platform version 0.19.0 and chart version 1.0.8 are used.
+```commandline
+$ aws ecr get-login-password --region us-east-1  --profile ${AWSMarketplaceProfile} | docker login --username AWS --password-stdin 709825985650.dkr.ecr.us-east-1.amazonaws.com
+$ docker pull ghcr.io/opendatadiscovery/odd-platform:0.19.0
+$ docker tag ghcr.io/opendatadiscovery/odd-platform:0.19.0 709825985650.dkr.ecr.us-east-1.amazonaws.com/provectus/odd:0.19.0
+$ docker push 709825985650.dkr.ecr.us-east-1.amazonaws.com/provectus/odd:0.19.0
+$ helm dependency update .
+$ helm package .
+$ aws ecr get-login-password --region us-east-1 --profile ${AWSMarketplaceProfile} | helm registry login --username AWS --password-stdin 709825985650.dkr.ecr.us-east-1.amazonaws.com
+$ export HELM_EXPERIMENTAL_OCI=1 && helm push ./odd-quicklaunch-1.0.8.tgz oci://709825985650.dkr.ecr.us-east-1.amazonaws.com/provectus/
+```
+
+# Version Information in AWS Marketplace (Examples)
+
+| Property                    | Value                                                                                                                                     |
+|-----------------------------|-------------------------------------------------------------------------------------------------------------------------------------------|
+| Version title               | odd-platform 0.19.0 quicklaunch                                                                                                           |
+| Release notes               | Data Quality Dashboard with QuickLaunch capabilities (odd-patform and postgres in one helm chart)                                         | 
+| Helm chart                  | 709825985650.dkr.ecr.us-east-1.amazonaws.com/provectus/odd-quicklaunch:1.0.13                                                             |                        
+| Container Image             | 709825985650.dkr.ecr.us-east-1.amazonaws.com/provectus/odd:0.19.0                                                                         |                               
+| Delivery option title       | Open Data Discovery Platfrom Quick Launch                                                                                                 |
+| Delivery option description | This delivery method allows to deploy odd-platform and its database with one helm chart on newly created AWS EKS cluster with QuickLaunch | 
+| Usage instructions          | Utilizing QuickLaunch through CloudFormation enables a hassle-free setup of the Open Data Discovery platform. QuickLaunch streamlines the process by establishing a new EKS cluster and deploying a pre-set Helm chart that includes the odd-platform and a corresponding Postgresql database. You can explore the Chart's details and its AWS Marketplace inclusion at ODD Helm Chart GitHub - https://github.com/opendatadiscovery/charts/charts/tree/main/charts/odd-quicklaunch.  When setting up, choose a name of your liking for the stack and the EKS cluster (for example, you might use 'ODD-EKS' for both). Keep the 'Helm release name' as 'odd-quicklaunch'. For 'postgresPassword', select a secure password that will be used by the 'postgres' admin in the database and for connecting the odd-platform to the database. For 'AllowedIPAddresses', input a CIDR-formatted IP range or single IP (like 8.8.8.8/32) that will have access to the odd-platform. Determine your IP address at resources such as https://whatismyipaddress.com/. Remember, this IP might change with network reconfigurations, and you'll need to update access rules accordingly, as guided here: https://blog.opendatadiscovery.org/quick-launch-of-open-data-discovery-odd-platform-on-amazon-elastic-kubernetes-service-eks-5a0a4489e492.  The entire setup process should take approximately 30-40 minutes. Once completed, navigate to the EKS section in the AWS Console. Locate your named EKS cluster, and under 'Resources', proceed to 'Service and networking', then 'Services'. Here, find and click on 'odd-quicklaunch-odd-platform'. The service page will display 'Load Balancer URLs', leading to your odd-platform. Note that accessing these URLs defaults to an HTTPS protocol, which QuickLaunch does not support. Ensure you access it via HTTP, like http://[your-load-balancer-URL].  Important Considerations: This setup only supports HTTP, not HTTPS, and is not secure for transmitting sensitive data. For production, consider enabling HTTPS.  The database data is not persistent. A restart of the postgres pod will erase data, so for production use, consider a persistent postgresql setup.  This setup excludes odd collectors and adapters, meaning no automatic data ingestion. To add these, refer to our blog post https://blog.opendatadiscovery.org/introducing-odd-collector-configuration-for-aws-eks-bcc2bf04ae7e.  We are always happy to assist if you need any help! Just reach out to our community at ODD Community Slack https://go.opendatadiscovery.org/slack                                                                                                                                          |
+| Supported services          | Amazon Elastic Kubernetes Service (EKS)                                                                                                   |
+| Helm release name           | odd-quicklaunch                                                                                                                           |                                                                                                                           
+| QuickLaunch                 | Enable QuickLaunch                                                                                                                        |
+
+# Override parameters
+
+| Override parameter | Override parameter key | Override parameter default | CloudFormation parameter name | CloudFormation parameter description |  Hide passwords and secrets | 
+| ------------- | ------------- | ------------- | ------------- | ------------- | ------------- |
+| Override parameter 1 | global.platformServiceType | LoadBalancer | platformServiceType | Leave default LoadBalancer so that odd-platform has a public host | false |
+| Override parameter 2 | global.postgresql.auth.postgresPassword |  | postgresPassword | Password for user postgres that is used to connect odd-platform to its database | true |
+| Override parameter 3 | global.loadBalancerSourceRanges | 1.1.1.1/32 | AllowedIPAddresses | List of CIDRs separated with comma that would have connection to odd-platform. For instance, it could be your IP address found at https://whatismyipaddress.com/.Format for single IP: x.x.x.x/32 | false |

charts/odd-quicklaunch/charts/odd-platform/.helmignore

@@ -0,0 +1,23 @@
+# Patterns to ignore when building packages.
+# This supports shell glob matching, relative path matching, and
+# negation (prefixed with !). Only one pattern per line.
+.DS_Store
+# Common VCS dirs
+.git/
+.gitignore
+.bzr/
+.bzrignore
+.hg/
+.hgignore
+.svn/
+# Common backup files
+*.swp
+*.bak
+*.tmp
+*.orig
+*~
+# Various IDEs
+.project
+.idea/
+*.tmproj
+.vscode/

charts/odd-quicklaunch/charts/odd-platform/Chart.yaml

@@ -0,0 +1,23 @@
+apiVersion: v2
+name: odd-platform
+description: A Helm chart for Kubernetes
+
+# A chart can be either an 'application' or a 'library' chart.
+#
+# Application charts are a collection of templates that can be packaged into versioned archives
+# to be deployed.
+#
+# Library charts provide useful utilities or functions for the chart developer. They're included as
+# a dependency of application charts to inject those utilities and functions into the rendering
+# pipeline. Library charts do not define any templates and therefore cannot be deployed.
+type: application
+
+# This is the chart version. This version number should be incremented each time you make changes
+# to the chart and its templates, including the app version.
+# Versions are expected to follow Semantic Versioning (https://semver.org/)
+version: 0.1.8
+
+# This is the version number of the application being deployed. This version number should be
+# incremented each time you make changes to the application. Versions are not expected to
+# follow Semantic Versioning. They should reflect the version the application is using.
+appVersion: latest

charts/odd-quicklaunch/charts/odd-platform/README.md

@@ -0,0 +1,44 @@
+# odd-platform
+
+![Version: 0.1.6](https://img.shields.io/badge/Version-0.1.6-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: latest](https://img.shields.io/badge/AppVersion-latest-informational?style=flat-square)
+
+A Helm chart for Kubernetes
+
+## Values
+
+| Key | Type | Default | Description |
+|-----|------|---------|-------------|
+| affinity | object | `{}` |  |
+| autoscaling.enabled | bool | `false` |  |
+| autoscaling.maxReplicas | int | `100` |  |
+| autoscaling.minReplicas | int | `1` |  |
+| autoscaling.targetCPUUtilizationPercentage | int | `80` |  |
+| config | string | `nil` |  |
+| fullnameOverride | string | `""` |  |
+| image.pullPolicy | string | `"IfNotPresent"` |  |
+| image.repository | string | `"ghcr.io/opendatadiscovery/odd-platform"` |  |
+| image.tag | string | `""` |  |
+| imagePullSecrets | list | `[]` |  |
+| ingress.annotations | object | `{}` |  |
+| ingress.className | string | `""` |  |
+| ingress.enabled | bool | `false` |  |
+| ingress.hosts[0].host | string | `"chart-example.local"` |  |
+| ingress.hosts[0].paths | list | `[]` |  |
+| ingress.tls | list | `[]` |  |
+| nameOverride | string | `""` |  |
+| nodeSelector | object | `{}` |  |
+| podAnnotations | object | `{}` |  |
+| podSecurityContext | object | `{}` |  |
+| replicaCount | int | `1` |  |
+| resources | object | `{}` |  |
+| securityContext | object | `{}` |  |
+| service.annotations | object | `{}` |  |
+| service.port | int | `80` |  |
+| service.type | string | `"ClusterIP"` |  |
+| serviceAccount.annotations | object | `{}` |  |
+| serviceAccount.create | bool | `true` |  |
+| serviceAccount.name | string | `""` |  |
+| tolerations | list | `[]` |  |
+
+----------------------------------------------
+Autogenerated from chart metadata using [helm-docs v1.11.0](https://github.com/norwoodj/helm-docs/releases/v1.11.0)

charts/odd-quicklaunch/charts/odd-platform/templates/_helpers.tpl

@@ -0,0 +1,63 @@
+{{/* vim: set filetype=mustache: */}}
+{{/*
+Expand the name of the chart.
+*/}}
+{{- define "odd-platform.name" -}}
+{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
+{{- end }}
+
+{{/*
+Create a default fully qualified app name.
+We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
+If release name contains chart name it will be used as a full name.
+*/}}
+{{- define "odd-platform.fullname" -}}
+{{- if .Values.fullnameOverride }}
+{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
+{{- else }}
+{{- $name := default .Chart.Name .Values.nameOverride }}
+{{- if contains $name .Release.Name }}
+{{- .Release.Name | trunc 63 | trimSuffix "-" }}
+{{- else }}
+{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
+{{- end }}
+{{- end }}
+{{- end }}
+
+{{/*
+Create chart name and version as used by the chart label.
+*/}}
+{{- define "odd-platform.chart" -}}
+{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
+{{- end }}
+
+{{/*
+Common labels
+*/}}
+{{- define "odd-platform.labels" -}}
+helm.sh/chart: {{ include "odd-platform.chart" . }}
+{{ include "odd-platform.selectorLabels" . }}
+{{- if .Chart.AppVersion }}
+app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
+{{- end }}
+app.kubernetes.io/managed-by: {{ .Release.Service }}
+{{- end }}
+
+{{/*
+Selector labels
+*/}}
+{{- define "odd-platform.selectorLabels" -}}
+app.kubernetes.io/name: {{ include "odd-platform.name" . }}
+app.kubernetes.io/instance: {{ .Release.Name }}
+{{- end }}
+
+{{/*
+Create the name of the service account to use
+*/}}
+{{- define "odd-platform.serviceAccountName" -}}
+{{- if .Values.serviceAccount.create }}
+{{- default (include "odd-platform.fullname" .) .Values.serviceAccount.name }}
+{{- else }}
+{{- default "default" .Values.serviceAccount.name }}
+{{- end }}
+{{- end }}

charts/odd-quicklaunch/charts/odd-platform/templates/configmap.yaml

@@ -0,0 +1,13 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: {{ include "odd-platform.fullname" . }}-application-conf
+  labels:
+    {{- include "odd-platform.labels" . | nindent 4 }}
+data:
+  application.yml: |-
+    spring:
+      datasource:
+        username: "postgres"
+        password: "{{ .Values.global.postgresql.auth.postgresPassword }}"
+        url: "jdbc:postgresql://odd-quicklaunch-postgresql:5432/odd-platform"

charts/odd-quicklaunch/charts/odd-platform/templates/deployment.yaml

@@ -0,0 +1,74 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: {{ include "odd-platform.fullname" . }}
+  labels:
+    {{- include "odd-platform.labels" . | nindent 4 }}
+spec:
+{{- if not .Values.autoscaling.enabled }}
+  replicas: {{ .Values.replicaCount }}
+{{- end }}
+  selector:
+    matchLabels:
+      {{- include "odd-platform.selectorLabels" . | nindent 6 }}
+  template:
+    metadata:
+    {{- with .Values.podAnnotations }}
+      annotations:
+        {{- toYaml . | nindent 8 }}
+    {{- end }}
+      labels:
+        {{- include "odd-platform.selectorLabels" . | nindent 8 }}
+    spec:
+      {{- with .Values.imagePullSecrets }}
+      imagePullSecrets:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      serviceAccountName: {{ include "odd-platform.serviceAccountName" . }}
+      securityContext:
+        {{- toYaml .Values.podSecurityContext | nindent 8 }}
+      containers:
+        - name: {{ .Chart.Name }}
+          securityContext:
+            {{- toYaml .Values.securityContext | nindent 12 }}
+          image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
+          imagePullPolicy: {{ .Values.image.pullPolicy }}
+          ports:
+            - name: http
+              containerPort: 8080
+              protocol: TCP
+          livenessProbe:
+            tcpSocket:
+              port: http
+          readinessProbe:
+            tcpSocket:
+              port: http
+          resources:
+            {{- toYaml .Values.resources | nindent 12 }}
+          {{- if (.Values.config).env }}
+          env:
+            {{- toYaml .Values.config.env | nindent 12 }}
+          {{- end }}
+          volumeMounts:
+            {{- if (.Values.config).yaml }}
+            - name: odd-platform-application-conf
+              mountPath: /app/config/
+            {{- end }}
+      volumes:
+        {{- if (.Values.config).yaml }}
+        - name: odd-platform-application-conf
+          configMap:
+            name: {{ include "odd-platform.fullname" . }}-application-conf
+        {{- end }}
+      {{- with .Values.nodeSelector }}
+      nodeSelector:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      {{- with .Values.affinity }}
+      affinity:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      {{- with .Values.tolerations }}
+      tolerations:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}

charts/odd-quicklaunch/charts/odd-platform/templates/hpa.yaml

@@ -0,0 +1,28 @@
+{{- if .Values.autoscaling.enabled }}
+apiVersion: autoscaling/v2beta1
+kind: HorizontalPodAutoscaler
+metadata:
+  name: {{ include "odd-platform.fullname" . }}
+  labels:
+    {{- include "odd-platform.labels" . | nindent 4 }}
+spec:
+  scaleTargetRef:
+    apiVersion: apps/v1
+    kind: Deployment
+    name: {{ include "odd-platform.fullname" . }}
+  minReplicas: {{ .Values.autoscaling.minReplicas }}
+  maxReplicas: {{ .Values.autoscaling.maxReplicas }}
+  metrics:
+  {{- if .Values.autoscaling.targetCPUUtilizationPercentage }}
+    - type: Resource
+      resource:
+        name: cpu
+        targetAverageUtilization: {{ .Values.autoscaling.targetCPUUtilizationPercentage }}
+  {{- end }}
+  {{- if .Values.autoscaling.targetMemoryUtilizationPercentage }}
+    - type: Resource
+      resource:
+        name: memory
+        targetAverageUtilization: {{ .Values.autoscaling.targetMemoryUtilizationPercentage }}
+  {{- end }}
+{{- end }}