Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add ementas service with Pulumi #64

Merged
merged 45 commits into from
Aug 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
bf0733c
feat: initial pulumi setup
limwa May 22, 2024
0d7e77e
feat: initial work with pulumi
limwa Jun 24, 2024
99da13e
feat: add script to sync crds
limwa Jun 24, 2024
542146e
feat: new method of getting CRDs
limwa Jun 25, 2024
3cb92d0
feat: add ementas service
limwa Jun 25, 2024
14da41a
chore: remove unused file
limwa Jun 25, 2024
e352932
fix: add resources to be deployed
limwa Jun 25, 2024
1e1a9c3
chore: rework crd sync system
limwa Jun 25, 2024
c316cfb
fix: deploy ementas service
limwa Jun 25, 2024
7293d08
wip: add vpn for improved dev experience
limwa Jun 28, 2024
9bad573
feat: add mongodb resource for creating db users
limwa Jul 1, 2024
8140dd7
feat: finish mongodb controller
limwa Jul 1, 2024
092187e
refactor: add mongodb user management to pulumi
limwa Jul 1, 2024
965d187
fix: set operator to watch all namespaces
limwa Jul 1, 2024
1049297
feat: secret replication working
limwa Jul 5, 2024
f56c76d
refactor: unify mongodb deployment for prod and dev
limwa Jul 5, 2024
3f676bc
chore: remove unused file
limwa Jul 5, 2024
0a68e52
refactor: logic for concatenation of lists
limwa Jul 5, 2024
64afbeb
refactor: use commit signals and pending values
limwa Jul 5, 2024
04103b5
refactor: move mongodb chart to pulumi
limwa Jul 5, 2024
b6f61f5
fix: use longhorn storage class
limwa Jul 5, 2024
f6c87c1
chore: rename charts folder to assets
limwa Jul 5, 2024
18dbfa1
refactor: move things around and separate concerns
limwa Jul 7, 2024
ff6bb57
chore: comment out clustersecret crds
limwa Jul 13, 2024
c22a550
fix: add yq check to sync-crds.sh
limwa Jul 13, 2024
8df46a5
chore: fix typo
limwa Jul 13, 2024
d54cdce
refactor: minor touchups
limwa Jul 15, 2024
0dbf12f
chore: add eslint and prettier
limwa Jul 15, 2024
96c5858
style: format files
limwa Jul 15, 2024
9e7248a
chore: fix some eslint problems
limwa Jul 15, 2024
1865872
chore: add scripts for eslint and prettier
limwa Jul 15, 2024
8f78b91
chore: configure typed linting
limwa Jul 16, 2024
8b3b737
style: format code
limwa Jul 16, 2024
944abeb
ci: add checks for pulumi code
limwa Jul 16, 2024
dfa4a9d
ci: fix path for execution
limwa Jul 16, 2024
fe0d8ae
chore: rework project structure
limwa Jul 17, 2024
2e6bcdf
chore: minor changes to sync-crds
limwa Jul 17, 2024
49b1152
chore: update main package.json
limwa Jul 17, 2024
5259b67
chore: update package names
limwa Jul 17, 2024
4c60d16
ci: reuse preparation steps
limwa Jul 17, 2024
5352042
chore: silence yq warnings
limwa Jul 17, 2024
1995438
chore: recreate stack
limwa Jul 17, 2024
f1aa9f7
Merge branch 'main' into feature/pulumi
limwa Jul 22, 2024
c610b82
chore: comment out ementas service
limwa Aug 8, 2024
320a449
Merge branch 'main' into feature/pulumi
limwa Aug 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions .github/actions/prepare-pulumi-project/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: "Prepare Pulumi project"
description: "Installs dependencies and clones the project"
inputs:
node-version:
description: "Node version"
required: false
default: "22"
crd2pulumi-version:
description: "crd2pulumi version"
required: false
default: "v1.4.0"
yq-version:
description: "yq version"
required: false
default: "v4.40.3"
runs:
using: composite
steps:

- name: Install yq
shell: sh
run: sudo curl -L https://github.com/mikefarah/yq/releases/download/${{ inputs.yq-version }}/yq_linux_amd64 -o /usr/bin/yq && sudo chmod +x /usr/bin/yq

- name: Install crd2pulumi
shell: sh
run: |
mkdir -p ./bin/crd2pulumi
cd ./bin/crd2pulumi
echo "crd2pulumi" > files.txt
curl -L https://github.com/pulumi/crd2pulumi/releases/download/${{ inputs.crd2pulumi-version }}/crd2pulumi-${{ inputs.crd2pulumi-version }}-linux-amd64.tar.gz | tar -T files.txt -xvz
sudo mv crd2pulumi /usr/bin/crd2pulumi
sudo chmod +x /usr/bin/crd2pulumi

- uses: pnpm/action-setup@v4
with:
package_json_file: services/pulumi/package.json

- uses: actions/setup-node@v4
with:
node-version: ${{ inputs.node-version }}
cache: "pnpm"
cache-dependency-path: services/pulumi/pnpm-lock.yaml

- name: Sync CRDs
shell: sh
run: ./services/pulumi/sync-crds.sh
30 changes: 30 additions & 0 deletions .github/workflows/pulumi-linter.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: pulumi-lint

on:
pull_request:
branches: [ "main" ]
paths: [ "services/pulumi/**" ]
push:
branches: [ "main" ]
paths: [ "services/pulumi/**" ]

jobs:
format:
name: Prettier
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/prepare-pulumi-project
- name: Run prettier
working-directory: services/pulumi/niployments
run: pnpm run prettier

lint:
name: ESLint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/prepare-pulumi-project
- name: Run ESLint
working-directory: services/pulumi/niployments
run: pnpm run lint
108 changes: 108 additions & 0 deletions schemas/crds.schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
{
"$schema": "https://json-schema.org/draft-07/schema",
"$id": "https://niployments.niaefeup.pt/schemas/crds.schema.json",
"title": "CRDs Download Configuration",
"description": "Configuration for the download and extraction of CRDs, used for Pulumi-based resources",
"type": "object",
"required": ["manifests"],
"additionalProperties": false,
"properties": {
"repositories": {
"description": "List of repositories to add to the Helm client",
"type": "array",
"minItems": 1,
"items": {
"type": "object",
"required": ["name", "url"],
"additionalProperties": false,
"properties": {
"name": {
"type": "string"
},
"url": {
"type": "string"
}
}
}
},
"manifests": {
"description": "List of manifests to download and extract CRDs from",
"type": "array",
"minItems": 1,
"items": {
"type": "object",
"oneOf": [
{
"type":"object",
"required": ["type", "chart"],
"additionalProperties": false,
"properties": {
"type": {
"description": "Method for downloading the CRDs",
"type": "string",
"enum": ["helm"]
},
"chart": {
"description": "Name of the Helm chart to download",
"type": "string",
"examples": ["traefik/traefik"]
},
"version": {
"description": "Version of the Helm chart to download",
"type": "string",
"examples": ["28.3.0"]
}
}
},
{
"type":"object",
"required": ["type", "chart"],
"additionalProperties": false,
"properties": {
"type": {
"description": "Method for downloading the CRDs",
"type": "string",
"enum": ["template"]
},
"chart": {
"description": "Name of the Helm chart to download",
"type": "string",
"examples": ["jetstack/cert-manager"]
},
"version": {
"description": "Version of the Helm chart to download",
"type": "string",
"examples": ["v1.14.7"]
},
"options": {
"description": "Options to pass to the Helm template command",
"type": "string"
}
}
},
{
"type":"object",
"required": ["type", "name", "url"],
"additionalProperties": false,
"properties": {
"type": {
"description": "Method for downloading the CRDs",
"type": "string",
"enum": ["curl"]
},
"name": {
"description": "Name of the manifest to download",
"type": "string",
"examples": ["cloudnative-pg/cloudnative-pg"]
},
"url": {
"description": "URL to download the chart from",
"type": "string"
}
}
}
]
}
}
}
}
2 changes: 1 addition & 1 deletion services/cert-manager/deploy-dev.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,6 @@ helm repo update

kubectl apply -f $(dirname $0)/00-namespace.yaml

helm upgrade --install -f $(dirname $0)/values.yaml cert-manager jetstack/cert-manager --namespace cert-manager
helm upgrade --install -f $(dirname $0)/values.yaml cert-manager jetstack/cert-manager --version v1.14.7 --namespace cert-manager

kubectl apply -f $(dirname $0)/01-cluster-issuer-dev.yaml
19 changes: 0 additions & 19 deletions services/databases/mongodb/deploy-mongodb-dev.sh

This file was deleted.

14 changes: 0 additions & 14 deletions services/databases/mongodb/deploy-mongodb-prod.sh

This file was deleted.

46 changes: 0 additions & 46 deletions services/databases/mongodb/mongodb-cluster.yaml

This file was deleted.

3 changes: 3 additions & 0 deletions services/pulumi/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
bin/
node_modules/
/crds/
55 changes: 55 additions & 0 deletions services/pulumi/crds.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# yaml-language-server: $schema=../../schemas/crds.schema.json
repositories:
- name: jetstack
url: https://charts.jetstack.io
# - name: clustersecret
# url: https://charts.clustersecret.io/
# mongodb official charts are not needed while temporary fix is in use
# - name: mongodb
# url: https://mongodb.github.io/helm-charts
- name: mittwald
url: https://helm.mittwald.de
- name: longhorn
url: https://charts.longhorn.io
- name: traefik
url: https://traefik.github.io/charts

manifests:
# cert-manager
- type: template
chart: jetstack/cert-manager
version: v1.14.7
options: |
--values=../cert-manager/values.yaml
--namespace=cert-manager
# cilium - no CRDs
# cluster-secret
# - type: helm
# chart: clustersecret/cluster-secret
# databases/mongodb
## [version for official helm charts below]
# - type: template
# chart: mongodb/community-operator
# options: |
# --namespace=mongodb
## [version for temporary fix version below]
- type: curl
name: mongodb/community-operator
url: https://raw.githubusercontent.com/limwa/mongodb-kubernetes-operator/master/config/crd/bases/mongodbcommunity.mongodb.com_mongodbcommunity.yaml
# databases/postgresql
- type: curl
name: cloudnative-pg/cloudnative-pg
url: https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/release-1.22/releases/cnpg-1.22.2.yaml
# image-registry (harbor) - no CRDs
# replicator - no CRDs
# storage/longhorn
- type: template
chart: longhorn/longhorn
version: 1.6.1
options: |
--values=../storage/longhorn/prod-values.yaml
--namespace=longhorn
# traefik
- type: helm
chart: traefik/traefik
version: 28.3.0
7 changes: 7 additions & 0 deletions services/pulumi/niployments/.prettierignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Pnpm files
/node_modules/
/bin/
package.json

# Assets
/assets/
3 changes: 3 additions & 0 deletions services/pulumi/niployments/.prettierrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"tabWidth": 2
}
6 changes: 6 additions & 0 deletions services/pulumi/niployments/Pulumi.prod.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
encryptionsalt: v1:upriG3hjats=:v1:Mjju6ZYDjjKKyVPn:Bn6nuCMHoUCltZWaXycxDl5bvv7ndA==
config:
niployments:mongodb/admin-password:
secure: v1:GXCb2z0V4Az3053O:y6EgjlKVYr6BWebVazmAmdC/YCI=
niployments:mongodb/nimentas-password:
secure: v1:HxF47P+9RXfi4nfc:47vWUGoYJgjmqxaWwT2M8SSCgI0=
7 changes: 7 additions & 0 deletions services/pulumi/niployments/Pulumi.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
name: niployments
runtime: nodejs
description: Pulumi program to manage the niployments cluster
config:
pulumi:tags:
value:
pulumi:template: kubernetes-typescript
Loading
Loading