Skip to content

Commit 03e41fc

Browse files
maltesandersoenkeliebaurazvanfhennig
authored
First implementation for Trino (#9)
* Copied hdfs skeleton operator and replaced all names with Trino. Compiles * Added stuff to get basic pod operations working. * Fixed package and command handling. * WIP - added product-config * Fixed command & jvm config * wip * compute_files() should not put all properties in all files since trino will refuse to start. * Added .gitignore. * WIP - Monitoring and cluster working. No "discovery" yet. * WIP - Monitoring and cluster working. No "discovery" yet. * WIP - Discovery working. * Added tests. Generating node-id. Moved CRD check to operator binary. * Added command subcommands. * WIP - started adding hive support. * WIP - removed hardcoded hive configmap code * Added packaging. * WIP password.db gets written now * Queries via hive and S3 working * Added cargo deny and fixed fmt. * Removed S3 credentials. * Made S3Connection in the custom resource accessible globally. * WIP pem file is written * WIP spike: TLS is enabled and works next: cleanup required * WIP cleanup * Mapped HTTPS port into CRD * Added HTTPS port in the container builder * formatting fix * fixed stupid merge mistake * Works with trino-opa-authorizer, opa operator and regorule operator. * Fixed clippy. * Updated cargo deny. * WIP - Added License, CHANGELOG.md, Makefile and started with documentation. * Added missing cli subcommands. * Removed unwraps, extended usage doc. * Removed the 'required' from the HTTPS port for the coordinator * Removed commented code, changed some warn logs to debug logs. * Update usage example. * Use backticks for the regorule example to make antora happy. * Adapted to PR review. * Adapted hive dependency to main branch. Co-authored-by: Sönke Liebau <[email protected]> Co-authored-by: Razvan-Daniel Mihai <[email protected]> Co-authored-by: Felix Hennig <[email protected]> Co-authored-by: Felix Hennig <[email protected]>
1 parent b2e56ab commit 03e41fc

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

48 files changed

+6732
-0
lines changed

‎.github/dependabot.yml

+29
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# =============
2+
# This file is automatically generated from the templates in stackabletech/operator-templating
3+
# DON'T MANUALLY EDIT THIS FILE
4+
# =============
5+
version: 2
6+
updates:
7+
- package-ecosystem: "github-actions"
8+
directory: "/"
9+
schedule:
10+
interval: "weekly"
11+
labels:
12+
- "type/dependencies"
13+
reviewers:
14+
- "stackabletech/developers"
15+
16+
- package-ecosystem: "cargo"
17+
directory: "/"
18+
schedule:
19+
interval: "weekly"
20+
labels:
21+
- "type/dependencies"
22+
reviewers:
23+
- "stackabletech/rust-developers"
24+
ignore:
25+
# We never want to be notified about a kube-rs update.
26+
# It often contains breaking changes so it has to be updated manually anyway
27+
# and it needs to be updated together with kube-runtime, kube-derive etc.
28+
- dependency-name: "kube*"
29+
- dependency-name: "k8s-openapi"

‎.github/pull_request_template.md

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
## Description
2+
3+
## Review Checklist
4+
- [ ] Code contains useful comments
5+
- [ ] (Integration-)Test cases added (or not applicable)
6+
- [ ] Documentation added (or not applicable)
7+
- [ ] Changelog updated (or not applicable)
8+
- [ ] Cargo.toml only contains references to git tags (not specific commits or branches)

‎.github/workflows/daily_security.yml

+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# =============
2+
# This file is automatically generated from the templates in stackabletech/operator-templating
3+
# DON'T MANUALLY EDIT THIS FILE
4+
# =============
5+
name: Security audit
6+
7+
on:
8+
schedule:
9+
- cron: '0 0 * * *'
10+
workflow_dispatch:
11+
12+
jobs:
13+
audit:
14+
runs-on: ubuntu-latest
15+
steps:
16+
- uses: actions/[email protected]
17+
- uses: actions-rs/[email protected]
18+
with:
19+
token: ${{ secrets.GITHUB_TOKEN }}

‎.github/workflows/docker.yml

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# =============
2+
# This file is automatically generated from the templates in stackabletech/operator-templating
3+
# DON'T MANUALLY EDIT THIS FILE
4+
# =============
5+
name: Docker
6+
7+
on:
8+
push:
9+
branches:
10+
- main
11+
12+
jobs:
13+
build:
14+
name: Build Docker image
15+
runs-on: ubuntu-latest
16+
steps:
17+
- uses: actions/[email protected]
18+
- shell: bash
19+
env:
20+
NEXUS_PASSWORD: ${{ secrets.NEXUS_PASSWORD }}
21+
run: make docker

‎.github/workflows/publish_dev_artifacts.yml

+79
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# =============
2+
# This file is automatically generated from the templates in stackabletech/operator-templating
3+
# DON'T MANUALLY EDIT THIS FILE
4+
# =============
5+
name: Publish Dev Artifacts
6+
7+
on:
8+
push:
9+
branches:
10+
- main
11+
pull_request:
12+
13+
env:
14+
PRODUCT_NAME: trino
15+
CARGO_TERM_COLOR: always
16+
CARGO_INCREMENTAL: '0'
17+
CARGO_PROFILE_DEV_DEBUG: '0'
18+
RUSTFLAGS: "-D warnings -W rust-2021-compatibility"
19+
REPO_APT_DEV_URL: https://repo.stackable.tech/repository/deb-dev
20+
REPO_RPM_DEV_URL: https://repo.stackable.tech/repository/rpm-dev
21+
22+
jobs:
23+
debian10:
24+
runs-on: debian10
25+
steps:
26+
- uses: actions/checkout@v2
27+
- name: Change version if is PR
28+
if: ${{ github.event_name == 'pull_request' }}
29+
# We use "mr" instead of "pr" to denote pull request builds, as this prefix comes before "nightly" when lexically
30+
# sorting packages by version. This means that when installing the package without specifying a version the
31+
# nighly version is considered more current than mr versions and installed by default
32+
run: sed -i -e 's/^version = "\([0-9]\+\.[0-9]\+\.[0-9]\+\).*/version = "\1-mr${{ github.event.number }}"/' rust/operator-binary/Cargo.toml
33+
- name: Build
34+
run: ~/.cargo/bin/cargo +nightly build --verbose --release
35+
- name: Build apt package
36+
run: ~/.cargo/bin/cargo deb --manifest-path rust/operator-binary/Cargo.toml --no-build
37+
- name: Publish apt package
38+
env:
39+
NEXUS_PASSWORD: ${{ secrets.NEXUS_PASSWORD }}
40+
if: env.NEXUS_PASSWORD != null
41+
run: >-
42+
/usr/bin/curl
43+
--fail
44+
-u 'github:${{ secrets.NEXUS_PASSWORD }}'
45+
-H "Content-Type: multipart/form-data"
46+
--data-binary "@./$(find target/debian/ -name *.deb)"
47+
"${{ env.REPO_APT_DEV_URL }}/"
48+
- name: Clean
49+
run: ~/.cargo/bin/cargo clean
50+
51+
centos:
52+
runs-on: centos${{ matrix.node }}
53+
strategy:
54+
matrix:
55+
node: [ 7, 8 ]
56+
steps:
57+
- uses: actions/checkout@v2
58+
- name: Change version if is PR
59+
if: ${{ github.event_name == 'pull_request' }}
60+
# We use "mr" instead of "pr" to denote pull request builds, as this prefix comes before "nightly" when lexically
61+
# sorting packages by version. This means that when installing the package without specifying a version the
62+
# nighly version is considered more current than mr versions and installed by default
63+
run: sed -i -e 's/^version = "\([0-9]\+\.[0-9]\+\.[0-9]\+\).*/version = "\1-mr${{ github.event.number }}"/' rust/operator-binary/Cargo.toml
64+
- name: Build
65+
run: ~/.cargo/bin/cargo +nightly build --verbose --release
66+
- name: Build RPM package
67+
run: packaging/buildrpm.sh stackable-${{ env.PRODUCT_NAME }}-operator-binary stackable-${{ env.PRODUCT_NAME }}-operator
68+
- name: Publish RPM package
69+
env:
70+
NEXUS_PASSWORD: ${{ secrets.NEXUS_PASSWORD }}
71+
if: env.NEXUS_PASSWORD != null
72+
run: >-
73+
/usr/bin/curl
74+
--fail
75+
-u 'github:${{ secrets.NEXUS_PASSWORD }}'
76+
--upload-file "./$(find target/rpm/RPMS/x86_64/ -name *.rpm)"
77+
"${{ env.REPO_RPM_DEV_URL }}/el${{ matrix.node }}/"
78+
- name: Clean
79+
run: ~/.cargo/bin/cargo clean

‎.github/workflows/publish_release_artifacts.yml

+78
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
# =============
2+
# This file is automatically generated from the templates in stackabletech/operator-templating
3+
# DON'T MANUALLY EDIT THIS FILE
4+
# =============
5+
name: Publish-Release-Artifacts
6+
7+
on:
8+
push:
9+
tags:
10+
- "*"
11+
12+
env:
13+
PRODUCT_NAME: trino
14+
CARGO_TERM_COLOR: always
15+
CARGO_INCREMENTAL: '0'
16+
CARGO_PROFILE_DEV_DEBUG: '0'
17+
RUSTFLAGS: "-D warnings -W rust-2021-compatibility"
18+
REPO_APT_RELEASE_URL: https://repo.stackable.tech/repository/deb-release
19+
REPO_RPM_RELEASE_URL: https://repo.stackable.tech/repository/rpm-release
20+
21+
jobs:
22+
debian10:
23+
runs-on: debian10
24+
steps:
25+
- uses: actions/checkout@v2
26+
- name: Change version if is PR
27+
if: ${{ github.event_name == 'pull_request' }}
28+
# We use "mr" instead of "pr" to denote pull request builds, as this prefix comes before "nightly" when lexically
29+
# sorting packages by version. This means that when installing the package without specifying a version the
30+
# nighly version is considered more current than mr versions and installed by default
31+
run: sed -i -e 's/^version = "\([0-9]\+\.[0-9]\+\.[0-9]\+\).*/version = "\1-mr${{ github.event.number }}"/' rust/operator-binary/Cargo.toml
32+
- name: Build
33+
run: ~/.cargo/bin/cargo +nightly build --verbose --release
34+
- name: Build apt package
35+
run: ~/.cargo/bin/cargo deb --manifest-path rust/operator-binary/Cargo.toml --no-build
36+
- name: Publish apt package
37+
env:
38+
NEXUS_PASSWORD: ${{ secrets.NEXUS_PASSWORD }}
39+
if: env.NEXUS_PASSWORD != null
40+
run: >-
41+
/usr/bin/curl
42+
--fail
43+
-u 'github:${{ secrets.NEXUS_PASSWORD }}'
44+
-H "Content-Type: multipart/form-data"
45+
--data-binary "@./$(find target/debian/ -name *.deb)"
46+
"${{ env.REPO_APT_RELEASE_URL }}/"
47+
- name: Clean
48+
run: ~/.cargo/bin/cargo clean
49+
50+
centos:
51+
runs-on: centos${{ matrix.node }}
52+
strategy:
53+
matrix:
54+
node: [ 7, 8 ]
55+
steps:
56+
- uses: actions/checkout@v2
57+
- name: Change version if is PR
58+
if: ${{ github.event_name == 'pull_request' }}
59+
# We use "mr" instead of "pr" to denote pull request builds, as this prefix comes before "nightly" when lexically
60+
# sorting packages by version. This means that when installing the package without specifying a version the
61+
# nighly version is considered more current than mr versions and installed by default
62+
run: sed -i -e 's/^version = "\([0-9]\+\.[0-9]\+\.[0-9]\+\).*/version = "\1-mr${{ github.event.number }}"/' rust/operator-binary/Cargo.toml
63+
- name: Build
64+
run: ~/.cargo/bin/cargo +nightly build --verbose --release
65+
- name: Build RPM package
66+
run: packaging/buildrpm.sh stackable-${{ env.PRODUCT_NAME }}-operator-binary stackable-${{ env.PRODUCT_NAME }}-operator
67+
- name: Publish RPM package
68+
env:
69+
NEXUS_PASSWORD: ${{ secrets.NEXUS_PASSWORD }}
70+
if: env.NEXUS_PASSWORD != null
71+
run: >-
72+
/usr/bin/curl
73+
--fail
74+
-u 'github:${{ secrets.NEXUS_PASSWORD }}'
75+
--upload-file "./$(find target/rpm/RPMS/x86_64/ -name *.rpm)"
76+
"${{ env.REPO_RPM_RELEASE_URL }}/el${{ matrix.node }}/"
77+
- name: Clean
78+
run: ~/.cargo/bin/cargo clean

‎.github/workflows/rust.yml

+117
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
# =============
2+
# This file is automatically generated from the templates in stackabletech/operator-templating
3+
# DON'T MANUALLY EDIT THIS FILE
4+
# =============
5+
name: Rust
6+
7+
on:
8+
push:
9+
branches:
10+
- main
11+
pull_request:
12+
13+
env:
14+
CARGO_TERM_COLOR: always
15+
CARGO_INCREMENTAL: '0'
16+
CARGO_PROFILE_DEV_DEBUG: '0'
17+
RUSTFLAGS: "-D warnings -W rust-2021-compatibility"
18+
19+
jobs:
20+
21+
test:
22+
name: Run tests
23+
runs-on: ubuntu-latest
24+
steps:
25+
- uses: actions/[email protected]
26+
- uses: actions-rs/[email protected]
27+
with:
28+
profile: minimal
29+
toolchain: stable
30+
override: true
31+
- uses: Swatinem/[email protected]
32+
- uses: actions-rs/[email protected]
33+
with:
34+
command: test
35+
- uses: actions-rs/[email protected]
36+
with:
37+
command: clean
38+
39+
rustfmt:
40+
name: Run rustfmt
41+
runs-on: ubuntu-latest
42+
steps:
43+
- uses: actions/[email protected]
44+
- uses: actions-rs/[email protected]
45+
with:
46+
profile: minimal
47+
toolchain: stable
48+
components: rustfmt
49+
override: true
50+
- uses: actions-rs/[email protected]
51+
with:
52+
command: fmt
53+
args: --all -- --check
54+
55+
doc:
56+
name: Run rustdoc
57+
runs-on: ubuntu-latest
58+
steps:
59+
- uses: actions/[email protected]
60+
- uses: actions-rs/[email protected]
61+
with:
62+
profile: minimal
63+
toolchain: stable
64+
components: rustfmt
65+
override: true
66+
- uses: Swatinem/[email protected]
67+
- uses: actions-rs/[email protected]
68+
with:
69+
command: doc
70+
args: --document-private-items
71+
72+
clippy:
73+
name: Run clippy
74+
runs-on: ubuntu-latest
75+
steps:
76+
- uses: actions/[email protected]
77+
- uses: actions-rs/[email protected]
78+
with:
79+
profile: minimal
80+
toolchain: stable
81+
components: clippy
82+
override: true
83+
# We need this due to: https://github.com/actions-rs/clippy-check/issues/2
84+
- name: Check workflow permissions
85+
id: check_permissions
86+
uses: scherermichael-oss/[email protected]
87+
with:
88+
required-permission: write
89+
env:
90+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
91+
- name: Run clippy action to produce annotations
92+
uses: actions-rs/clippy-check@v1
93+
if: steps.check_permissions.outputs.has-permission
94+
with:
95+
args: --all-targets -- -D warnings
96+
token: ${{ secrets.GITHUB_TOKEN }}
97+
- name: Run clippy manually without annotations
98+
if: ${{ !steps.check_permissions.outputs.has-permission }}
99+
run: cargo clippy --all-targets -- -D warnings
100+
101+
cargo-deny:
102+
name: Run cargo deny
103+
runs-on: ubuntu-latest
104+
strategy:
105+
matrix:
106+
checks:
107+
- advisories
108+
- bans licenses sources
109+
110+
# Prevent sudden announcement of a new advisory from failing ci:
111+
continue-on-error: ${{ matrix.checks == 'advisories' }}
112+
113+
steps:
114+
- uses: actions/[email protected]
115+
- uses: EmbarkStudios/[email protected]
116+
with:
117+
command: check ${{ matrix.checks }}

‎.gitignore

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
*.iml
2+
.idea
3+
target
4+

‎CHANGELOG.md

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# Changelog
2+
3+
All notable changes to this project will be documented in this file.
4+
5+
## [Unreleased]

0 commit comments

Comments
 (0)