Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Add pre-release PR template to run and update nightly demos #64

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions .github/PULL_REQUEST_TEMPLATES/pre-release-nightly-demos.md
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There probably needs to be another list for things to check during the demos.

  • Check that the correct images are being used
    # Ensure correct product versions are used as well as 0.0.0-dev
    kubectl get pods -n default -o yaml | yq -y '.items[] | .status.containerStatuses[] | .image'
    # Ensure the 0.0.0-dev operators are used.
    kubectl get pods -n stackable-operators -o yaml | yq -y '.items[] | .status.containerStatuses[] | .image'
  • Check the demo (duh)

Copy link
Member

@NickLarsenNZ NickLarsenNZ Jul 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the release upgrade testing, we would modify it to ensure that the correct product versions are used, and the SDP X.XX version is used (not 0.0.0-dev).

On top of that, I think the demo should be run (according to the docs), and then the upgrade should be done. This mimics a real upgrade where there was data.

Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Run and Update Demos on Nightly for Stackable Release XX.(X)X

<!--
Make sure to update the link in 'issues/.github/ISSUE_TEMPLATE/release.md'
when you rename this file.
-->

<!--
Replace 'TRACKING_ISSUE' with the applicable release tracking issue number.
-->

Part of <https://github.com/stackabletech/issues/issues/TRACKING_ISSUE>

> [!NOTE]
> Demos run on the latest stable release by default. This PR includes changes
> to commands, documentation and UI screenshots when running the demos on
> nightly instead. The recommended workflow looks like this:
>
> 1. Create a `.env` file in your local copy of this repository.
> 2. Create local stack and demo definition files, like `stacks-v2.local.yaml`
> and `demos-v2.local.yaml`. They should use the nightly version instead of
> stable. They also might need to point to local manifests instead of remote
> ones.
> 3. Add the `STACKABLE_STACK_FILES` and `STACKABLE_DEMO_FILES` env vars to the
> file. They should point to the absolute path of your local stack and demo
> definition files.
Comment on lines +19 to +26
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just adding a different perspective to consider...

Tbh, I prefer using the checked out stack files explicitly on the command line.
The same changes get made, as you described, but if there are actual fixes needed, it is a matter of editing the same files and only checking in those changes (and reverting the other ones that get it working with nightly).

Not at all suggesting my way is better, but for me, doing the above, I would forget to unset the variable, or I would run a new shell that doesn't have it set, and invalidate the testing). I use the shell history heavily.

> 4. Install the demo as usual ` stackablectl demo install <DEMO>`.
> 5. Update the documentation and images when required.

<!--
The following list was generated by:

yq '.demos | keys' demos/demos-v2.yaml \
| sed -e 's/- //g' \
| sort \
| xargs -I {} echo "- [ ] [{}](https://docs.stackable.tech/home/nightly/demos/{})"
-->

```[tasklist]
### Demos
- [ ] [airflow-scheduled-job](https://docs.stackable.tech/home/nightly/demos/airflow-scheduled-job)
- [ ] [data-lakehouse-iceberg-trino-spark](https://docs.stackable.tech/home/nightly/demos/data-lakehouse-iceberg-trino-spark)
- [ ] [end-to-end-security](https://docs.stackable.tech/home/nightly/demos/end-to-end-security)
- [ ] [hbase-hdfs-load-cycling-data](https://docs.stackable.tech/home/nightly/demos/hbase-hdfs-load-cycling-data)
- [ ] [jupyterhub-pyspark-hdfs-anomaly-detection-taxi-data](https://docs.stackable.tech/home/nightly/demos/jupyterhub-pyspark-hdfs-anomaly-detection-taxi-data)
- [ ] [logging](https://docs.stackable.tech/home/nightly/demos/logging)
- [ ] [nifi-kafka-druid-earthquake-data](https://docs.stackable.tech/home/nightly/demos/nifi-kafka-druid-earthquake-data)
- [ ] [nifi-kafka-druid-water-level-data](https://docs.stackable.tech/home/nightly/demos/nifi-kafka-druid-water-level-data)
- [ ] [signal-processing](https://docs.stackable.tech/home/nightly/demos/signal-processing)
- [ ] [spark-k8s-anomaly-detection-taxi-data](https://docs.stackable.tech/home/nightly/demos/spark-k8s-anomaly-detection-taxi-data)
- [ ] [trino-iceberg](https://docs.stackable.tech/home/nightly/demos/trino-iceberg)
- [ ] [trino-taxi-data](https://docs.stackable.tech/home/nightly/demos/trino-taxi-data)
```
9 changes: 8 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,8 @@
.env
# Environment variables, for example for stackablectl
.env

# Local demo and stack definition files
stacks-v1.local.yaml
stacks-v2.local.yaml
demos-v1.local.yaml
demos-v2.local.yaml