Skip to content

Commit

Permalink
Examples roundtrip test (#760)
Browse files Browse the repository at this point in the history 
**Pull Request Checklist**
- [x] Another step towards #567 - we still need the actual Python
examples
- [x] Tests added (but some failing)
- [x] Documentation/examples added
- [x] [Good commit messages](https://cbea.ms/git-commit/) and/or PR
title

**Description of PR**
Adds a fetcher for all upstream files, then performs a
`Hera_class.from_yaml(file).to_yaml()` round trip to confirm the spec is
the same.

Also fixed borked workflows-examples.md page

Also removed unused `Inputable` protocol that I thought I removed in
#755 but I guess I forgot to commit

---------

Signed-off-by: Elliot Gunton <[email protected]>
Co-authored-by: Sambhav Kothari <[email protected]>
  • Loading branch information
@elliotgunton @sambhav
elliotgunton and sambhav authored Sep 2, 2023
1 parent 046b49f commit cd7cdad
Show file tree
Hide file tree
Showing 112 changed files with 5,566 additions and 18 deletions.
2 changes: 2 additions & 0 deletions docs/examples/workflows-examples.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,9 @@ The "Hera" collection shows off features and idiomatic usage of Hera such as the
Explore the examples through the side bar!

## List of **missing** examples

*You can help by contributing these examples!*

| Example |
|---------|
| [artifacts-workflowtemplate](https://github.com/argoproj/argo-workflows/blob/master/examples/artifacts-workflowtemplate.yaml) |
Expand Down
135 changes: 135 additions & 0 deletions examples/workflows/upstream/artifacts-workflowtemplate.upstream.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
name: artifacts
annotations:
workflows.argoproj.io/description: |
This example shows how to produce different types of artifact.
spec:
entrypoint: main
templates:
- name: main
volumes:
- name: in
emptyDir: { }
- name: out
emptyDir: { }
inputs:
artifacts:
- name: temps
path: /in/annual.csv
http:
url: https://datahub.io/core/global-temp/r/annual.csv
containerSet:
volumeMounts:
- mountPath: /in
name: in
- mountPath: /out
name: out
containers:
- name: setup
image: argoproj/argosay:v2
command:
- sh
- -c
args:
- |
mkdir -p /out/assets
- name: gnuplot
image: remuslazar/gnuplot
dependencies:
- setup
args:
- -e
- |
set xlabel 'Year'; set ylabel 'Mean';
set grid;
set datafile separator ',';
set term png size 600,400;
set output '/out/assets/global-temp.png';
plot '/in/annual.csv' every 2::0 skip 1 using 2:3 title 'Global Temperature' with lines linewidth 2;
- name: main
image: argoproj/argosay:v2
dependencies:
- setup
command:
- sh
- -c
args:
- |
cowsay "hello world" > /out/hello.txt
cat > /out/hello.json <<EOF
{"hello": {"world": true}}
EOF
echo '* {font-family: sans-serif}' > /out/assets/styles.css
cat > /out/index.html <<EOF
<html>
<head>
<link rel='stylesheet' href='assets/styles.css' type='text/css'/>
</head>
<body>
<h1>Global Temperature</h1>
<img src='assets/global-temp.png'/>
</body>
</html>
EOF
cat > /out/malicious.html <<EOF
<html>
<body>
<script>alert(1)</script>
<p>This page attempts to run a script that shows an alert, but the Argo Server UI Content-Security-Policy will prevent that.</p>
<p>To check, open your Web Console and see that "Blocked script execution ... because the document's frame is sandboxed." (or similar) is printed.</p>
</body>
</html>
EOF
outputs:
artifacts:
# saving single files
- name: text-file
path: /out/hello.txt
s3:
key: hello.txt
archive:
none: { }
# JSON files are shown with syntax highlighting.
- name: json-file
path: /out/hello.json
s3:
key: hello.json
archive:
none: { }
# CSS in not considered a known file type
- name: css-file
path: /out/assets/styles.css
s3:
key: styles.css
archive:
none: { }
# this artifact tries to run JavaScript
- name: malicious-file
path: /out/malicious.html
s3:
key: malicious.html
archive:
none: { }
# save a whole directory
- name: report
path: /out
s3:
key: report/
archive:
none: { }
# this will be a tgz with a single file
- name: tgz-file
path: /out/hello.txt
s3:
key: file.tgz
# this will be a tgz with two entries, one dir and one file
- name: tgz-dir
path: /out
s3:
key: dir.tgz
163 changes: 163 additions & 0 deletions examples/workflows/upstream/buildkit-template.upstream.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
# SUMMARY:
#
# Build and push an image using Docker Buildkit.
#
# DESCRIPTION:
#
# This does not need privileged access, unlike Docker in Docker (DIND). It has three stages:
#
# * clone the Git repository
# * build the binary
# * build and push the image containing the binary
#
# USAGE:
#
# Publishing images requires an access token. For hub.docker.com you can create one at https://hub.docker.com/settings/security
# This needs to be mounted as `$DOCKER_CONFIG/config.json`. To do this, you'll need to create a secret as follows:
#
# export DOCKER_USERNAME=******
# export DOCKER_TOKEN=******
# kubectl create secret generic docker-config --from-literal="config.json={\"auths\": {\"https://index.docker.io/v1/\": {\"auth\": \"$(echo -n $DOCKER_USERNAME:$DOCKER_TOKEN|base64)\"}}}"
#
# REFERENCES:
#
# * https://github.com/moby/buildkit#expose-buildkit-as-a-tcp-service
# * https://blog.alexellis.io/building-containers-without-docker/
# * https://hub.docker.com/r/moby/buildkit
#
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
name: buildkit
spec:
arguments:
parameters:
- name: repo
value: https://github.com/argoproj/argo-workflows
- name: branch
value: master
- name: path
value: test/e2e/images/argosay/v2
- name: image
value: alexcollinsintuit/argosay:v2
entrypoint: main
# We use a volume claim template so that we can have a shared workspace.
volumeClaimTemplates:
- metadata:
name: work
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 64Mi
templates:
- name: main
dag:
tasks:
- name: clone
template: clone
arguments:
parameters:
- name: repo
value: "{{workflow.parameters.repo}}"
- name: branch
value: "{{workflow.parameters.branch}}"
- name: build
template: build
arguments:
parameters:
- name: path
value: "{{workflow.parameters.path}}"
depends: "clone"
- name: image
template: image
arguments:
parameters:
- name: path
value: "{{workflow.parameters.path}}"
- name: image
value: "{{workflow.parameters.image}}"
depends: "build"
- name: clone
inputs:
parameters:
- name: repo
- name: branch
container:
volumeMounts:
- mountPath: /work
name: work
image: alpine/git:v2.26.2
workingDir: /work
# Do a shallow clone, which is the fastest way to clone, by using the
# --depth, --branch, and --single-branch options
args:
- clone
- --depth
- "1"
- --branch
- "{{inputs.parameters.branch}}"
- --single-branch
- "{{inputs.parameters.repo}}"
- .
- name: build
inputs:
parameters:
- name: path
container:
image: golang:1.13
volumeMounts:
- mountPath: /work
name: work
workingDir: /work/{{inputs.parameters.path}}
env:
# Because this is not a Gomodule, we must turn modules off.
- name: GO111MODULE
value: "off"
command:
- go
args:
- build
- -v
- -o
- argosay
- ./...
- name: image
inputs:
parameters:
- name: path
- name: image
# Mount the configuration so we can push the image.
# This should create the /.docker/config.json file.
volumes:
- name: docker-config
secret:
secretName: docker-config
container:
readinessProbe:
exec:
command: [ sh, -c, "buildctl debug workers" ]
image: moby/buildkit:v0.9.3-rootless
volumeMounts:
- name: work
mountPath: /work
- name: docker-config
mountPath: /.docker
workingDir: /work/{{inputs.parameters.path}}
env:
- name: BUILDKITD_FLAGS
value: --oci-worker-no-process-sandbox
- name: DOCKER_CONFIG
value: /.docker
command:
- buildctl-daemonless.sh
args:
- build
- --frontend
- dockerfile.v0
- --local
- context=.
- --local
- dockerfile=.
- --output
- type=image,name=docker.io/{{inputs.parameters.image}},push=true
Loading

0 comments on commit cd7cdad

Please sign in to comment.