Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/json schema for workflows #159

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,13 @@ jobs:
with:
command: clippy
args: --tests -- -D warnings

- name: Install JSON schema validator
run: pip3 install jsonschema-cli

- name: Run jsonschema-cli validate
run: |
for file in specs/*/*.yaml; do
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we support both yaml and yml? IMO we should be flexible and support both (this is what Github actoins does as well, fwiw https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#about-yaml-syntax-for-workflows)

echo "Validating '$file'..."
jsonschema-cli validate ./schemas/workflow.json "$file"
done
5 changes: 5 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"yaml.schemas": {
"schemas/workflow.json": "specs/*/*.yaml"
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: could we end all files with a new line

142 changes: 142 additions & 0 deletions schemas/workflow.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
{
"$schema": "http://json-schema.org/draft-07/schema",
"definitions": {
"url": {
"oneOf": [
{
"type": "string",
"minLength": 1,
"pattern": "^https?://.+"
},
{
"type": "null"
}
]
},
"nullable-string": {
"oneOf": [
{
"type": "string",
"minLength": 1
},
{
"type": "null"
}
]
}
},
"title": "workflow",
"description": "A workflow",
"type": "object",
"required": [
"name",
"command"
],
"properties": {
"name": {
"title": "name",
"description": "A name of the current workflow\nhttps://github.com/EmilySeville7cfg/warp-workflows/blob/main/FORMAT.md#name",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you mind updating all of these links to point at the warpdotdev repo instead of your fork of this repo?https://github.com/warpdotdev/workflows/blob/main/FORMAT.md

"type": "string",
"minLength": 1,
"examples": [
"Root your emulator"
]
},
"command": {
"title": "command",
"description": "A command of the current workflow\nhttps://github.com/EmilySeville7cfg/warp-workflows/blob/main/FORMAT.md#command",
"type": "string",
"minLength": 1,
"examples": [
"adb root"
]
},
"tags": {
"title": "tags",
"description": "Tags of the current workflow\nhttps://github.com/EmilySeville7cfg/warp-workflows/blob/main/FORMAT.md#tags",
"type": "array",
"uniqueItems": true,
"items": {
"description": "A tag of the current workflow\nhttps://github.com/EmilySeville7cfg/warp-workflows/blob/main/FORMAT.md#tags",
"type": "string",
"minLength": 1,
"examples": [
"android"
]
}
},
"description": {
"title": "description",
"description": "A description of the current workflow\nhttps://github.com/EmilySeville7cfg/warp-workflows/blob/main/FORMAT.md#description",
"$ref": "#/definitions/nullable-string"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm by no means a JSON schema expert, but could you clarify why we need the nullable-string definition here? Why can't we use JSON schema's support for Required properties https://json-schema.org/understanding-json-schema/reference/object.html#required-properties

},
"source_url": {
"title": "source url",
"description": "A source url of the current workflow\nhttps://github.com/EmilySeville7cfg/warp-workflows/blob/main/FORMAT.md#source_url",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you mind using the description that was originally used in the FORMAT.md file? "The URL from where the Workflow was originally generated from. This is surfaced in commands.dev for attribution purposes"

"$ref": "#/definitions/url"
},
"author": {
"title": "author",
"description": "An author of the current workflow\nhttps://github.com/EmilySeville7cfg/warp-workflows/blob/main/FORMAT.md#author",
"$ref": "#/definitions/nullable-string"
},
"author_url": {
"title": "author url",
"description": "An author url of the current workflow\nhttps://github.com/EmilySeville7cfg/warp-workflows/blob/main/FORMAT.md#author_url",
"$ref": "#/definitions/url"
},
"shells": {
"title": "shells",
"description": "Shells where the current workflow is valid\nhttps://github.com/EmilySeville7cfg/warp-workflows/blob/main/FORMAT.md#shells",
"type": "array",
"uniqueItems": true,
"items": {
"description": "A shell where the current workflow is valid\nhttps://github.com/EmilySeville7cfg/warp-workflows/blob/main/FORMAT.md#shells",
"type": "string",
"enum": [
"zsh",
"bash",
"fish"
Comment on lines +96 to +99
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The existing format accepts either zsh or Zsh

#[serde(alias = "fish")]
Fish,
#[serde(alias = "bash")]
Bash,
#[serde(alias = "zsh")]
Zsh,
.

I think we should make that explicit in this format because there could be existing workflow yaml files in the wild that use the uppercase format.

]
}
},
"arguments": {
"title": "arguments",
"description": "Arguments of the current workflow\nhttps://github.com/EmilySeville7cfg/warp-workflows/blob/main/FORMAT.md#arguments",
"type": "array",
"uniqueItems": true,
"items": {
"description": "An argument of the current workflow\nhttps://github.com/EmilySeville7cfg/warp-workflows/blob/main/FORMAT.md#arguments",
"type": "object",
"required": [
"name"
],
"properties": {
"name": {
"title": "name",
"description": "A name of the current argument\nhttps://github.com/EmilySeville7cfg/warp-workflows/blob/main/FORMAT.md#argumentsname",
"type": "string",
"minLength": 1
},
"description": {
"title": "description",
"description": "A description of the current argument\nhttps://github.com/EmilySeville7cfg/warp-workflows/blob/main/FORMAT.md#argumentsdescription",
"$ref": "#/definitions/nullable-string"
},
"default_value": {
"title": "default value",
"description": "A default value of the current argument\nhttps://github.com/EmilySeville7cfg/warp-workflows/blob/main/FORMAT.md#argumentsdefault_value",
"type": [
"boolean",
"number",
"string",
"null"
]
}
},
"additionalProperties": false
}
}
},
"additionalProperties": false
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we also end this file with a newline?

38 changes: 25 additions & 13 deletions script/bootstrap
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,28 @@

set -e

if ! command -v brew; then
echo "Installing brew..."
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
echo "Please make sure brew is set correctly in your PATH"
exit 1
fi

if ! command -v cargo; then
echo "Installing rust..."
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
echo "Please start a new terminal session so that cargo is in your PATH"
exit 1
fi
reset='\e[0m'
red='\e[31m'
green='\e[32m'
cyan='\e[36m'

install_dependency() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Would you mind documenting each argument to this function?

id_dependency="$1"
id_description="$2"
id_installation_command="$3"

if ! command -v "$id_dependency" > /dev/null; then
printf "${green}Installing '$red%s$green' ('$cyan%s$green' binary)\n$reset" "$id_description" "$id_dependency"
sh -c "$id_installation_command"
printf "${green}Installed '$cyan%s$green' successfully.
Please make sure '$cyan%s$green' binary is in your \$PATH.\n$reset" "$id_dependency" "$id_dependency"
exit 1
fi
}

# shellcheck disable=SC2016
install_dependency brew Homebrew '/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"'
install_dependency rustc Rust 'curl --proto "=https" --tlsv1.2 -sSf https://sh.rustup.rs | sh'

cargo install cargo-bundle
cargo install cargo-diff-tools --git https://github.com/warpdotdev/cargo-diff-tools
Expand All @@ -23,3 +32,6 @@ cargo install cargo-diff-tools --git https://github.com/warpdotdev/cargo-diff-to
brew update

rustup component add clippy

install_dependency python3-pip 'Python package manager' 'sudo apt install python3-pip'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should avoid sudo apt install here, the current script/bootstrap is intended to run on Mac (which is why we install brew).

Also, I'm curious your thoughts on whether we should use https://www.npmjs.com/package/ajv-cli instead. It's the one of the official CLI JSON validators mentioned by the JSON schema org https://json-schema.org/implementations.html.

install_dependency jsonschema-cli 'JSON schema validator' 'pip3 install jsonschema-cli'
12 changes: 9 additions & 3 deletions script/presubmit
Original file line number Diff line number Diff line change
@@ -1,17 +1,18 @@
#!/bin/bash
#!/bin/sh

# Script to run presubmit checks. Devs can run this script locally before sending out PRs for review to ensure
# CI is passing for their PR.

set -e
cd "$(dirname "$0")/.."

export -f f
FMT_COMMAND="cargo fmt"
echo "Running $FMT_COMMAND..."
set -e
EXIT_CODE=0
# shellcheck enable=SC2000
$FMT_COMMAND -- --check || EXIT_CODE=$?
if [[ $EXIT_CODE -ne 0 ]]; then
if [ $EXIT_CODE -ne 0 ]; then
echo 'Run `'$FMT_COMMAND'` to fix.'
exit $EXIT_CODE
fi
Expand All @@ -28,4 +29,9 @@ echo "Running cargo test..."
cargo test
echo "Tests succeeded..."

for file in specs/*/*.yaml; do
echo "Validating '$file'..."
jsonschema-cli validate ./schemas/workflow.json "$file"
done

echo "Congrats! All presubmits checks passed."
File renamed without changes.
File renamed without changes.
2 changes: 1 addition & 1 deletion specs/git/rebase_master_into_current_branch.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ tags:
- git
description: Rebases master into the feature branch
arguments: []
source_url: ""
source_url: ~
author: Varun Jindal
author_url: "https://www.linkedin.com/in/varun-jindal/"
shells: []
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
4 changes: 2 additions & 2 deletions specs/shell/array_append_values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,5 @@ source_url: "https://opensource.com/article/18/5/you-dont-know-bash-intro-bash-a
author: Wyatt-Stanke
author_url: "https://github.com/Wyatt-Stanke"
shells:
- Zsh
- Bash
- zsh
- bash
4 changes: 2 additions & 2 deletions specs/shell/array_create.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ source_url: "https://opensource.com/article/18/5/you-dont-know-bash-intro-bash-a
author: Wyatt-Stanke
author_url: "https://github.com/Wyatt-Stanke"
shells:
- Zsh
- Bash
- zsh
- bash
4 changes: 2 additions & 2 deletions specs/shell/array_get_size.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ source_url: "https://opensource.com/article/18/5/you-dont-know-bash-intro-bash-a
author: Wyatt-Stanke
author_url: "https://github.com/Wyatt-Stanke"
shells:
- Zsh
- Bash
- zsh
- bash
4 changes: 2 additions & 2 deletions specs/shell/array_get_value.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,5 @@ source_url: "https://opensource.com/article/18/5/you-dont-know-bash-intro-bash-a
author: Wyatt-Stanke
author_url: "https://github.com/Wyatt-Stanke"
shells:
- Zsh
- Bash
- zsh
- bash
4 changes: 2 additions & 2 deletions specs/shell/array_loop_through.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,5 @@ source_url: "https://opensource.com/article/18/5/you-dont-know-bash-intro-bash-a
author: Wyatt-Stanke
author_url: "https://github.com/Wyatt-Stanke"
shells:
- Zsh
- Bash
- zsh
- bash
4 changes: 2 additions & 2 deletions specs/shell/array_set_value.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,5 +18,5 @@ source_url: "https://opensource.com/article/18/5/you-dont-know-bash-intro-bash-a
author: Wyatt-Stanke
author_url: "https://github.com/Wyatt-Stanke"
shells:
- Zsh
- Bash
- zsh
- bash
4 changes: 2 additions & 2 deletions specs/shell/chain_commands_together_pipe.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,5 @@ source_url: "https://tldp.org/LDP/abs/html/io-redirection.html"
author: Mendel Cooper
author_url: ~
shells:
- Zsh
- Bash
- zsh
- bash
4 changes: 2 additions & 2 deletions specs/shell/check_if_a_file_exists.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ source_url: "https://linuxhint.com/bash-test-command/"
author: Sidratul Muntaha
author_url: "https://linuxhint.com/author/lh_sidratul_muntaha/"
shells:
- Zsh
- Bash
- zsh
- bash
4 changes: 2 additions & 2 deletions specs/shell/check_if_a_file_exists_and_is_a_regular_file.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ source_url: "https://linuxhint.com/bash-test-command/"
author: Sidratul Muntaha
author_url: "https://linuxhint.com/author/lh_sidratul_muntaha/"
shells:
- Zsh
- Bash
- zsh
- bash
4 changes: 2 additions & 2 deletions specs/shell/check_if_a_file_exists_and_is_directory.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ source_url: "https://linuxhint.com/bash-test-command/"
author: Sidratul Muntaha
author_url: "https://linuxhint.com/author/lh_sidratul_muntaha/"
shells:
- Zsh
- Bash
- zsh
- bash
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,5 @@ source_url: "https://linuxhint.com/bash-test-command/"
author: Sidratul Muntaha
author_url: "https://linuxhint.com/author/lh_sidratul_muntaha/"
shells:
- Zsh
- Bash
- zsh
- bash
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,5 @@ source_url: "https://linuxhint.com/bash-test-command/"
author: Sidratul Muntaha
author_url: "https://linuxhint.com/author/lh_sidratul_muntaha/"
shells:
- Zsh
- Bash
- zsh
- bash
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,5 @@ source_url: "https://linuxhint.com/bash-test-command/"
author: Sidratul Muntaha
author_url: "https://linuxhint.com/author/lh_sidratul_muntaha/"
shells:
- Zsh
- Bash
- zsh
- bash
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,5 @@ source_url: "https://linuxhint.com/bash-test-command/"
author: Sidratul Muntaha
author_url: "https://linuxhint.com/author/lh_sidratul_muntaha/"
shells:
- Zsh
- Bash
- zsh
- bash
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ source_url: "https://linuxhint.com/bash-test-command/"
author: Sidratul Muntaha
author_url: "https://linuxhint.com/author/lh_sidratul_muntaha/"
shells:
- Zsh
- Bash
- zsh
- bash
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ source_url: "https://linuxhint.com/bash-test-command/"
author: Sidratul Muntaha
author_url: "https://linuxhint.com/author/lh_sidratul_muntaha/"
shells:
- Zsh
- Bash
- zsh
- bash
Loading