Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: adds versions description and upgrade guide #36

Merged
merged 3 commits into from
Jan 25, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@ on:
push:
branches:
- main
pull_request:
workflow_dispatch:

permissions:
contents: read
Expand Down
7 changes: 6 additions & 1 deletion .vitepress/config.mts
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ export default defineConfig({
{ text: "Tasks", link: "/language-guide/tasks.md" },
{ text: "Workflows", link: "/language-guide/workflows.md" },
{ text: "Imports", link: "/language-guide/imports.md" },
{ text: "Versions", link: "/language-guide/versions.md" },
],
},
{
Expand All @@ -94,7 +95,11 @@ export default defineConfig({
text: "Reference",
items: [
{
text: "Standard Library",
text: "Upgrade guide",
link: "/reference/upgrade-guide"
},
{
text: "Standard library",
items: [
{
text: "Numeric functions",
Expand Down
54 changes: 54 additions & 0 deletions language-guide/versions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Verisons

The Workflow Description Language has two concepts of versions within the project:

* The WDL _language_ has a two-number version (e.g., `1.2`). An increase in the
minor (second) version number (e.g., `1.1` to `1.2`) indicates the addition
of, or non-breaking changes to, the language or standard library functions. An
increase in the major (first) version number (e.g., `1.0` to `2.0`) indicates
that breaking changes have been made.

* The WDL _specification_ has a three-number version (e.g., `1.2.0`). The
specification version tracks the language version, but there may also be patch
releases (indicated by a change to the patch, or third, version number) that
include fixes for typos, additional examples, or non-breaking clarifications
of ambiguous language.

In general, users of WDL only need to care about the version of the WDL
_language_—you'll rarely, if ever, need to care about the version of the
_specification_ itself.

## Upgrading

If you're interested in learning more about the finer details of upgrading
between versions, such as what features new versions introduce, common pitfalls
of upgrading, and how to get help, see the [Upgrade guide] in the Reference section.

## Version specification

The WDL version is a statement that appears at the top of a WDL document—when included,
it **must** be the first non-comment within the document.

:::tip NOTE
Technically, the version statement is not _required_, but it is highly recommended that
you do include it in all of your WDL documents. Omission of the version statement
defaults to a very early version of the WDL specification (`draft-2`).
:::


You can specify the version of your WDL document like so:

```wdl
version 1.2

# ... other document contents ...
```

## Compatability considerations

Documents may only import other WDL documents of the same version. This is
because the imported documents are effectively comingled within their importer's
context and processed holistically (instead of, for example, being compiled
independently).

[Upgrade guide]: ../reference/upgrade-guide.md
164 changes: 164 additions & 0 deletions reference/upgrade-guide.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
---
outline: [1, 2]
---

# Overview

The Workflow Description Language has a number of **versions** that are
constantly improving. This guide lists all major versions of the language—the
major improvements for each version, common pitfalls when upgrading, and how to
get help.

If you're interested in learning more about how versioning works and how you can
specify the version of your WDL documents, see the [Versioning section] within
the Language Guide.

::: tip
Changes are grouped in terms of their **impact**. The word "impact" here is a
combination of "how important is the change" along with "how many existing WDL
documents are likely to be affected by this change". They are subjective
determinations made by the upgrade guide curator during the curator process, but
they are often helpful in quickly scanning upgrade elements to see which might
apply to your situation.
:::

# WDL v1.2

WDL v1.2 introduces a great deal of new syntax and changes to the specification.
Notably, changes were backwards compatible with all previous `v1.x` releases.
This sections only covers the high points related to upgrading—the full release
notes can be found [here][wdl-v1.2-release-notes].

### Checklist

Use this checklist to ensure you hit all of the sections.

#### Moderate impact changes

<input type="checkbox" /> New `Directory` type ([link](#new-directory-type)).<br />
<input type="checkbox" /> Replacement of `runtime` section with `requirements` and `hints` sections ([link](#replacement-of-runtime-with-requirements-and-hints)). <br />
<input type="checkbox" /> Deprecation of `runtime` section ([link](#deprecation-of-runtime-section)). <br />
<input type="checkbox" /> New standard library functions
([link](#new-standard-library-functions)). <br />
<input type="checkbox" /> Multi-line strings ([link](#multi-line-strings)). <br />
<input type="checkbox" /> Optional `input:` statement ([link](#optional-input-statement)). <br />

#### Low impact changes

<input type="checkbox" /> Addition of workflow `hints` ([link](#addition-of-workflow-hints)). <br />
<input type="checkbox" /> New requirements and hints keys ([link](#new-requirements-and-hints-keys)). <br />
<input type="checkbox" /> Metadata sections for structs ([link](#metadata-sections-for-structs)). <br />
<input type="checkbox" /> Exponentiation operator ([link](#exponentiation-operator)). <br />

## Moderate impact changes

### New `Directory` type

The `Directory` type was introduced in
[#641](https://github.com/openwdl/wdl/pull/641) to better semantically indicate
the use of a directory. If the intention of any of your arguments is to be used
to refer to a directory on the filesystem, you are encouraged to update the
parameters to a `Directory` type.

### Deprecation of `runtime` section

The `runtime` section, which previously held both requirement constraints and
hints to the execution engine, has now been split into the `requirements`
([#540](https://github.com/openwdl/wdl/issues/540)) section and `hints`
([#541](https://github.com/openwdl/wdl/issues/541)) section respectively. You
should split out these keys based on the definitions in the
specification<sup>[1](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-requirements-section),
[2](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-hints-section)</sup>.

🗑️ This change deprecates the `runtime` section, which will be removed in WDL v2.0.
That being said, if desired, you can continue to use the `runtime` section in your WDL
documents—it's just not recommended.

### New standard library functions

The following are new standard library functions and their definitions. You are
encouraged to read through them and replace any custom functionality that would
now be duplicated in favor of these functions.

- `contains_key`: whether or not a `Map` or `Object` contain a specific member
([link](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-contains_key)).
- `values`: get the values from a `Map`
([link](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-values)).
- `find`: search for a regular expression in a string
([link](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-find)).
- `matches`: whether a string match a regular expression
([link](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-matches)).
- `chunk`: split an array into sub-arrays
([link](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-chunk)).
- `join_paths`: join two or more paths
([link](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-join_paths)).
- `contains`: whether an array contains a specified value
([link](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-contains)).


### Multi-line strings

Multi-line strings were introduced with the following syntax.

```wdl
String foo = <<<
my
multi-line
string
>>>
```

### Optional `input:` statement

As noted in [#524](https://github.com/openwdl/wdl/pull/524), the `input:`
statement that precedes call bodies is unnecessary historical boilerplate. This
statement is now optional in `call` bodies. You are encouraged to remove these
from your `call` bodies.

## Low impact changes

### Addition of workflow `hints`

Workflows gained a `hints` section in
[#543](https://github.com/openwdl/wdl/issues/543). You are encouraged to go read
the supported workflow
hints<sup>[1](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#workflow-hints)</sup>
and apply them to your workflow if any are relevant.

### New requirements and hints keys

- The following keys were added to the task `requirements` section: `fpga`
<sup>[1](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#hardware-accelerators-gpu-and--fpga)</sup>.
- The following keys were added to the task `hints` section: `disks`
<sup>[1](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-disks)</sup>,
`gpu`
<sup>[2](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-gpu-and--fpga)</sup>,
and `fpga`
<sup>[2](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-gpu-and--fpga)</sup>.

You are encouraged to examine these keys are determine if any of these should be
specified for your tasks.

### Metadata sections for structs

Similarly to tasks and workflows, structs now have `meta` and `parameter_meta`
sections. You are encouraged to use these fields according to the definition in
the
specification<sup>[1](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#struct-definition)</sup>.

### Exponentiation operator

The exponentiation operator (`**`) was added in this release. You are encouraged
to go and update any manual exponentiation to use this operator instead.

You are encouraged to go read the specification section on this concept
([link](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#multi-line-strings))
and use them where appropriate.

# Prior WDL versions

Versions of WDL prior to the ones outlined in this guide did not exist at the
time the upgrade guide was created. As such, they are not included in the guide.

[wdl-v1.2-release-notes]: https://github.com/openwdl/wdl/releases/tag/v1.2.0
[Versioning section]: ../language-guide/versions.md