Skip to content
Merged
6 changes: 6 additions & 0 deletions api/openapi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5992,6 +5992,12 @@ components:
timestamp:
description: Timestamp of workflow creation
type: string
project:
description: Project name or identifier for grouping workflows
type: string
metadata:
description: Arbitrary metadata as JSON string
type: string

compute_node_expiration_buffer_seconds:
default: 180
Expand Down
40 changes: 40 additions & 0 deletions docs/src/core/reference/workflow-spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ The top-level container for a complete workflow definition.
| `name` | string | _required_ | Name of the workflow |
| `user` | string | current user | User who owns this workflow |
| `description` | string | none | Description of the workflow |
| `project` | string | none | Project name or identifier for grouping workflows |
| `metadata` | string | none | Arbitrary metadata as JSON string |
| `parameters` | map\<string, string\> | none | Shared parameters that can be used by jobs and files via `use_parameters` |
| `jobs` | [[JobSpec](#jobspec)] | _required_ | Jobs that make up this workflow |
| `files` | [[FileSpec](#filespec)] | none | Files associated with this workflow |
Expand All @@ -29,6 +31,44 @@ The top-level container for a complete workflow definition.
| `compute_node_wait_for_healthy_database_minutes` | integer | none | Compute nodes wait this many minutes for database recovery |
| `jobs_sort_method` | [ClaimJobsSortMethod](#claimjobssortmethod) | `none` | Method for sorting jobs when claiming them |

### Examples with project and metadata

The `project` and `metadata` fields are useful for organizing and categorizing workflows. For more
detailed guidance on organizing workflows, see
[Organizing and Managing Workflows](../workflows/organizing-workflows.md).

**YAML example:**

```yaml
name: "ml_training_workflow"
project: "customer-churn-prediction"
metadata: '{"environment":"staging","version":"1.0.0","team":"ml-engineering"}'
description: "Train and evaluate churn prediction model"
jobs:
- name: "preprocess"
command: "python preprocess.py"
- name: "train"
command: "python train.py"
depends_on: ["preprocess"]
```

**JSON example:**

```json
{
"name": "data_pipeline",
"project": "analytics-platform",
"metadata": "{\"cost_center\":\"eng-data\",\"priority\":\"high\"}",
"description": "Daily data processing pipeline",
"jobs": [
{
"name": "extract",
"command": "python extract.py"
}
]
}
```

## JobSpec

Defines a single computational task within a workflow.
Expand Down
7 changes: 7 additions & 0 deletions docs/src/core/workflows/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,15 @@

This section covers how to create, configure, and manage workflows.

## Creating and Formatting

- [Creating Workflows](./creating-workflows.md) - Getting started with workflow creation
- [Workflow Specification Formats](./workflow-formats.md) - JSON, YAML, and other formats

## Organization and Management

- [Organizing and Managing Workflows](./organizing-workflows.md) - Using project and metadata fields
to categorize and track workflows
- [Visualizing Workflow Structure](./visualizing-workflows.md) - Viewing workflow graphs
- [Exporting and Importing Workflows](./export-import-workflows.md) - Moving workflows between
systems
Expand Down
Loading
Loading