You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/en/guides/40-load-data/05-continuous-data-pipelines/02-task.md
+35-81Lines changed: 35 additions & 81 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,90 +3,44 @@ title: Automating Data Loading with Tasks
3
3
sidebar_label: Task
4
4
---
5
5
6
-
A task encapsulates specific SQL statements that are designed to be executed either at predetermined intervals, triggered by specific events, or as part of a broader sequence of tasks. Tasks in Databend Cloud are commonly used to regularly capture data changes from streams, such as newly added records, and then synchronize this data with designated target destinations. Furthermore, tasks offer support for [Webhook](https://en.wikipedia.org/wiki/Webhook) and other messaging systems, facilitating the delivery of error messages and notifications as needed.
7
-
8
-
## Task Building Blocks
9
-
10
-
Create tasks with the [CREATE TASK](/sql/sql-commands/ddl/task/ddl-create_task) command. The illustration below shows how the clauses combine into a workflow:
6
+
Tasks wrap SQL so Databend can run it for you on a schedule or when a condition is met. Keep the following knobs in mind when you define one with [CREATE TASK](/sql/sql-commands/ddl/task/ddl-create_task):
11
7
12
8

13
9
14
-
When defining a task, decide on the following building blocks:
15
-
16
-
1.**Task name & warehouse** – Every task runs on a warehouse. To create or resize a warehouse, see [Work with Warehouses](/guides/cloud/using-databend-cloud/warehouses).
17
-
18
-
```sql title='Example: set the identity of a task'
19
-
CREATE TASK ingest_orders
20
-
WAREHOUSE ='etl_wh'
21
-
AS
22
-
SELECT1;
23
-
```
24
-
25
-
2.**Trigger** – Choose a fixed schedule or make the task depend on another task.
26
-
27
-
```sql title='Examples: schedule options'
28
-
-- Run every 2 minutes
29
-
CREATE TASK mytask
30
-
WAREHOUSE ='default'
31
-
// highlight-next-line
32
-
SCHEDULE =2 MINUTE
33
-
AS ...;
34
-
35
-
-- Run daily at midnight in Asia/Tokyo
36
-
CREATE TASK mytask
37
-
WAREHOUSE ='default'
38
-
// highlight-next-line
39
-
SCHEDULE = USING CRON '0 0 0 * * *''Asia/Tokyo'
40
-
AS ...;
41
-
42
-
-- Run after another task in a DAG
43
-
CREATE TASK mytask
44
-
WAREHOUSE ='default'
45
-
// highlight-next-line
46
-
AFTER task_root
47
-
AS ...;
48
-
```
49
-
50
-
3.**Optional guard** – Gate the execution with a boolean expression such as `STREAM_STATUS`.
51
-
52
-
```sql title='Example: only run when a stream has rows'
53
-
CREATE TASK mytask
54
-
WAREHOUSE ='default'
55
-
SCHEDULE =2 MINUTE
56
-
// highlight-next-line
57
-
WHEN STREAM_STATUS('mystream') = TRUE
58
-
AS ...;
59
-
```
60
-
61
-
4.**Error handling** – Suspend the task after repeated failures or route errors to an integration.
62
-
63
-
```sql title='Examples: guard against failures'
64
-
CREATE TASK mytask
65
-
WAREHOUSE ='default'
66
-
SCHEDULE =5 MINUTE
67
-
// highlight-next-line
68
-
SUSPEND_TASK_AFTER_NUM_FAILURES =3
69
-
AS ...;
70
-
71
-
CREATE TASK mytask
72
-
WAREHOUSE ='default'
73
-
SCHEDULE =5 MINUTE
74
-
// highlight-next-line
75
-
ERROR_INTEGRATION ='my_webhook'
76
-
AS ...;
77
-
```
78
-
79
-
5.**SQL payload** – Provide the statements you want the task to run.
80
-
81
-
```sql title='Example: run an update every year'
82
-
CREATE TASK bump_age
83
-
WAREHOUSE ='default'
84
-
SCHEDULE = USING CRON '0 0 1 1 * *''UTC'
85
-
// highlight-next-line
86
-
AS
87
-
UPDATE employees
88
-
SET age = age +1;
89
-
```
10
+
-**Name & warehouse** – every task needs a warehouse.
0 commit comments