Skip to content

Commit bd5bd2d

Browse files
committed
docs: streamline stream/task intros
1 parent 19db154 commit bd5bd2d

File tree

2 files changed

+36
-82
lines changed

2 files changed

+36
-82
lines changed

docs/en/guides/40-load-data/05-continuous-data-pipelines/01-stream.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ FROM sensor_readings_stream WITH CONSUME;
6565
SELECT * FROM sensor_readings_stream; -- now empty
6666
```
6767

68-
`WITH CONSUME` 只读取一次并清空增量,便于下一轮继续捕获新的 INSERT。
68+
`WITH CONSUME` reads the stream once and clears the delta so the next round can capture fresh INSERTs.
6969

7070
## Example 2: Standard Stream (Updates & Deletes)
7171

docs/en/guides/40-load-data/05-continuous-data-pipelines/02-task.md

Lines changed: 35 additions & 81 deletions
Original file line numberDiff line numberDiff line change
@@ -3,90 +3,44 @@ title: Automating Data Loading with Tasks
33
sidebar_label: Task
44
---
55

6-
A task encapsulates specific SQL statements that are designed to be executed either at predetermined intervals, triggered by specific events, or as part of a broader sequence of tasks. Tasks in Databend Cloud are commonly used to regularly capture data changes from streams, such as newly added records, and then synchronize this data with designated target destinations. Furthermore, tasks offer support for [Webhook](https://en.wikipedia.org/wiki/Webhook) and other messaging systems, facilitating the delivery of error messages and notifications as needed.
7-
8-
## Task Building Blocks
9-
10-
Create tasks with the [CREATE TASK](/sql/sql-commands/ddl/task/ddl-create_task) command. The illustration below shows how the clauses combine into a workflow:
6+
Tasks wrap SQL so Databend can run it for you on a schedule or when a condition is met. Keep the following knobs in mind when you define one with [CREATE TASK](/sql/sql-commands/ddl/task/ddl-create_task):
117

128
![alt text](/img/load/task.png)
139

14-
When defining a task, decide on the following building blocks:
15-
16-
1. **Task name & warehouse** – Every task runs on a warehouse. To create or resize a warehouse, see [Work with Warehouses](/guides/cloud/using-databend-cloud/warehouses).
17-
18-
```sql title='Example: set the identity of a task'
19-
CREATE TASK ingest_orders
20-
WAREHOUSE = 'etl_wh'
21-
AS
22-
SELECT 1;
23-
```
24-
25-
2. **Trigger** – Choose a fixed schedule or make the task depend on another task.
26-
27-
```sql title='Examples: schedule options'
28-
-- Run every 2 minutes
29-
CREATE TASK mytask
30-
WAREHOUSE = 'default'
31-
// highlight-next-line
32-
SCHEDULE = 2 MINUTE
33-
AS ...;
34-
35-
-- Run daily at midnight in Asia/Tokyo
36-
CREATE TASK mytask
37-
WAREHOUSE = 'default'
38-
// highlight-next-line
39-
SCHEDULE = USING CRON '0 0 0 * * *' 'Asia/Tokyo'
40-
AS ...;
41-
42-
-- Run after another task in a DAG
43-
CREATE TASK mytask
44-
WAREHOUSE = 'default'
45-
// highlight-next-line
46-
AFTER task_root
47-
AS ...;
48-
```
49-
50-
3. **Optional guard** – Gate the execution with a boolean expression such as `STREAM_STATUS`.
51-
52-
```sql title='Example: only run when a stream has rows'
53-
CREATE TASK mytask
54-
WAREHOUSE = 'default'
55-
SCHEDULE = 2 MINUTE
56-
// highlight-next-line
57-
WHEN STREAM_STATUS('mystream') = TRUE
58-
AS ...;
59-
```
60-
61-
4. **Error handling** – Suspend the task after repeated failures or route errors to an integration.
62-
63-
```sql title='Examples: guard against failures'
64-
CREATE TASK mytask
65-
WAREHOUSE = 'default'
66-
SCHEDULE = 5 MINUTE
67-
// highlight-next-line
68-
SUSPEND_TASK_AFTER_NUM_FAILURES = 3
69-
AS ...;
70-
71-
CREATE TASK mytask
72-
WAREHOUSE = 'default'
73-
SCHEDULE = 5 MINUTE
74-
// highlight-next-line
75-
ERROR_INTEGRATION = 'my_webhook'
76-
AS ...;
77-
```
78-
79-
5. **SQL payload** – Provide the statements you want the task to run.
80-
81-
```sql title='Example: run an update every year'
82-
CREATE TASK bump_age
83-
WAREHOUSE = 'default'
84-
SCHEDULE = USING CRON '0 0 1 1 * *' 'UTC'
85-
// highlight-next-line
86-
AS
87-
UPDATE employees
88-
SET age = age + 1;
89-
```
10+
- **Name & warehouse** – every task needs a warehouse.
11+
```sql
12+
CREATE TASK ingest_orders
13+
WAREHOUSE = 'etl_wh'
14+
AS SELECT 1;
15+
```
16+
- **Trigger** – fixed interval, CRON, or `AFTER another_task`.
17+
```sql
18+
CREATE TASK mytask
19+
WAREHOUSE = 'default'
20+
SCHEDULE = 2 MINUTE
21+
AS ...;
22+
```
23+
- **Guards** – only run when a predicate is true.
24+
```sql
25+
CREATE TASK mytask
26+
WAREHOUSE = 'default'
27+
WHEN STREAM_STATUS('mystream') = TRUE
28+
AS ...;
29+
```
30+
- **Error handling** – pause after N failures or send notifications.
31+
```sql
32+
CREATE TASK mytask
33+
WAREHOUSE = 'default'
34+
SUSPEND_TASK_AFTER_NUM_FAILURES = 3
35+
AS ...;
36+
```
37+
- **SQL payload** – whatever you place after `AS` is what the task executes.
38+
```sql
39+
CREATE TASK bump_age
40+
WAREHOUSE = 'default'
41+
SCHEDULE = USING CRON '0 0 1 1 * *' 'UTC'
42+
AS UPDATE employees SET age = age + 1;
43+
```
9044

9145
## Example 1: Scheduled Copy
9246

0 commit comments

Comments
 (0)