Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add new page about node selection process #6655

Closed
wants to merge 6 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 82 additions & 0 deletions website/docs/reference/node-selection/node-selection.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
---
title: "Node selection"
description: "Learn how to select and exclude resources in dbt."
sidebar_label: "About node selection"
---

This page outlines the steps dbt follows when selecting resources. To understand dbt resource selection:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'd love to clarify if we're talking about nodes here or resources?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we're talking commands and flags, it seems like we mean resources adn node nodes?


- Think of `--select` and `--exclude` as two filtering steps applied sequentially.
- Recognize that the type of command determines the final resources executed.
- Understand that edge cases like `dbt test` or `source:` aren't exceptions but logical outcomes of this schematic.

Check warning on line 11 in website/docs/reference/node-selection/node-selection.md

View workflow job for this annotation

GitHub Actions / vale

[vale] website/docs/reference/node-selection/node-selection.md#L11

[custom.LatinAbbreviations] Avoid Latin abbreviations: 'for example'. Consider using 'edge' instead.
Raw output
{"message": "[custom.LatinAbbreviations] Avoid Latin abbreviations: 'for example'. Consider using 'edge' instead.", "location": {"path": "website/docs/reference/node-selection/node-selection.md", "range": {"start": {"line": 11, "column": 19}}}, "severity": "WARNING"}

### Resource selection

dbt selects any resource that matches one or more `--models` or `--select` criteria in the order of:

- Selection methods: Methods like `tag:`, `source:`, `model:`.
- Graph operators: Operators like `+` (which selects upstream/downstream dependencies).
- Set operators: Combining resources with `,` or other logical operators.
- Selected resources may include:
- models
- sources
- seeds
- snapshots
-
- tests

### Applying --exclude criteria

dbt removes any resource that matches one or more `--exclude` criteria. If the command is `dbt test`, dbt also de-selects any test that directly depends on an excluded resource.

### Resource filtering by command type

After applying selection and exclusion rules, dbt retains only the resources relevant to the current command:

- `dbt run`: Keeps models.
- `dbt seed`: Keeps seeds.
- `dbt test`: Keeps tests (and selects tests that depend on selected resources).
- Other commands: Filters resources specific to the actual command.

### Examples of resource selection

The following examples show how dbt selects resources based on the specific command and the `--select` and `--exclude` criteria.

#### Example 1
Selecting a model and downstream dependencies:

```bash
dbt run -s my_model+
```

This command selects `my_model` and all downstream dependencies (`+`). It selects only models because `dbt run` filters out other resource types.

#### Example 2
Testing a model with its tests:

```bash
dbt test -s my_model
```

This command selects `my_model` and any tests directly associated with it. It filters out models, keeping only tests for execution.

#### Example 3
Excluding a resource:

```bash
dbt run -s my_model --exclude tag:deprecated
```

This selects `my_model` and all downstream dependencies. It excludes any resource tagged as `deprecated` and filters resources to retain only models (because `dbt run` is invoked).

#### Example 4
YAML selectors differ slightly:

The `exclude:` key in YAML selectors supports more complex set logic, allowing fine-grained control over resource inclusion and exclusion.

### Considerations
Certain cases may seem inconsistent but follow the same selection process:

- Tests that depend on excluded models are de-selected automatically in dbt test.
- `source:` and `model:` selection methods behave differently.
- Using `+` or `,` can feel unintuitive but works predictably based on graph structure.
1 change: 1 addition & 0 deletions website/docs/reference/node-selection/syntax.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: "Syntax overview"
sidebar_label: "Syntax overview"
---

dbt's node selection syntax makes it possible to run only specific resources in a given invocation of dbt. This selection syntax is used for the following subcommands:
Expand Down
1 change: 1 addition & 0 deletions website/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -1087,6 +1087,7 @@ const sidebarSettings = {
type: "category",
label: "Node selection",
items: [
"reference/node-selection/node-selection",
"reference/node-selection/syntax",
"reference/node-selection/graph-operators",
"reference/node-selection/set-operators",
Expand Down
Loading