You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit af8c9fb
Browse filesBrowse the repository at this point in the historyBrowse files
Copy file name to clipboardExpand all lines: website/docs/docs/build/semantic-models.md
+9-6Lines changed: 9 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,17 +8,20 @@ sidebar_label: Semantic models
8
8
tags: [Metrics, Semantic Layer]
9
9
---
10
10
11
-
Semantic models serve as the foundation for defining data in MetricFlow, which powers the dbt Semantic Layer. You can think of semantic models as nodes in your semantic graph, connected via entities as edges. MetricFlow takes semantic models defined in YAML configuration files as inputs and creates a semantic graph that can be used to query metrics.
11
+
Semantic models are the foundation for data definition in MetricFlow, which powers the dbt Semantic Layer:
12
12
13
-
Each semantic model corresponds to a dbt model in your DAG. Therefore you will have one YAML config for each semantic model in your dbt project. You can create multiple semantic models out of a single dbt model, as long as you give each semantic model a unique name.
14
-
15
-
You can configure semantic models in your dbt project directory in a `YAML` file. Depending on your project structure, you can nest semantic models under a `metrics:` folder or organize them under project sources.
13
+
- Think of semantic models as nodes connected by entities in a semantic graph.
14
+
- MetricFlow uses YAML configuration files to create this graph for querying metrics.
15
+
- Each semantic model corresponds to a dbt model in your DAG, requiring a unique YAML configuration for each semantic model.
16
+
- You can create multiple semantic models from a single dbt model, as long as you give each semantic model a unique name.
17
+
- Configure semantic models in a YAML file within your dbt project directory.
18
+
- Organize them under a `metrics:` folder or within project sources as needed.
16
19
17
20
Semantic models have 6 components and this page explains the definitions with some examples:
18
21
19
22
| Component | Description | Type |
20
23
| --------- | ----------- | ---- |
21
-
|[Name](#name)|Unique name for the semantic model | Required |
24
+
|[Name](#name)|Choose a unique name for the semantic model. Avoid using double underscores (__) in the name as they're not supported.| Required |
22
25
|[Description](#description)| Includes important details in the description | Optional |
23
26
|[Model](#model)| Specifies the dbt model for the semantic model using the `ref` function | Required |
24
27
|[Defaults](#defaults)| The defaults for the model, currently only `agg_time_dimension` is supported. | Required |
@@ -107,7 +110,7 @@ semantic_models:
107
110
108
111
### Name
109
112
110
-
Define the name of the semantic model. You must define a unique name for the semantic model. The semantic graph will use this name to identify the model, and you can update it at any time.
113
+
Define the name of the semantic model. You must define a unique name for the semantic model. The semantic graph will use this name to identify the model, and you can update it at any time. Avoid using double underscores (__) in the name as they're not supported.
@@ -12,99 +12,189 @@ These instructions are not intended for general audiences at this time.
12
12
13
13
:::
14
14
15
-
## Installing dbt Cloud CLI
16
15
17
-
### Install and update with Brew on MacOS (recommended)
16
+
dbt Cloud natively supports developing using a command line (CLI), empowering team members to contribute with enhanced flexibility and collaboration. The dbt Cloud CLI allows you to run dbt commands against your dbt Cloud development environment from your local command line.
18
17
19
-
1. Install the dbt Cloud CLI:
18
+
dbt commands are run against dbt Cloud's infrastructure and benefit from:
19
+
20
+
* Secure credential storage in the dbt Cloud platform.
21
+
* Automatic deferral of build artifacts to your Cloud project's production environment.
22
+
* Speedier, lower-cost builds.
23
+
* Support for dbt Mesh ([cross-project `ref`](/docs/collaborate/govern/project-dependencies)),
24
+
* Significant platform improvements, to be released over the coming months.
25
+
26
+
27
+
## Install dbt Cloud CLI
28
+
29
+
You can install the dbt Cloud CLI on the command line by using one of these methods:
30
+
31
+
<TabsqueryString="install">
32
+
33
+
<TabItemvalue="brew"label="macOS">
34
+
35
+
Before you begin, make sure you have [Homebrew installed](http://brew.sh/) in your code editor or command line terminal. If your operating system runs into path conflicts, refer to the [FAQs](#faqs).
36
+
37
+
1. Run the following command to verify that there is no conflict with a dbt Core installation on your system:
38
+
39
+
```bash
40
+
which dbt
41
+
```
42
+
- This should return a `dbt not found`. If the dbt help text appears, use `pip uninstall dbt` to deactivate dbt Core from your machine.
43
+
44
+
2. Install the dbt Cloud CLI with Homebrew:
20
45
21
46
```bash
22
47
brew tap dbt-labs/dbt-cli
23
48
brew install dbt-cloud-cli
24
49
```
25
50
26
-
2. Verify the installation by requesting your homebrew installation path (not your dbt core installs). If the `which dbt` command returns nothing, then you should modify your PATH in `~.zshrc` or create an alias.
51
+
3. Verify the installation by running `dbt --help` from the command line. If the help text doesn't indicate that you're using the dbt Cloud CLI, make sure you've deactivated your pyenv or venv and don't have a version of dbt globally installed.
52
+
53
+
* You no longer need to use the `dbt deps` command. Previously, you had to run that command.
54
+
55
+
</TabItem>
56
+
57
+
<TabItemvalue="windows"label="Windows">
58
+
59
+
If your operating system runs into path conflicts, refer to the [FAQs](#faqs).
60
+
61
+
1. Download the latest Windows release for your platform from [GitHub](https://github.com/dbt-labs/dbt-cli/releases).
62
+
63
+
2. Extract the `dbt.exe` executeable into the same folder as your dbt project.
64
+
65
+
:::info
66
+
67
+
Advanced users can configure multiple projects to use the same dbt Cloud CLI by placing the executable in the Program Files folder and [adding it to their Windows PATH environment variable](https://medium.com/@kevinmarkvi/how-to-add-executables-to-your-path-in-windows-5ffa4ce61a53).
68
+
69
+
Note that if you are using VS Code, you'll need to restart it to pick up modified environment variables.
70
+
:::
71
+
72
+
3. Verify the installation by running `./dbt --help` from the command line. If the help text doesn't indicate that you're using the dbt Cloud CLI, make sure you've deactivated your pyenv or venv and don't have a version of dbt globally installed.
73
+
74
+
* You no longer need to use the `dbt deps` command. Previously, you had to run that command.
75
+
76
+
</TabItem>
77
+
78
+
<TabItemvalue="linux"label="Linux">
79
+
80
+
Refer to the [FAQs](#faqs) if your operating system runs into path conflicts.
81
+
82
+
1. Download the latest Linux release for your platform from [GitHub](https://github.com/dbt-labs/dbt-cli/releases). (Pick the file based on your CPU architecture)
83
+
84
+
2. Extract the `dbt-cloud-cli` binary to the same folder as your dbt project.
27
85
28
86
```bash
29
-
which dbt
30
-
dbt --help
87
+
tar -xf dbt_0.29.9_linux_amd64.tar.gz
88
+
./dbt --version
31
89
```
32
90
33
-
### Manually install (Windows and Linux)
91
+
:::info
34
92
35
-
1. Download the latest release for your platform from [GitHub](https://github.com/dbt-labs/dbt-cli/releases).
36
-
2. Add the `dbt` executable to your path.
37
-
3. Move to a directory with a dbt project, and create a `dbt_cloud.yml` file containing your `project-id` from dbt Cloud.
38
-
4. Invoke `dbt --help` from your terminal to see a list of supported commands.
93
+
Advanced users can configure multiple projects to use the same Cloud CLI executable by adding it to their PATH environment variable in their shell profile.
39
94
40
-
#### Updating your dbt Cloud installation (Windows + Linux)
95
+
:::
41
96
42
-
Follow the same process in [Installing dbt Cloud CLI](#manually-install-windows-only) and replace the existing `dbt` executable with the new one. You should not have to go through the security steps again.
97
+
3. Verify the installation by running `./dbt --help` from the command line. If the help text doesn't indicate that you're using the dbt Cloud CLI, make sure you've deactivated your pyenv or venv and don't have a version of dbt globally installed.
43
98
44
-
## Setting up the CLI
99
+
* You no longer need to use the `dbt deps` command. Previously, you had to run that command.
45
100
46
-
The following instructions are for setting up the dbt Cloud CLI.
101
+
</TabItem>
47
102
48
-
1. Ensure that you have created a project in [dbt Cloud](https://cloud.getdbt.com/).
103
+
</Tabs>
49
104
50
-
2. Ensure that your personal [development credentials](https://cloud.getdbt.com/settings/profile/credentials) are set on the project.
105
+
## Update dbt Cloud CLI
51
106
52
-
3. Navigate to [your profile](https://cloud.getdbt.com/settings/profile) and enable the **Beta** flag under **Experimental Features.**
107
+
The following instructions explain how to update the dbt CLoud CLI to the latest version depending on your operating system.
53
108
54
-
4. Create an environment variable with your [dbt Cloud API key](https://cloud.getdbt.com/settings/profile#api-access):
109
+
<Tabs>
110
+
111
+
<TabItemvalue="mac"label="macOS">
55
112
56
-
```bash
57
-
vi ~/.zshrc
113
+
To update the dbt Cloud CLI, run `brew upgrade dbt-cloud-cli`.
58
114
59
-
# dbt Cloud CLI
60
-
export DBT_CLOUD_API_KEY="1234"# Replace "1234" with your API key
61
-
```
115
+
</TabItem>
116
+
<TabItemvalue="windowslinux"label="Windows and Linux">
117
+
118
+
To update, follow the same process explained in [Install manually (Windows)](/docs/cloud/cloud-cli-installation?install=windows#install-dbt-cloud-cli) and replace the existing `dbt.exe` executable with the new one.
119
+
120
+
</TabItem>
62
121
63
-
5. Load the new environment variable. Note: You may need to reactivate your Python virtual environment after sourcing your shell's dot file. Alternatively, restart your shell instead of sourcing the shell's dot file
122
+
</Tabs>
123
+
124
+
## Configure the dbt Cloud CLI
125
+
126
+
After installation, you can configure the dbt Cloud CLI for your dbt Cloud project and use it to run [dbt commands](/reference/dbt-commands) similar to dbt Core. For example, you can execute the following command to compile a project using dbt Cloud:
64
127
65
128
```bash
66
-
source~/.zshrc
129
+
dbt compile
67
130
```
68
131
69
-
6. Navigate to a dbt project
132
+
**Prerequisites**
133
+
134
+
- You must set up a project in dbt Cloud.
135
+
- You must have your [personal development credentials](/docs/dbt-cloud-environments#set-developer-credentials) set for that project. The dbt Cloud CLI will use these credentials, stored securely in dbt Cloud, to communicate with your data platform.
136
+
- You must [enroll](/docs/dbt-versions/experimental-features) in the dbt Cloud beta features.
137
+
- To enroll, navigate to your **Profile Settings** and enable the **Beta** flag under **Experimental Features**.
138
+
139
+
Once you install the dbt Cloud CLI, you need to configure it to connect to a dbt Cloud project.
140
+
141
+
1. Ensure you meet the prerequisites above.
142
+
2. Create an environment variable with your [dbt Cloud API key](/docs/dbt-cloud-apis/user-tokens):
143
+
- On MacOS, Linux, or Windows add an environment variable:
144
+
145
+
```bash
146
+
export DBT_CLOUD_API_KEY="1234"# Replace 1234 with your API key
147
+
```
148
+
149
+
- In Powershell, add an environment variable:
150
+
- Note that this variable resets if you restart your shell. To add an environment variable permanently, add a system environment variable in your platform.
151
+
152
+
3. Navigate to a dbt project in your terminal:
70
153
71
154
```bash
72
155
cd~/dbt-projects/jaffle_shop
73
156
```
74
157
75
-
7. Create a `dbt_cloud.yml`in the root project directory. The file is required to have a `project-id` field with a valid [project ID](#glossary). Enter the following commands:
158
+
4. In your `dbt_project.yml`file, ensure there is a section titled `dbt-cloud`. This section is required to have a `project-id` field with a valid project ID.
You can find your project ID by selecting your project and clicking on **Develop** in the navigation bar. Your project ID is the number in the URL: https://cloud.getdbt.com/develop/26228/projects/PROJECT_ID.
171
+
- To find your project ID, go to **Develop**in the navigation menu. Select the dbt Cloud project URL, such as `https://cloud.getdbt.com/develop/26228/projects123456`, where the project ID is `123456`.
92
172
93
-
If `dbt_cloud.yml` already exists, edit the file, and verify the project ID field uses a valid project ID.
94
173
95
-
#### Upgrade the CLI with Brew
174
+
## Use the dbt Cloud CLI
96
175
97
-
```bash
98
-
brew update
99
-
brew upgrade dbt-cloud-cli
100
-
```
176
+
The dbt Cloud CLI shares the same set of commands as dbt Core. When you invoke a dbt command, that command is sent to dbt Cloud for processing.
177
+
178
+
The dbt Cloud CLI supports [project dependencies](/docs/collaborate/govern/project-dependencies), which is an exciting way to depend on another project using the metadata service in dbt Cloud. It instantly resolves references (or `ref`) to public models defined in other projects. You don't need to execute or analyze these upstream models yourself. Instead, you treat them as an API that returns a dataset.
179
+
180
+
Share feedback or request features you'd like to see on the [dbt community Slack](https://getdbt.slack.com/archives/C05M77P54FL).
181
+
182
+
183
+
## FAQs
184
+
185
+
<details>
186
+
187
+
<summary>What's the difference between the dbt Cloud CLI and dbt Core?</summary>
188
+
The dbt Cloud CLI and <a href="https://github.com/dbt-labs/dbt-core">dbt Core</a>, an open-source project, are both command line tools that enable you to run dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its <a href="https://docs.getdbt.com/docs/cloud/about-cloud/dbt-cloud-features">features</a>.
189
+
190
+
</details>
101
191
102
-
## Using dbt Cloud CLI
192
+
<details>
193
+
<summary>How do I solve for path conflicts</summary>
194
+
For compatibility, both the dbt Cloud CLI and dbt Core are invoked by running `dbt`. This can create path conflicts if your operating system selects one over the other based on your $PATH environment variable (settings).
103
195
104
-
**Coming soon**
196
+
If you have dbt Core installed locally, ensure that you deactivate your Python environment or uninstall it using `pip uninstall dbt` before proceeding. Alternatively, advanced users can modify the $PATH environment variable to correctly point to the dbt Cloud CLI binary to use both dbt Cloud CLI and dbt Core together.
105
197
106
-
## Glossary
198
+
You can always uninstall the Cloud CLI to return to using dbt Core.
199
+
</details>
107
200
108
-
-**dbt cloud API key:** Your API key found by navigating to the **gear icon**, clicking **Profile Settings**, and scrolling down to **API**.
109
-
-**Project ID:** The ID of the dbt project you're working with. Can be retrieved from the dbt Cloud URL after a project has been selected, for example, `https://cloud.getdbt.com/deploy/{accountID}/projects/{projectID}`
110
-
-**Development credentials:** Your personal warehouse credentials for the project you’re working with. They can be set by selecting the project and entering them in dbt Cloud. Navigate to the **gear icon**, click **Profile Settings**, and click **Credentials** from the left-side menu.
|`threads`|`8`| How many threads dbt should use (default is `1`) |
86
-
|`roles`|`system: analyst`| Catalog roles |
86
+
|`roles`|`system: analyst`| Catalog roles can be set under the optional `roles` parameter using following format: `catalog: role`.|
87
87
|`session_properties`|`query_max_run_time: 4h`| Sets Trino session properties used in the connection. Execute `SHOW SESSION` to see available options |
88
88
|`prepared_statements_enabled`|`true` or `false`| Enable usage of Trino prepared statements (used in `dbt seed` commands) (default: `true`) |
89
89
|`retries`|`10`| Configure how many times all database operation is retried when connection issues arise (default: `3`) |
title: "Enhancement: Native support for the dbt retry command"
3
+
description: "October 2023: Rerun errored jobs from start or from the failure point"
4
+
sidebar_label: "Enhancement: Support for dbt retry"
5
+
tags: [Oct-2023, Scheduler]
6
+
date: 2023-10-06
7
+
sidebar_position: 10
8
+
---
9
+
10
+
Previously in dbt Cloud, you could only rerun an errored job from start but now you can also rerun it from its point of failure.
11
+
12
+
You can view which job failed to complete successully, which command failed in the run step, and choose how to rerun it. To learn more, refer to [Retry jobs](/docs/deploy/retry-jobs).
13
+
14
+
15
+
<Lightboxsrc="/img/docs/deploy/native-retry.gif"width="70%"title="Example of the Rerun options in dbt Cloud"/>
description: "September 2023: The Product docs team merged 107 PRs, made various updates to dbt Cloud and Core, such as GAing continuous integration jobs, Semantic Layer GraphQL API doc, a new community plugin, and more"
5
+
sidebar_label: "Update: Product docs changes"
6
+
tags: [Sept-2023, product-docs]
7
+
date: 2023-10-10
8
+
sidebar_position: 09
9
+
---
10
+
11
+
Hello from the dbt Docs team: @mirnawong1, @matthewshaver, @nghi-ly, and @runleonarun! First, we’d like to thank the 15 new community contributors to docs.getdbt.com. We merged [107 PRs](https://github.com/dbt-labs/docs.getdbt.com/pulls?q=is%3Apr+merged%3A2023-09-01..2023-09-31) in September.
12
+
13
+
Here's what's new to [docs.getdbt.com](http://docs.getdbt.com/):
14
+
15
+
* Migrated docs.getdbt.com from Netlify to Vercel.
16
+
17
+
## ☁ Cloud projects
18
+
- Continuous integration jobs are now generally available and no longer in beta!
19
+
- Added [Postgres PrivateLink set up page](/docs/cloud/secure/postgres-privatelink)
20
+
- Published beta docs for [dbt Explorer](/docs/collaborate/explore-projects).
21
+
- Added a new Semantic Layer [GraphQL API doc](/docs/dbt-cloud-apis/sl-graphql) and updated the [integration docs](/docs/use-dbt-semantic-layer/avail-sl-integrations) to include Hex. Responded to dbt community feedback and clarified Metricflow use cases for dbt Core and dbt Cloud.
22
+
- Added an [FAQ](/faqs/Git/git-migration) describing how to migrate from one git provider to another in dbt Cloud.
23
+
- Clarified an example and added a [troubleshooting section](/docs/cloud/connect-data-platform/connect-snowflake#troubleshooting) to Snowflake connection docs to address common errors and provide solutions.
24
+
25
+
26
+
## 🎯 Core projects
27
+
28
+
- Deprecated dbt Core v1.0 and v1.1 from the docs.
29
+
- Added configuration instructions for the [AWS Glue](/docs/core/connect-data-platform/glue-setup) community plugin.
30
+
- Revised the dbt Core quickstart, making it easier to follow. Divided this guide into steps that align with the [other guides](/quickstarts/manual-install?step=1).
31
+
32
+
## New 📚 Guides, ✏️ blog posts, and FAQs
33
+
34
+
Added a [style guide template](/guides/best-practices/how-we-style/6-how-we-style-conclusion#style-guide-template) that you can copy & paste to make sure you adhere to best practices when styling dbt projects!
35
+
36
+
## Upcoming changes
37
+
38
+
Stay tuned for a flurry of releases in October and a filterable guides section that will make guides easier to find!
0 commit comments