Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add context about yaml config #6755

Merged
merged 8 commits into from
Jan 10, 2025
40 changes: 34 additions & 6 deletions website/docs/docs/build/metricflow-time-spine.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ tags: [Metrics, Semantic Layer]
---
<VersionBlock firstVersion="1.9">

<!-- this whole section is for 1.9 and higher + Release Tracks -->
<!-- this whole section is for 1.9 and higher + Release tracks -->
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could probably remove this whole thing since the versionblock is in

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it! it was more for any new person coming in 🙏

mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved

It's common in analytics engineering to have a date dimension or "time spine" table as a base table for different types of time-based joins and aggregations. The structure of this table is typically a base column of daily or hourly dates, with additional columns for other time grains, like fiscal quarters, defined based on the base column. You can join other tables to the time spine on the base column to calculate metrics like revenue at a point in time, or to aggregate to a specific time grain.

Expand All @@ -23,7 +23,7 @@ To see the generated SQL for the metric and dimension types that use time spine

## Configuring time spine in YAML

Time spine models are normal dbt models with extra configurations that tell dbt and MetricFlow how to use specific columns by defining their properties. Add the [`models` key](/reference/model-properties) for the time spine in your `models/` directory. If your project already includes a calendar table or date dimension, you can configure that table as a time spine. Otherwise, review the [example time-spine tables](#example-time-spine-tables) to create one.
Time spine models are normal dbt models with extra configurations that tell dbt and MetricFlow how to use specific columns by defining their properties. Add the [`models` key](/reference/model-properties) for the time spine in your `models/` directory. If your project already includes a calendar table or date dimension, you can configure that table as a time spine. Otherwise, review the [example time-spine tables](#example-time-spine-tables) to create one. If the relevant model file (`util/_models.yml`) doesn't exist, create it and add the configuration mentioned in the [next section](#creating-a-time-spine-table).

Some things to note when configuring time spine models:

Expand All @@ -34,9 +34,9 @@ To see the generated SQL for the metric and dimension types that use time spine
- If you're looking to specify the grain of a time dimension so that MetricFlow can transform the underlying column to the required granularity, refer to the [Time granularity documentation](/docs/build/dimensions?dimension=time_gran)

:::tip
If you previously used a model called `metricflow_time_spine`, you no longer need to create this specific model. You can now configure MetricFlow to use any date dimension or time spine table already in your project by updating the `model` setting in the Semantic Layer.

If you don’t have a date dimension table, you can still create one by using the code snippet in the [next section](#creating-a-time-spine-table) to build your time spine model.
- If you previously used a `metricflow_time_spine.sql` model, you can delete it after configuring the `time_spine` property in YAML. The Semantic Layer will automatically recognize the new configuration. No additional `.yml` files are needed.
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved
- You can also configure MetricFlow to use any date dimension or time spine table already in your project by updating the `model` setting in the Semantic Layer.
- If you don’t have a date dimension table, you can still create one by using the code snippet in the [next section](#creating-a-time-spine-table) to build your time spine model.
:::

### Creating a time spine table
Expand Down Expand Up @@ -112,9 +112,37 @@ models:

For an example project, refer to our [Jaffle shop](https://github.com/dbt-labs/jaffle-sl-template/blob/main/models/marts/_models.yml) example.

### Migrating from SQL to YAML
If your project already includes a time spine (`metricflow_time_spine.sql`), you can migrate its configuration to YAML to address any deprecation warnings you may get.

1. For example, add the following configuration in a new or existing YAML file using the [`models` key](/reference/model-properties) for the time spine in your `models/` directory. Name the YAML file whatever you want (for example, we'll use `util/_models.yml`):
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved

<File name="models/_models.yml">

```yaml
models:
- name: all_days
description: A time spine with one row per day, ranging from 2020-01-01 to 2039-12-31.
time_spine:
standard_granularity_column: date_day # Column for the standard grain of your table
columns:
- name: date_day
granularity: day # Set the granularity of the column
```
</File>

2. After adding the YAML configuration, delete the existing `metricflow_time_spine.sql` file from your project to avoid any issues.

3. Test the configuration to ensure compatibility with your production jobs.

Note that if you're migrating from a `metricflow_time_spine.sql` file:

- Replace its functionality by adding the `time_spine` property to YAML as shown in the previous example.
- Once configured, MetricFlow will recognize the YAML settings, and then the SQL model file can be safely removed.

### Considerations when choosing which granularities to create{#granularity-considerations}

- MetricFlow will use the time spine with the largest compatible granularity for a given query to ensure the most efficient query possible. For example, if you have a time spine at a monthly grain, and query a dimension at a monthly grain, MetricFlow will use the monthly time spine. If you only have a daily time spine, MetricFlow will use the daily time spine and date_trunc to month.
- MetricFlow will use the time spine with the largest compatible granularity for a given query to ensure the most efficient query possible. For example, if you have a time spine at a monthly grain, and query a dimension at a monthly grain, MetricFlow will use the monthly time spine. If you only have a daily time spine, MetricFlow will use the daily time spine and `date_trunc` to month.
- You can add a time spine for each granularity you intend to use if query efficiency is more important to you than configuration time, or storage constraints. For most engines, the query performance difference should be minimal and transforming your time spine to a coarser grain at query time shouldn't add significant overhead to your queries.
- We recommend having a time spine at the finest grain used in any of your dimensions to avoid unexpected errors. For example, if you have dimensions at an hourly grain, you should have a time spine at an hourly grain.

Expand Down
Loading