Skip to content

Commit

Permalink
Pagination for Snowflake objects (#6445)
Browse files Browse the repository at this point in the history
## What are you changing in this pull request and why?

Closes dbt-labs/dbt-snowflake#1235

## Checklist
- [ ] I have reviewed the [Content style
guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md)
so my content adheres to these guidelines.
- [ ] The topic I'm writing about is for specific dbt version(s) and I
have versioned it according to the [version a whole
page](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#adding-a-new-version)
and/or [version a block of
content](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#versioning-blocks-of-content)
guidelines.
- [ ] I have added checklist item(s) to this list for anything anything
that needs to happen before this PR is merged, such as "needs technical
review" or "change base branch."
- [ ] The content in this PR requires a dbt release note, so I added one
to the [release notes
page](https://docs.getdbt.com/docs/dbt-versions/dbt-cloud-release-notes).
<!--
PRE-RELEASE VERSION OF dbt (if so, uncomment):
- [ ] Add a note to the prerelease version [Migration
Guide](https://github.com/dbt-labs/docs.getdbt.com/tree/current/website/docs/docs/dbt-versions/core-upgrade)
-->
<!-- 
ADDING OR REMOVING PAGES (if so, uncomment):
- [ ] Add/remove page in `website/sidebars.js`
- [ ] Provide a unique filename for new pages
- [ ] Add an entry for deleted pages in `website/vercel.json`
- [ ] Run link testing locally with `npm run build` to update the links
that point to deleted pages
-->

<!-- vercel-deployment-preview -->
---
🚀 Deployment available! Here are the direct links to the updated files:


-
https://docs-getdbt-com-git-snowflake-pagination-dbt-labs.vercel.app/reference/resource-configs/snowflake-configs

<!-- end-vercel-deployment-preview -->

---------

Co-authored-by: Amy Chen <[email protected]>
  • Loading branch information
matthewshaver and amychen1776 authored Nov 8, 2024
1 parent e9c141e commit 784ea5b
Showing 1 changed file with 24 additions and 0 deletions.
24 changes: 24 additions & 0 deletions website/docs/reference/resource-configs/snowflake-configs.md
Original file line number Diff line number Diff line change
Expand Up @@ -678,3 +678,27 @@ Per the [Snowflake documentation](https://docs.snowflake.com/en/sql-reference/in
>- DDL operations.
>- DML operations (for tables only).
>- Background maintenance operations on metadata performed by Snowflake.

<VersionBlock firstVersion="1.9">

## Pagination for object results

By default, when dbt encounters a schema with up to 100,000 objects, it will paginate the results from `show objects` at 10,000 per page for up to 10 pages.

Environments with more than 100,000 objects in a schema can customize the number of results per page and the page limit using the following [flags](/reference/global-configs/about-global-configs) in the `dbt_project.yml`:

- `list_relations_per_page` &mdash; The number of relations on each page (Max 10k as this is the most Snowflake allows).
- `list_relations_page_limit` &mdash; The maximum number of pages to include in the results.

For example, if you wanted to include 10,000 objects per page and include up to 100 pages (1 million objects), configure the flags as follows:


```yml
flags:
list_relations_per_page: 10000
list_relations_page_limit: 100
```

</VersionBlock>

0 comments on commit 784ea5b

Please sign in to comment.