Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] dbt snowflake no longer works for microbatch #1248

Closed
2 tasks done
sarahjryan opened this issue Nov 12, 2024 · 2 comments
Closed
2 tasks done

[Bug] dbt snowflake no longer works for microbatch #1248

sarahjryan opened this issue Nov 12, 2024 · 2 comments
Assignees
Labels
bug Something isn't working microbatch Issues related to the microbatch incremental strategy triage

Comments

@sarahjryan
Copy link

sarahjryan commented Nov 12, 2024

Is this a new bug in dbt-snowflake?

  • I believe this is a new bug in dbt-snowflake
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

When calling dbt build on incremental models configured with strategy microbatch I'm now (as of afternoon 12th Nov 24) getting this error:

Compilation Error in model xxx
The flag require_batched_execution_for_custom_microbatch_strategy has not be registered.

in macro materialization_incremental_snowflake (macros\materializations\incremental.sql)
called by model xxx

Checking the dbt docs a change has been made so DBT_EXPERMINENTAL_MICROBATCH no longer affects runtime functionality. The article refers to the flag specified in the error and says false is default, no need to specify. Tried setting to true jsut to check & no microbatching occurs.

https://docs.getdbt.com/reference/global-configs/behavior-changes#custom-microbatch-strategy

Expected Behavior

The microbatching works w/o error as it did prior to the change, when the env var was used

Steps To Reproduce

Configure an incremental as microbatch and run for more than one batch (the first batch seems to work w/o issue)

Relevant log output

No response

Environment

- OS: windows 11
- Python: 3.10.9
- dbt-snowflake==1.9.0b1
- dbt-core==1.9.0b2

Additional Context

No response

@sarahjryan sarahjryan added bug Something isn't working triage labels Nov 12, 2024
@amychen1776 amychen1776 added microbath microbatch Issues related to the microbatch incremental strategy and removed triage microbath labels Nov 18, 2024
@QMalcolm
Copy link
Contributor

QMalcolm commented Nov 18, 2024

Hi @sarahjryan, are you stilling running into this issue? If you are, can you run a pip freeze | grep dbt-adapters and post the output?

To jump a head as to why I'm asking for the dbt-adapters version, we released a bad version of dbt-adapters, 1.11.0, which caused this issue. We've since yanked that version from PyPI. Moving to dbt-adapters 1.9.0 (which interestingly was released after 1.11.0 🫠) should resolve the issue.

@QMalcolm
Copy link
Contributor

@sarahjryan I'm going to close this out as we haven't heard back. If this is still an issue for you, please re-open this issue 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working microbatch Issues related to the microbatch incremental strategy triage
Projects
None yet
Development

No branches or pull requests

3 participants