Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

generate_base_model() doesn't work across databases on Redshift #167

Closed
1 of 5 tasks
jeremyyeo opened this issue Mar 21, 2024 · 2 comments
Closed
1 of 5 tasks

generate_base_model() doesn't work across databases on Redshift #167

jeremyyeo opened this issue Mar 21, 2024 · 2 comments
Labels
bug Something isn't working Stale triage

Comments

@jeremyyeo
Copy link
Contributor

jeremyyeo commented Mar 21, 2024

Describe the bug

If your redshift connection is to the database dev but your source is on a separate database sources - then the generate_base_model() macro does not work due to dbt-labs/dbt-redshift#736

Steps to reproduce

  1. Connect your redshift project to a database dev.
  2. Specify a source that is on a separate database:
# models/sources.yml
version: 2
sources:
  - name: my_sources
    database: sources_only
    schema: public
    tables:
      - name: source_foo
  1. Use the codegen.generate_base_model() macro on that source and see that we are not able to list the columns since [Bug] adapter.get_columns_in_relation does not work cross database dbt-redshift#736 meant that adapter.get_columns_in_relation() does not work across databases for Redshift. (Here is where this package uses get_columns_in_relation())
$ dbt debug
...
02:14:25  Connection:
...
02:14:25    user: root
02:14:25    port: 5439
02:14:25    database: dev
...
$ dbt run-operation generate_base_model --args '{"source_name": "my_sources", "table_name": "source_foo"}'
02:15:00  Running with dbt=1.7.9
02:15:01  Registered adapter: redshift=1.7.1
02:15:03  Found 1 model, 1 source, 0 exposures, 0 metrics, 594 macros, 0 groups, 0 semantic models


with source as (

    select * from {{ source('my_sources', 'source_foo') }}

),

renamed as (

    select

    from source

)

select * from renamed

Expected results

We should be able to get column information across databases and write them into the renamed cte.

Actual results

See above.

Screenshots and log output

See above.

System information

The contents of your packages.yml file:

Which database are you using dbt with?

  • postgres
  • redshift
  • bigquery
  • snowflake
  • other (specify: ____________)

The output of dbt --version:

Core:
  - installed: 1.7.9 
  - latest:    1.7.10 - Update available!

  Your version of dbt-core is out of date!
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

Plugins:
  - databricks: 1.7.3 - Update available!
  - bigquery:   1.7.3 - Update available!
  - snowflake:  1.7.2 - Up to date!
  - redshift:   1.7.1 - Update available!
  - postgres:   1.7.9 - Update available!
  - spark:      1.7.1 - Up to date!

  At least one plugin is out of date or incompatible with dbt-core.
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

The operating system you're using:

macOS

**The output of python --version:

Python 3.11.2

Additional context

This won't be fixed in this package until some point we actually fix it in the redshift adapter itself in dbt-labs/dbt-redshift#736

Are you interested in contributing the fix?

Copy link

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

@github-actions github-actions bot added the Stale label Sep 18, 2024
Copy link

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Sep 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Stale triage
Projects
None yet
Development

No branches or pull requests

1 participant