Skip to content
This repository has been archived by the owner on Apr 8, 2024. It is now read-only.

fal flow run --target TARGET loads non-flag target before switching #744

Open
chamini2 opened this issue Feb 7, 2023 · 2 comments
Open
Labels
bug Something isn't working

Comments

@chamini2
Copy link
Member

chamini2 commented Feb 7, 2023

Describe the bug
When running fal flow run --target ddb (my duckdb profile) I get the following error:

20:46:15  Error importing adapter: No module named 'dbt.adapters.fal'
Traceback (most recent call last):
  File "/Users/matteo/.pyenv/versions/3.8.13/envs/dbtdd/lib/python3.8/site-packages/dbt/adapters/factory.py", line 56, in load_plugin
    mod: Any = import_module("." + name, "dbt.adapters")
  File "/Users/matteo/.pyenv/versions/3.8.13/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'dbt.adapters.fal'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/matteo/.pyenv/versions/3.8.13/envs/dbtdd/lib/python3.8/site-packages/dbt/config/profile.py", line 179, in _credentials_from_profile
    cls = load_plugin(typename)
  File "/Users/matteo/.pyenv/versions/3.8.13/envs/dbtdd/lib/python3.8/site-packages/dbt/adapters/factory.py", line 201, in load_plugin
    return FACTORY.load_plugin(name)
  File "/Users/matteo/.pyenv/versions/3.8.13/envs/dbtdd/lib/python3.8/site-packages/dbt/adapters/factory.py", line 62, in load_plugin
    raise RuntimeException(f"Could not find adapter type {name}!")
dbt.exceptions.RuntimeException: Runtime Error
  Could not find adapter type fal!

And the ddb target has type: duckdb

fal_test:
  target: fal_ddb

  outputs:
# relevant section of my profiles.yml
    fal_ddb:
      type: fal
      db_profile: ddb
    ddb:
      type: duckdb
      path: "/Users/user/duck_db_dbt_dump.db" 
      s3_region: "us-east"
      s3_access_key_id: aa
      s3_secret_access_key: "xx"

Your environment

  • OS: mac
  • Paste the following commands output:
❯ fal --version
fal 0.7.3

❯ dbt --version
Core:
  - installed: 1.3.0
  - latest:    1.4.1 - Update available!

  Your version of dbt-core is out of date!
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

Plugins:
  - duckdb: 1.3.1 - Update available!

  At least one plugin is out of date or incompatible with dbt-core.
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

How to reproduce
Point to a not-installed profile in your target: property in profiles.yml, and pass a yes-installed profile.

Expected behavior
Should completely ignore the target: of profiles.yml and instead use --target setting.

Actual behavior
It will load the profile with the not-installed profile before actually using the correct one, and raise an exception.

@chamini2 chamini2 added the bug Something isn't working label Feb 7, 2023
@iddar
Copy link

iddar commented Jun 6, 2023

Same issue here on mac os user brew version.

❯ dbt --version
Core:
  - installed: 1.5.0
  - latest:    1.5.1 - Update available!

  Your version of dbt-core is out of date!
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

Plugins:
  - postgres: 1.5.0 - Update available!

  At least one plugin is out of date or incompatible with dbt-core.
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

❯ fal --version
fal 0.9.1

error:

❯ dbt run --target fal_dev
23:36:05  Running with dbt=1.5.0
23:36:05  Error importing adapter: No module named 'dbt.adapters.fal'
23:36:05  Encountered an error:
Runtime Error
  Credentials in profile "flux_analytics", target "fal_dev" invalid: Runtime Error
    Could not find adapter type fal!

@chamini2
Copy link
Member Author

chamini2 commented Jun 9, 2023

Hello, @iddar

From your output it seems you do not have dbt-fal installed. fal and dbt-fal are two separate packages. To use dbt with a fal type adapter, you need to install dbt-fal.

Can you run pip install dbt-fal==1.5.2 and let us know if know it works?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants