Update testing structure for dbt Labs testing support - postgres

[emmyoop]

Resolves: #1700

This only adds support for testing with postgres. Postgres is run inside the GitHub runner and I do not need to worry about getting credentials for the various warehouses set up this way.

To add additional adapter testing support in the future, you will just need to modify

• .github/workflows/ci.yml to pass through the env vars
• modify profiles.yml to meet the desire var format (example in dbt-package-testing)
• supported_adapters.env to list the adapters you want to test
• tox.ini to add the commands for the adapter tests

Why Do We Need These Changes

As explained in #1700, by setting up tox as a standard entry-point, the Core development team at dbt Labs can include this package's integration tests (along with other popular dbt packages) as part of our continuous development & delivery of dbt.

[emmyoop]

After installing dependencies I get the following results from the command py.test -vvv --target postgres

==================================================================== short test summary info ====================================================================
FAILED e2e/report/test_report.py::test_report_keys - FileNotFoundError: [Errno 2] No such file or directory: 'edr_target/elementary_output.json'
FAILED e2e/report/test_report.py::test_group - FileNotFoundError: [Errno 2] No such file or directory: 'edr_target/elementary_output.json'
FAILED e2e/report/test_report.py::test_group_views - FileNotFoundError: [Errno 2] No such file or directory: 'edr_target/elementary_output.json'
FAILED e2e/report/test_report.py::test_duplicate_test_runs - FileNotFoundError: [Errno 2] No such file or directory: 'edr_target/elementary_output.json'
FAILED e2e/report/test_report.py::test_test_runs_are_sorted - FileNotFoundError: [Errno 2] No such file or directory: 'edr_target/elementary_output.json'
FAILED e2e/report/test_report.py::test_report_tests - FileNotFoundError: [Errno 2] No such file or directory: 'edr_target/elementary_output.json'
FAILED tests_with_db/test_dbt_runner.py::TestSubprocessDbtRunner::test_run_operation - IndexError: list index out of range
FAILED tests_with_db/test_dbt_runner.py::TestSubprocessDbtRunner::test_ls - assert 'elementary_tests.one' in ['\x1b[0m13:02:02  Encountered an error:', 'Runtime Error', "  Could not find profile named 'elementary_tests'"]
FAILED tests_with_db/test_dbt_runner.py::TestSubprocessDbtRunner::test_seed - assert False is True
FAILED tests_with_db/test_dbt_runner.py::TestSubprocessDbtRunner::test_run - assert False is True
FAILED tests_with_db/test_dbt_runner.py::TestSubprocessDbtRunner::test_test - assert False is True
FAILED tests_with_db/test_dbt_runner.py::TestAPIDbtRunner::test_run_operation - IndexError: list index out of range
FAILED tests_with_db/test_dbt_runner.py::TestAPIDbtRunner::test_ls - assert False
 +  where False = isinstance(None, list)
FAILED tests_with_db/test_dbt_runner.py::TestAPIDbtRunner::test_seed - assert False is True
FAILED tests_with_db/test_dbt_runner.py::TestAPIDbtRunner::test_run - assert False is True
FAILED tests_with_db/test_dbt_runner.py::TestAPIDbtRunner::test_test - assert False is True
========================================================== 16 failed, 111 passed in 108.76s (0:01:48) ===========================================================

I have a postgres target set up for elementary_tests but the test_dbt_runner failures end up raising this and I'm not sure how to proceed.

13:41:57  Encountered an error:
Compilation Error
  dbt found 2 package(s) specified in packages.yml, but only 0 package(s) installed in dbt_packages. Run "dbt deps" to install package dependencies.

or

{"data": {"exc": "Runtime Error\n  No dbt_project.yml found at expected path /private/var/folders/qs/elementary/monitor/dbt_project/dbt_project.yml\n  Verify that each entry within packages.yml (and their transitive dependencies) contains a file named dbt_project.yml\n  "}, "info": {"category": "", "code": "Z002", "extra": {}, "invocation_id": "d21524a9-dc01-4d39-97be-804b58567f42", "level": "error", "msg": "Encountered an error:\nRuntime Error\n  No dbt_project.yml found at expected path /private/var/folders/qs/elementary/monitor/dbt_project/dbt_project.yml\n  Verify that each entry within packages.yml (and their transitive dependencies) contains a file named dbt_project.yml\n  ", "name": "MainEncounteredError", "pid": 39468, "thread": "MainThread", "ts": "2024-10-11T13:41:57.061663Z"}}

I'm also unsure where the missing files for the e2e tests should be coming from.

Any help getting these tests passing would be appreciated!

[haritamar]

Hi @emmyoop !
Sorry for the long delay here.
I think this is actually not the repo you want to test - we actually have two different repository:

1. dbt-data-reliability - the Elementary dbt package.
2. elementary - the elementary CLI (used for reporting and sending alerts)

As I understand it, I think it makes more sense for you to run the integration tests of the first (which is also done with py.test).
In any case, I'll check when we can help with this + ensure that it doesn't break our internal CI process.

In any case, really cool initiative and appreciate the effort here.

Thanks,
Itamar