-
Notifications
You must be signed in to change notification settings - Fork 3
Snowflake Dialect and Testing #363
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Changes from 51 commits
Commits
Show all changes
54 commits
Select commit
Hold shift + click to select a range
6abec11
Enable de-correlation of pure semi/anti joins, including pullup for p…
knassre-bodo 9b4fb04
Fix correl 16 WIP
knassre-bodo 319797f
Fixed bugs, need to deal with window squished CTEs for correl #16
knassre-bodo 634585c
Merge branch 'main' into kian/decor_semi_anti
knassre-bodo 6361404
Resolving conflicts [RUN CI]
knassre-bodo e4764ca
Squash merge Hadia/sf changes onto kian/decor_semi_anti
hadia206 8dbcd9c
[run SF]
hadia206 0c3167e
[run SF] add connection object as option for load_snowflake_connection
hadia206 19baea0
Use SF Functions directly instead of PyDough changes (#377)
hadia206 716f1aa
Merge branch 'main' of https://github.com/bodo-ai/PyDough into Hadia/sf
hadia206 f5c80b4
[run CI] [run SF] merge main and re-run tests
hadia206 f6e1f5e
Merge branch 'Hadia/sf' of https://github.com/bodo-ai/PyDough into Ha…
hadia206 e0a135d
[run CI] [run SF] merge and re-run tests
hadia206 597f5f9
merge
hadia206 95895c8
update tests and fix some SF issues
hadia206 a2c4d1f
add new test files
hadia206 7fbcdc6
update dayname
hadia206 0a8ac03
fix date_trunc
hadia206 1b6a84b
[run SF] [run CI] handle change in start of week
hadia206 15c0242
comment
hadia206 84cf1d7
Merge branch 'main' of https://github.com/bodo-ai/PyDough into Hadia/sf
hadia206 91dc3bb
[run CI] [run SF] fix week
hadia206 c8cc53f
[run CI] [run SF] update some SQL files
hadia206 ad0a1dd
[run SF][run CI] fix date_diff, add defog, 1 test only for params, docs
hadia206 a24829d
add decimal/int none/empty to harmonize [run CI]
hadia206 7a87630
add defog tests
hadia206 04ee554
modify sf_conn_db_context to accept db and schema as arguments
hadia206 b4d77de
handle relsum(boolean)
hadia206 fd8cbfb
fix length in slicing
hadia206 bdb71c5
relsum to sum
hadia206 2a6717b
add defog metatdata and SQL
hadia206 70cd17f
update PyDoughSQLComparisonTest to use reference database
hadia206 4ee0ea3
updates
hadia206 fc2f910
merge conflicts
hadia206 510ef58
update tests
hadia206 43dbdc7
task script and check in connection
hadia206 30b3812
task script and check in connection
hadia206 a703b41
fix date_trunc in sqlite and uncomment row
hadia206 8c40bac
add run-all option
hadia206 296f0c1
mysql in dev dependency
hadia206 dc06546
add connectors to docs
hadia206 55545c2
move week7 in week_offset
hadia206 1558b7c
update try/except for mysql
hadia206 377b523
move week7 in week_offset
hadia206 fcc698b
remove str_count
hadia206 ae623b3
[run all] update mysql init script
hadia206 3de8b89
undo changes for sql text
hadia206 ed8f82e
modify week_offset to be like mysql for consistency
hadia206 1c33249
modify week_offset to be like mysql for consistency
hadia206 5f31407
[run all] disable dialect for SF only
hadia206 8a28f4a
[run all] fix link and remove old test
hadia206 35c1c2e
address comments [run all]
hadia206 0d6eddf
Fixing MySQL testing issues [RUN ALL]
knassre-bodo 45b43de
Fixing start of week problems in MySQL setup script [RUN ALL]
knassre-bodo File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -97,8 +97,47 @@ script must be run so that the `defog.db` file is located in the `tests` directo | |
|
|
||
| ## Running CI Tests | ||
|
|
||
| To run our CI tests on your PR, you must include the flag `[run CI]` in latest | ||
| commit message. | ||
| When submitting a PR, you can control which CI tests run by adding special flags | ||
| to your **latest commit message**. | ||
|
|
||
| - To run **PyDough CI tests**, add: `[run CI]` | ||
|
||
| - To run **PyDough and all dialect tests**, add: `[run all]` | ||
| - To run **specific dialect tests**, use the corresponding flag as described below. | ||
|
|
||
| ### Running Snowflake Tests on CI | ||
| To run **Snowflake CI tests**, add the flag `[run SF]` to your commit message. | ||
|
|
||
| **Running Snowflake tests locally:** | ||
|
|
||
| 1. Install the Snowflake Connector for Python with Pandas support | ||
| ```bash | ||
| pip install "snowflake-connector-python[pandas]" | ||
| ``` | ||
|
|
||
| 2. Set your Snowflake credentials as environment variables: | ||
| ```bash | ||
| export SF_USERNAME="your_username" | ||
| export SF_PASSWORD="your_password" | ||
| export SF_ACCOUNT="your_account" | ||
| ``` | ||
|
|
||
| ### Running MySQL Tests on CI | ||
| To run **MySQL CI tests**, add the flag `[run mysql]` to your commit message. | ||
|
|
||
| **Running MySQL tests locally:** | ||
|
|
||
| 1. Make sure you have [**Docker Desktop**](https://www.docker.com/get-started/) | ||
| installed and running. | ||
|
|
||
| 2. Install the MySQL Connector for Python | ||
| ```bash | ||
| pip install mysql-connector-python | ||
| ``` | ||
|
|
||
| 3. Set your MySQL credentials as environment variables: | ||
| ```bash | ||
| export MYSQL_USERNAME="your_username" | ||
| export MYSQL_PASSWORD="your_password" | ||
|
|
||
| ## Runtime Dependencies | ||
|
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's specify that all of these flags are case insensitive