Skip to content
Merged
Show file tree
Hide file tree
Changes from 51 commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
6abec11
Enable de-correlation of pure semi/anti joins, including pullup for p…
knassre-bodo Jun 17, 2025
9b4fb04
Fix correl 16 WIP
knassre-bodo Jun 17, 2025
319797f
Fixed bugs, need to deal with window squished CTEs for correl #16
knassre-bodo Jun 18, 2025
634585c
Merge branch 'main' into kian/decor_semi_anti
knassre-bodo Jun 18, 2025
6361404
Resolving conflicts [RUN CI]
knassre-bodo Jun 19, 2025
e4764ca
Squash merge Hadia/sf changes onto kian/decor_semi_anti
hadia206 Jun 20, 2025
8dbcd9c
[run SF]
hadia206 Jun 20, 2025
0c3167e
[run SF] add connection object as option for load_snowflake_connection
hadia206 Jun 20, 2025
19baea0
Use SF Functions directly instead of PyDough changes (#377)
hadia206 Jun 20, 2025
716f1aa
Merge branch 'main' of https://github.com/bodo-ai/PyDough into Hadia/sf
hadia206 Jul 7, 2025
f5c80b4
[run CI] [run SF] merge main and re-run tests
hadia206 Jul 7, 2025
f6e1f5e
Merge branch 'Hadia/sf' of https://github.com/bodo-ai/PyDough into Ha…
hadia206 Jul 7, 2025
e0a135d
[run CI] [run SF] merge and re-run tests
hadia206 Jul 7, 2025
597f5f9
merge
hadia206 Aug 5, 2025
95895c8
update tests and fix some SF issues
hadia206 Aug 6, 2025
a2c4d1f
add new test files
hadia206 Aug 6, 2025
7fbcdc6
update dayname
hadia206 Aug 6, 2025
0a8ac03
fix date_trunc
hadia206 Aug 6, 2025
1b6a84b
[run SF] [run CI] handle change in start of week
hadia206 Aug 6, 2025
15c0242
comment
hadia206 Aug 6, 2025
84cf1d7
Merge branch 'main' of https://github.com/bodo-ai/PyDough into Hadia/sf
hadia206 Aug 8, 2025
91dc3bb
[run CI] [run SF] fix week
hadia206 Aug 11, 2025
c8cc53f
[run CI] [run SF] update some SQL files
hadia206 Aug 11, 2025
ad0a1dd
[run SF][run CI] fix date_diff, add defog, 1 test only for params, docs
hadia206 Aug 11, 2025
a24829d
add decimal/int none/empty to harmonize [run CI]
hadia206 Aug 13, 2025
7a87630
add defog tests
hadia206 Aug 13, 2025
04ee554
modify sf_conn_db_context to accept db and schema as arguments
hadia206 Aug 13, 2025
b4d77de
handle relsum(boolean)
hadia206 Aug 14, 2025
fd8cbfb
fix length in slicing
hadia206 Aug 14, 2025
bdb71c5
relsum to sum
hadia206 Aug 14, 2025
2a6717b
add defog metatdata and SQL
hadia206 Aug 19, 2025
70cd17f
update PyDoughSQLComparisonTest to use reference database
hadia206 Aug 19, 2025
4ee0ea3
updates
hadia206 Aug 21, 2025
fc2f910
merge conflicts
hadia206 Aug 21, 2025
510ef58
update tests
hadia206 Aug 21, 2025
43dbdc7
task script and check in connection
hadia206 Aug 21, 2025
30b3812
task script and check in connection
hadia206 Aug 21, 2025
a703b41
fix date_trunc in sqlite and uncomment row
hadia206 Aug 21, 2025
8c40bac
add run-all option
hadia206 Aug 22, 2025
296f0c1
mysql in dev dependency
hadia206 Aug 22, 2025
dc06546
add connectors to docs
hadia206 Aug 22, 2025
55545c2
move week7 in week_offset
hadia206 Aug 22, 2025
1558b7c
update try/except for mysql
hadia206 Aug 22, 2025
377b523
move week7 in week_offset
hadia206 Aug 22, 2025
fcc698b
remove str_count
hadia206 Aug 22, 2025
ae623b3
[run all] update mysql init script
hadia206 Aug 25, 2025
3de8b89
undo changes for sql text
hadia206 Aug 25, 2025
ed8f82e
modify week_offset to be like mysql for consistency
hadia206 Aug 25, 2025
1c33249
modify week_offset to be like mysql for consistency
hadia206 Aug 25, 2025
5f31407
[run all] disable dialect for SF only
hadia206 Aug 25, 2025
8a28f4a
[run all] fix link and remove old test
hadia206 Aug 25, 2025
35c1c2e
address comments [run all]
hadia206 Aug 26, 2025
0d6eddf
Fixing MySQL testing issues [RUN ALL]
knassre-bodo Aug 26, 2025
45b43de
Fixing start of week problems in MySQL setup script [RUN ALL]
knassre-bodo Aug 26, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 11 additions & 3 deletions .github/workflows/pr_testing.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,11 @@ on:
description: "Python 3.12"
type: boolean
default: true
run-all:
description: "Run All Tests"
type: boolean
required: false
default: false
run-python:
description: "Run Main PyDough Tests"
type: boolean
Expand Down Expand Up @@ -101,8 +106,9 @@ jobs:
# https://docs.github.com/en/actions/learn-github-actions/expressions#contains
# contains is case-insensitive
if: |
(github.event_name == 'pull_request' && contains(needs.get-msg.outputs.commitMsg, '[run all]')) ||
(github.event_name == 'pull_request' && contains(needs.get-msg.outputs.commitMsg, '[run ci]')) ||
(github.event_name == 'workflow_dispatch' && inputs.run-python)
(github.event_name == 'workflow_dispatch' && (inputs.run-all || inputs.run-python))
runs-on: ubuntu-latest
strategy:
matrix:
Expand Down Expand Up @@ -138,8 +144,9 @@ jobs:
name: Snowflake Tests
needs: [get-msg, get-py-ver-matrix]
if: |
(github.event_name == 'pull_request' && contains(needs.get-msg.outputs.commitMsg, '[run all]')) ||
(github.event_name == 'pull_request' && contains(needs.get-msg.outputs.commitMsg, '[run sf]')) ||
(github.event_name == 'workflow_dispatch' && inputs.run-sf)
(github.event_name == 'workflow_dispatch' && (inputs.run-all || inputs.run-sf))
uses: ./.github/workflows/sf_testing.yml
secrets:
SF_USERNAME: ${{ secrets.SF_USERNAME }}
Expand All @@ -154,8 +161,9 @@ jobs:
name: MySQL Tests
needs: [get-msg, get-py-ver-matrix]
if: |
(github.event_name == 'pull_request' && contains(needs.get-msg.outputs.commitMsg, '[run all]')) ||
(github.event_name == 'pull_request' && contains(needs.get-msg.outputs.commitMsg, '[run mysql]')) ||
(github.event_name == 'workflow_dispatch' && inputs.run-mysql)
(github.event_name == 'workflow_dispatch' && (inputs.run-all || inputs.run-mysql))
uses: ./.github/workflows/mysql_testing.yml # Path to MySQL workflow file
secrets:
MYSQL_USERNAME: ${{ secrets.MYSQL_USERNAME }}
Expand Down
43 changes: 41 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,8 +97,47 @@ script must be run so that the `defog.db` file is located in the `tests` directo

## Running CI Tests

To run our CI tests on your PR, you must include the flag `[run CI]` in latest
commit message.
When submitting a PR, you can control which CI tests run by adding special flags
to your **latest commit message**.
Comment on lines +100 to +101
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's specify that all of these flags are case insensitive


- To run **PyDough CI tests**, add: `[run CI]`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Specify that this one will not run tests from any sql database except sqlite

- To run **PyDough and all dialect tests**, add: `[run all]`
- To run **specific dialect tests**, use the corresponding flag as described below.

### Running Snowflake Tests on CI
To run **Snowflake CI tests**, add the flag `[run SF]` to your commit message.

**Running Snowflake tests locally:**

1. Install the Snowflake Connector for Python with Pandas support
```bash
pip install "snowflake-connector-python[pandas]"
```

2. Set your Snowflake credentials as environment variables:
```bash
export SF_USERNAME="your_username"
export SF_PASSWORD="your_password"
export SF_ACCOUNT="your_account"
```

### Running MySQL Tests on CI
To run **MySQL CI tests**, add the flag `[run mysql]` to your commit message.

**Running MySQL tests locally:**

1. Make sure you have [**Docker Desktop**](https://www.docker.com/get-started/)
installed and running.

2. Install the MySQL Connector for Python
```bash
pip install mysql-connector-python
```

3. Set your MySQL credentials as environment variables:
```bash
export MYSQL_USERNAME="your_username"
export MYSQL_PASSWORD="your_password"

## Runtime Dependencies

Expand Down
1 change: 1 addition & 0 deletions demos/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,5 @@ Once the introduction notebook is complete, you can explore the other notebooks:
- [4_tpch.ipynb](notebooks/4_tpch.ipynb) provides PyDough translations for most of the TPC-H benchmark queries.
- [5_what_if.ipynb](notebooks/5_what_if.ipynb) demonstrates how to do WHAT-IF analysis with PyDough.
- [MySQL_TPCH.ipynb](notebooks/MySQL_TPCH.ipynb) demonstrates how to connect a MySQL database with PyDough.
- [SF_TPCH_q1.ipynb](notebooks/SF_TPCH_q1.ipynb) demonstrates how to connect a Snowflake database with PyDough.

2 changes: 1 addition & 1 deletion demos/notebooks/MySQL_TPCH.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
"> You can also test with our **pre-built MySQL TPC-H database** available on **Docker Hub**.\n",
">\n",
"> #### 📋 Requirements\n",
"> - Make sure you have **Docker** installed.\n",
"> - Make sure you have **Docker Desktop** installed and running.\n",
">\n",
"> #### 📦 Pull and Run the Container\n",
"> ```bash\n",
Expand Down
Loading