Embed quaint into prisma-engines #3905

miguelff · 2023-04-20T14:57:49Z

This PR:

embeds github.com/prisma/quaint into this repository's /quaint path, preserving quaint's history
sets up the test pipeline of quaint to trigger only when there are changes to /quaint/** or the PR is merged into main

Notes for reviewers:

/quaint is pulled from the git repository's main branch. You only need to focus on files outside that directory.

- Postgres BYTEA - SQLite datetime text format - deserialize_bytes in serde module

All macros now test both versions. The tag for 5.7 is `mysql` and for 8 `mysql8`.

Adds support of Common Table Expressions (CTE). The starting point is the `with` method of `Select`, allowing the user to add expressions to the queries `WITH` block. Additionally, on MSSQL queries if certain conditions apply, the query is converted to use the CTEs: - If comparing a tuple (a,b) against a selection of (SELECT x, y ..) - ... and both sides have as many variables, more than one Then a query where we do ```sql SELECT * FROM X WHERE (a, b) IN (SELECT x, y FROM ..) ``` converts into the following on SQL Server. ```sql WITH cte_0 AS (SELECT x, y FROM ..) WHERE a IN (SELECT x FROM cte_0 WHERE b = y) ``` This should work in every position of the query. The CTE is created to the first selection when going up the tree, adding multiple CTEs if having more than one tuple against select comparision, upping the index of the `cte` by one. Additionally, when comparing tuples, e.g. in a join context, we can do in the open source database the following: ```sql ON (a,b) = (x,y) ``` ... but in MSSQL we convert it to: ```sql ON a = x AND b = y ``` The same conversion applies to all tuple conversions in every position, either when comparing equals `=` or not `<>`.

Deal with it by actually having a compatibility function in the visiors.

- Defaults to `dbo` - Added to every query as an identifier

* Test and fix raw_cmd() on MySQL The previous behaviour was to return errors only for the first statement, without giving feedback on the success or failure of subsequent statements. This would also leave the connection in an invalid state for the next queries. We now consume all result sets from the server response in `raw_cmd()` on MySQL. This commit also includes as part of the fix an upgrade to mysql_async 0.24. * Use temporary tables in new tests * rustfmt * Move tests * Run raw_cmd test on all connectors

This does not fix everything, far from it, but this is a start.

This allows referring to possibly-nonexistent tables with their unqualified names in raw commands. Co-authored-by: Lily Chung <[email protected]>

This reverts commit 50a83cc.

)" (#196) This reverts commit 6fc2ff6.

* Ingores conditional docs generation as it cannot be applied to the value! macro. * Updates to docs to omit warnings * Uses upstream mobc * Updates connection-string dependencies

codspeed-hq · 2023-04-20T15:10:49Z

CodSpeed Performance Report

Merging #3905 quaint-embedding (3c7284f) will not alter performances.

Summary

🔥 0 improvements
❌ 0 regressions
✅ 6 untouched benchmarks

🆕 0 new benchmarks
⁉️ 0 dropped benchmarks

* quaint/main: (800 commits) chore: move all files into quaint subdir fix: set search path via query when pgbouncer is on (#453) Revert "Back to mainline mysql-async: (#450)" (#451) feat(pg): optimize search_path network roundtrip (#448) Back to mainline mysql-async: (#450) tech-debt: use upstream mobc + small fixes (#438) Add to_vec to a value (#446) Revert search_path and client-encoding optimization (#445) fix(pg): quote schema path (#443) Revert "Cargo.toml: connection-string 0.10.0 -> 0.14.0 (#437)" (#442) perf(pg): remove one roundtrip on connection (#441) Move rust-postgres fork to Prisma (#440) Cargo.toml: connection-string 0.10.0 -> 0.14.0 (#437) Revert "Updating to the latest mysql_async (#433)" (#435) We don't need to use our fork of mysql anymore (#434) Updating to the latest mysql_async (#433) chore: bump tiberius to 0.11.6 (#432) Emit query log in quaint:query span, rather than in its parent (#431) feat(PostgreSQL): support channel_binding query param (#424) added missing #[cfg(feature = "uuid")] (#420) ...

From: https://doc.rust-lang.org/cargo/reference/workspaces.html#the-members-and-exclude-fields > All path dependencies residing in the workspace directory automatically become members. [...] The moment we embedded quaint in prisma-engines, their crates became workspace members, and thus compiled as part of the build of the workspace. Because one of the crates in quaint (test-macros) had a circular dependency with quaint, this time enabling the `serde_support` flag, cargo tried to compile quaint with that feature enabled as part of the workspace, causing the compilation error in the `sql-query-connector` crate that assumed the `serde_support` feature was not enabled.

tomhoule

Very nicely executed!

tomhoule · 2023-04-25T07:41:28Z

quaint/db/test.db

Is this on quaint main too? That looks like something that was potentially committed by mistake. We can merge and delete later too.

It is, I don't know if by mistake also in quain yet. Will investigate and remove if that's the case!

Julius de Bruijn and others added 30 commits September 12, 2020 08:10

Count rows correctly when converting tuples

f5859a6

Fix MSSQL unique constraint failure error message

c77ba0a

Add extra conversions

f4394d1

- Postgres BYTEA - SQLite datetime text format - deserialize_bytes in serde module

(postgres) Enable bytea roundtrip test

9b1a5ed

Test null with pg bytea

8c58aad

Test sqlite text datetimes, add more parse options

8ddca52

Support bytea[] for pg

0f57f15

Fix MSSQL foreign key constraint error

ae096f6

Convert values to decimals for MSSQL avg

09a50f7

Run tests against MySQL 5.7 and 8

c6b0f02

All macros now test both versions. The tag for 5.7 is `mysql` and for 8 `mysql8`.

Do not have boolean flags for top level detection

539eb10

Deal with it by actually having a compatibility function in the visiors.

Rename all_columns into is_only_columns

3c3f80e

Fixing a typo

d1c8c0a

Document the CTE conversion to make it more clear

f2c23c2

Get rid of unnecessary visit_using

2570920

Document things

ea722fa

Replace filter+map with filter_map

f208622

Conversion can do multiple things, so it is plural

3e0ff03

Be sure to not include CTEs with duplicate idents

364199c

Allow setting the schema for MSSQL

617438c

- Defaults to `dbo` - Added to every query as an identifier

Allow setting the SQL Server isolation level

b3c3ce0

Render 0-offset for MS only if needed

f8ed5bf

Fixing uri parsing until we get the connection-string crate

32b7e53

Quick round of clippy (#193)

1c2ab14

This does not fix everything, far from it, but this is a start.

sqlite: open files using the main database (#192)

50a83cc

This allows referring to possibly-nonexistent tables with their unqualified names in raw commands. Co-authored-by: Lily Chung <[email protected]>

Revert "sqlite: open files using the main database (#192)" (#195)

6fc2ff6

This reverts commit 50a83cc.

Revert "Revert "sqlite: open files using the main database (#192)" (#195

4ccef81

)" (#196) This reverts commit 6fc2ff6.

Allow disabling TLS for SQL Server

2df8fdf

Weakky and others added 8 commits April 5, 2023 17:02

fix(pg): quote schema path (#443)

29654ec

Revert search_path and client-encoding optimization (#445)

5b6b61b

Add to_vec to a value (#446)

5a70831

tech-debt: use upstream mobc + small fixes (#438)

961e7e6

* Ingores conditional docs generation as it cannot be applied to the value! macro. * Updates to docs to omit warnings * Uses upstream mobc * Updates connection-string dependencies

Back to mainline mysql-async: (#450)

86bb661

feat(pg): optimize search_path network roundtrip (#448)

82bcf9f

Revert "Back to mainline mysql-async: (#450)" (#451)

7d5729d

fix: set search path via query when pgbouncer is on (#453)

5103129

miguelff added this to the 4.14.0 milestone Apr 20, 2023

miguelff force-pushed the quaint-embedding branch 2 times, most recently from 2464cfd to f907d17 Compare April 24, 2023 11:43

miguelff added 4 commits April 24, 2023 13:54

chore: move all files into quaint subdir

032ea59

Add a test workflow

2e5c453

miguelff force-pushed the quaint-embedding branch from 8ab1f8a to 2e5c453 Compare April 24, 2023 12:04

miguelff changed the title ~~(draft) Quaint embedding - preserving history~~ Embed quaint into prisma-engines Apr 24, 2023

miguelff marked this pull request as ready for review April 24, 2023 12:42

miguelff added 2 commits April 24, 2023 16:36

Run clippy for quaint too

c1f495b

Don't run the quaint workflow if quaint was not modified

3c7284f

miguelff force-pushed the quaint-embedding branch from 5589009 to 3c7284f Compare April 24, 2023 14:37

prisma deleted a comment from CLAassistant Apr 24, 2023

miguelff force-pushed the quaint-embedding branch from ef524ca to 3c7284f Compare April 24, 2023 15:27

This comment was marked as outdated.

Sign in to view

tomhoule approved these changes Apr 25, 2023

View reviewed changes

miguelff merged commit 5bfea92 into main Apr 25, 2023

miguelff deleted the quaint-embedding branch April 25, 2023 07:51

jkomyno mentioned this pull request Jul 17, 2023

API calls failing with Error querying the database: db error: FATAL: remaining connection slots are reserved for non-replication superuser connections prisma/prisma#7120

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Embed quaint into prisma-engines #3905

Embed quaint into prisma-engines #3905

miguelff commented Apr 20, 2023 •

edited

Loading

codspeed-hq bot commented Apr 20, 2023 •

edited

Loading

This comment was marked as outdated.

tomhoule left a comment

tomhoule Apr 25, 2023

miguelff Apr 25, 2023

Embed quaint into prisma-engines #3905

Embed quaint into prisma-engines #3905

Conversation

miguelff commented Apr 20, 2023 • edited Loading

codspeed-hq bot commented Apr 20, 2023 • edited Loading

CodSpeed Performance Report

Summary

This comment was marked as outdated.

tomhoule left a comment

Choose a reason for hiding this comment

tomhoule Apr 25, 2023

Choose a reason for hiding this comment

miguelff Apr 25, 2023

Choose a reason for hiding this comment

miguelff commented Apr 20, 2023 •

edited

Loading

codspeed-hq bot commented Apr 20, 2023 •

edited

Loading