fix(zqlite): Postgres-correct LIKE/ILIKE on the SQLite replica - #6095
Merged
Conversation
The SQLite replica diverged from Postgres on pattern matching:
* Postgres LIKE is case-sensitive, but SQLite's LIKE operator is
case-insensitive by default, so plain LIKE matched too much.
* ILIKE was rewritten to LIKE, which only case-folds ASCII — non-English
ILIKE was effectively case-sensitive.
* No ESCAPE was emitted, but Postgres (and the in-memory IVM matcher in
zql/src/builder/like.ts) treat backslash as the default escape.
Fix, mirroring the IVM matcher so all three backends agree:
* Enable `PRAGMA case_sensitive_like = ON` on every connection so the bare
LIKE operator is case-sensitive (= Postgres LIKE).
* Compile ILIKE/NOT ILIKE as `lower(a) LIKE lower(b)`, using the
Unicode-aware lower() that @rocicorp/zero-sqlite3 provides via ICU.
* Emit an explicit `ESCAPE '\'` for all LIKE/ILIKE operators.
Full Unicode case-insensitivity for ILIKE requires the ICU-enabled build of
@rocicorp/zero-sqlite3; until that ships, ILIKE remains ASCII-folded (no
regression) while LIKE case-sensitivity and escaping take effect immediately.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
@arv is attempting to deploy a commit to the Rocicorp Team on Vercel. A member of the Team first needs to authorize it. |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR aligns zqlite (the SQLite replica) LIKE/ILIKE behavior with Postgres and the in-memory IVM matcher so query results are consistent across backends.
Changes:
- Routes
LIKE/NOT LIKE/ILIKE/NOT ILIKEthrough a dedicated SQL builder that addsESCAPE '\'and implementsILIKEvialower(lhs) LIKE lower(rhs). - Enables
PRAGMA case_sensitive_like = ONfor eachDatabaseconnection so bareLIKEbecomes case-sensitive (Postgres-compatible). - Adds unit tests that pin the generated SQL for all four operators.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| packages/zqlite/src/query-builder.ts | Adds likeConditionToSQL() to generate Postgres-compatible LIKE/ILIKE SQL (including explicit ESCAPE). |
| packages/zqlite/src/query-builder.test.ts | Adds assertions for the exact SQL emitted for LIKE/NOT LIKE/ILIKE/NOT ILIKE. |
| packages/zqlite/src/db.ts | Sets case_sensitive_like pragma at connection creation time to match Postgres LIKE case-sensitivity. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Addresses review feedback: use the Database wrapper's pragma() method so the call goes through #run() for consistent SqliteError context and slow-query logging, matching the adjacent page_size pragma. Update the slow-query log test to expect the extra (now-logged) construction-time pragma. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The new `case_sensitive_like = ON` pragma in the zqlite Database constructor (routed through Database.pragma()) advances the run-ast tests' mocked clock by a fixed +2, shifting the snapshotted start/end timing values. elapsed is unchanged. No behavior change — only the deterministic timing offsets. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The SQLite replica (
zqlite) diverged from Postgres onLIKE/ILIKE, so query results could differ between the server-side replica, Postgres, and the in-memory IVM matcher:LIKEis case-sensitive, but SQLite'sLIKEoperator is case-insensitive by default → plainLIKEmatched too much.ILIKEwas rewritten toLIKE, which only case-folds ASCII → non-EnglishILIKEwas effectively case-sensitive (e.g.Ä/ä, Cyrillic, Greek didn't match).ESCAPEwas emitted, but Postgres and the in-memory IVM matcher (zql/src/builder/like.ts) both treat backslash as the default escape character.Fix
Mirror the in-memory IVM matcher so all three backends (Postgres, IVM, SQLite replica) agree:
PRAGMA case_sensitive_like = ONon every connection (db.ts) so the bareLIKEoperator is case-sensitive — matching PostgresLIKE.ILIKE/NOT ILIKEaslower(a) LIKE lower(b), using the Unicode-awarelower()that@rocicorp/zero-sqlite3provides via ICU — matching thetoLowerCase()the IVM matcher uses.ESCAPE '\'for allLIKE/ILIKEoperators.The generated SQL now looks like:
LIKE"name" LIKE ? ESCAPE '\'NOT LIKE"name" NOT LIKE ? ESCAPE '\'ILIKElower("name") LIKE lower(?) ESCAPE '\'NOT ILIKElower("name") NOT LIKE lower(?) ESCAPE '\'Dependency note
Full Unicode case-insensitivity for
ILIKErequires the ICU-enabled build of@rocicorp/zero-sqlite3(see companion PR rocicorp/zero-sqlite3#31, which compiles SQLite withSQLITE_ENABLE_ICU). Until that release is picked up:LIKEcase-sensitivity andESCAPE '\'take effect immediately (core SQLite).ILIKEremains ASCII-folded vialower()— i.e. no regression vs. today, and it upgrades to full Unicode automatically once the ICU build lands.Safety of the global pragma
case_sensitive_like = ONis connection-scoped. The only internal SQLiteLIKEs are the lowercase introspection patterns inlite-tables.ts('sqlite_%','_zero.%'), which match lowercase identifiers and stay correct under case-sensitivity. All other internalLIKEusage is in Postgres queries, which the pragma doesn't affect.Tests
query-builder.test.tspin the generated SQL for all four operators.table-source.test.ts/query.test.ts/db.test.tspass (45 tests).tscandoxfmtclean.🤖 Generated with Claude Code