ClickHouse SQL Parsing Test Results

[alexey-milovidov]

Tested polyglot-sql v0.1.4 ClickHouse dialect against 7,425 .sql files from ClickHouse/tests/queries/0_stateless/.

Summary

Metric	Total	OK	Errors	Success Rate
Files	7,425	5,100	2,325	68.7%
Statements	141,952	~82,788	~59,164	58.3%

Note: The statement count is approximate — when a file fails to parse, all its statements are counted as errors because the parser stops at the first error.

Common Error Categories

ClickHouse-Specific Syntax Not Supported

• Ternary operator ? : (e.g., x > 0 ? 1 : 0)
• IGNORE() function — treated as a keyword instead of a function
• Tuple element access via dot on literals: ('a', 'b').2
• ARRAY JOIN with AS inside array literals: [0 as x]
• DROP TEMPORARY TABLE — TEMPORARY not recognized after DROP
• system keyword treated as reserved: system.one, system.numbers
• Nested column access via dot in column lists: INSERT INTO t (n.a, n.b)
• Lambda expressions: x -> -x in function arguments
• Parametric aggregate functions: groupArrayInsertAt('-')(...) (two argument lists)
• WITH TOTALS after GROUP BY inside subqueries
• CREATE TABLE ... AS SELECT ... (table from subquery)
• CREATE TABLE ... AS db.table (table cloning)
• DEFAULT column values in CREATE TABLE: b DEFAULT 0
• SET as table name (reserved word conflict)
• FORMAT clause: ... FORMAT JSONEachRow {...}
• INSERT ... FORMAT with inline data
• Enum8/Enum16 types with negative values

DDL and Other Constructs

• ALTER TABLE ... CLEAR COLUMN IN PARTITION
• MergeTree(date, id, 1) — old-style engine parameters
• Nested(...) column type
• set allow_deprecated_syntax_for_merge_tree=1 — parsed but affects subsequent DDL

[tobilg]

Thank you for the detailed analysis! I will try to improve coverage as soon as possible

[alexey-milovidov]

Also, no hex escape sequences in string literals: SELECT '\x40Hello''\'World\\';

[tobilg]

Hey,

I have some additional questions. I checked out the current master branch, and apparently there are "only" 7235 sql files found. My preliminary checks also only find 123,717 statements. Could you share some insights how you did this?

Furthermore, do you think some of the statement (types) don't make sense for polyglot to parse them? I found things like:

│                                    Filter                                     │ Count   │
│ system.* table references                                                     │ 3,004   │
│ ClickHouse-specific functions (file, url, s3, remote, merge, currentDatabase) │ 2,438   │
│ Error-only files (<=3 statements with serverError)                            │ 423     │

Thanks in advance!

[tobilg]

Also, no hex escape sequences in string literals: SELECT '\x40Hello''\'World\\';

This is fixed in the next 0.1.6 release

[alexey-milovidov]

I checked out the current master branch, and apparently there are "only" 7235 sql files found

milovidov@milovidov-pc:~/work/ClickHouse$ ls -1 tests/queries/0_stateless/*.sql | wc -l
7442

Maybe just checked out at a different time.

[alexey-milovidov]

Never mind, I have some uncommitted files: https://pastila.nl/?01ecfb3a/6f92e177fa716895a3ba7426e7c482a6#eDzNWMtMnCsMX7nE2BR8vw==GCM

[tobilg]

Thanks! I chose to use ClickHouse stable tags to extract the tests from, this makes it a bit easier for me to follow, see https://github.com/tobilg/polyglot/blob/main/Makefile#L24

It now uses 7,047 SQL tests files for parser tests. Version 0.1.6 was just released. Thanks again!

[tobilg]

I'll close this issue now, feel free to reopen it again in case I missed something. Thank you