Use sqlglot[rs]

[erikmunkby]

Hi!

First of all, really excited to see this project and great work! I'm a big fan and user of sqlglot so any speedup in the world of SQL parsing is music to my ears. Looking forward to potential python bindings.

When trying out the benchmark I noticed you did not use the rust tokenizer for sqlglot (sqlglot[rs]), I would suggest changing to those to give as fair comparison as possible. The difference is (geometric mean) ~8x faster to ~7x faster which is still really exciting.

["uv", "run", "--with", "sqlglot", "python3", os.path.join(PROJECT_ROOT, "tools", "bench-compare", "bench_sqlglot.py")]

To:

["uv", "run", "--with", "sqlglot[rs]", "python3", os.path.join(PROJECT_ROOT, "tools", "bench-compare", "bench_sqlglot.py")]

link

Could not find any contributing guidelines, so let me know if you'd prefer a different approach, happy to adjust.

Thanks again for building this!

[tobilg]

Thank you for the PR, I‘ll have a look!