Skip to content

Add escape sequence support for string literals (\t, \n, \r, etc.) #2331

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Copilot wants to merge 3 commits into
base: develop
Choose a base branch
from
Draft

Add escape sequence support for string literals (\t, \n, \r, etc.) #2331

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
69bc486
Initial plan
Copilot Dec 7, 2025
25a5322
Add escape sequence support to AlaSQL strings (\t, \n, \r, etc.)
Copilot Dec 7, 2025
57cfff3
Address PR feedback: Remove \b \f \v support, fix regex, update tests
Copilot Dec 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions src/58json.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ const JSONtoString = (alasql.utils.JSONtoString = function (obj) {

function JSONtoJS(obj, context, tableid, defcols) {
var s = '';
if (typeof obj == 'string') s = '"' + obj + '"';
if (typeof obj == 'string') s = '"' + escapeq(obj) + '"';
Copy link
Member

@mathiasrw mathiasrw Dec 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot

What is we have a string of data that contains \n, will it then get converted?

Copy link
Contributor Author

Copilot AI Dec 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is the correct behavior. When data contains actual newline characters (after SQL parsing converts \n to a newline), escapeq() converts them back to \n when generating JavaScript code. This ensures the generated JS code is syntactically valid.

Flow:

  1. SQL 'hello\nworld' → Parser converts to actual newline → Stored in memory as hello<LF>world
  2. When generating JS code → escapeq() converts back to \n → Generated code: "hello\nworld"
  3. JS runtime executes → Results in newline character again

Without escapeq(), strings with newlines/quotes would produce invalid JavaScript syntax.

else if (typeof obj == 'number') s = '(' + obj + ')';
else if (typeof obj == 'boolean') s = obj;
else if (typeof obj === 'bigint') s = obj.toString() + 'n';
Expand All @@ -56,7 +56,7 @@ function JSONtoJS(obj, context, tableid, defcols) {
} else if (!obj.toJS || obj instanceof yy.Json) {
let ss = [];
for (const k in obj) {
let keyStr = typeof k === 'string' ? `"${k}"` : k.toString();
let keyStr = typeof k === 'string' ? `"${escapeq(k)}"` : k.toString();
let valueStr = JSONtoJS(obj[k], context, tableid, defcols);
ss.push(`${keyStr}:${valueStr}`);
}
Expand Down
38 changes: 36 additions & 2 deletions src/alasqlparser.jison
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1460,9 +1460,43 @@ LogicValue

StringValue
: STRING
{ $$ = new yy.StringValue({value: $1.substr(1,$1.length-2).replace(/(\\\')/g,"'").replace(/(\'\')/g,"'")}); }
{
var str = $1.substr(1,$1.length-2).replace(/(\'\')/g,"'");
str = str.replace(/\\(.)/g, function(match, char) {
Copy link
Member

@mathiasrw mathiasrw Dec 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You dont know if the first \ is escaped beforehand. You will need to find pairs like \ (regex as \\) or not pair (what you got now \(.))

Please see if you can place regex outside of the executing function

Copy link
Contributor Author

Copilot AI Dec 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed the regex to be more specific: /\\(n|t|r|\\|'|")/g instead of /\\(.)/g. This only matches the supported escape sequences and properly handles \\ without needing to worry about pairs. (57cfff3)

switch(char) {
case 'n': return '\n';
case 't': return '\t';
case 'r': return '\r';
case 'b': return '\b';
case 'f': return '\f';
case 'v': return '\v';
case '\\': return '\\';
Copy link
Member

@mathiasrw mathiasrw Dec 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check if this is being solved via #2335

case "'": return "'";
case '"': return '"';
default: return char;
}
});
$$ = new yy.StringValue({value: str});
}
| NSTRING
{ $$ = new yy.StringValue({value: $1.substr(2,$1.length-3).replace(/(\\\')/g,"'").replace(/(\'\')/g,"'")}); }
{
var str = $1.substr(2,$1.length-3).replace(/(\'\')/g,"'");
str = str.replace(/\\(.)/g, function(match, char) {
switch(char) {
case 'n': return '\n';
case 't': return '\t';
case 'r': return '\r';
case 'b': return '\b';
case 'f': return '\f';
case 'v': return '\v';
case '\\': return '\\';
case "'": return "'";
case '"': return '"';
default: return char;
}
});
$$ = new yy.StringValue({value: str});
}
;

NullValue
Expand Down
38 changes: 36 additions & 2 deletions src/alasqlparser.js
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 6 additions & 5 deletions test/test125.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,12 @@ describe('Test 125 - remove comments', function () {
var res = alasql("select 'Cote d''Ivoir'");
assert.deepEqual(res, [{"'Cote d'Ivoir'": "Cote d'Ivoir"}]);
var res = alasql('select "Cote d\\"Ivoir"');
assert.deepEqual(res, [{"'Cote d\\\"Ivoir'": 'Cote d\\"Ivoir'}]);
var res = alasql('select "\\r"');
assert.deepEqual(res, [{"'\\r'": '\\r'}]);
var res = alasql('select "\\n"');
assert.deepEqual(res, [{"'\\n'": '\\n'}]);
assert.deepEqual(res, [{"'Cote d\"Ivoir'": 'Cote d"Ivoir'}]);
// Test escape sequences with aliases to make expectations clearer
var res = alasql('select "\\r" as col');
assert.deepEqual(res, [{col: '\r'}]);
var res = alasql('select "\\n" as col');
assert.deepEqual(res, [{col: '\n'}]);

alasql('drop database test125');
done();
Expand Down
74 changes: 74 additions & 0 deletions test/test134-B.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
if (typeof exports === 'object') {
var assert = require('assert');
var alasql = require('..');
}

describe('Test 134-B - Escape sequences in strings', function () {
const test = '134B';

before(function () {
alasql('create database test' + test);
alasql('use test' + test);
});

after(function () {
alasql('drop database test' + test);
});

it('A) Tab escape sequence', function () {
var res = alasql("SELECT 'hello\\tworld' AS result");
assert.deepEqual(res, [{result: 'hello\tworld'}]);
});

it('B) Newline escape sequence', function () {
var res = alasql("SELECT 'line1\\nline2' AS result");
assert.deepEqual(res, [{result: 'line1\nline2'}]);
});

it('C) Carriage return escape sequence', function () {
var res = alasql("SELECT 'text\\rmore' AS result");
assert.deepEqual(res, [{result: 'text\rmore'}]);
});

it('D) Backslash escape sequence', function () {
var res = alasql("SELECT 'back\\\\slash' AS result");
assert.deepEqual(res, [{result: 'back\\slash'}]);
});

it('E) Form feed escape sequence', function () {
var res = alasql("SELECT 'page\\fbreak' AS result");
assert.deepEqual(res, [{result: 'page\fbreak'}]);
});

it('F) Backspace escape sequence', function () {
var res = alasql("SELECT 'back\\bspace' AS result");
assert.deepEqual(res, [{result: 'back\bspace'}]);
});

it('G) Double quote escape sequence', function () {
var res = alasql('SELECT "quote\\"here" AS result');
assert.deepEqual(res, [{result: 'quote"here'}]);
});

it('H) Mixed escape sequences', function () {
var res = alasql("SELECT 'Line 1\\nLine 2\\tTabbed\\rCarriage' AS result");
assert.deepEqual(res, [{result: 'Line 1\nLine 2\tTabbed\rCarriage'}]);
});

it('I) Escape sequence in WHERE clause', function () {
alasql('CREATE TABLE test_escapes (id INT, text STRING)');
alasql("INSERT INTO test_escapes VALUES (1, 'hello\\tworld')");
alasql("INSERT INTO test_escapes VALUES (2, 'no tabs here')");

var res = alasql("SELECT * FROM test_escapes WHERE text = 'hello\\tworld'");
assert.deepEqual(res, [{id: 1, text: 'hello\tworld'}]);
});

it('J) Single quote still works', function () {
var res = alasql("SELECT 'Cote d\\'Ivoir' AS result");
assert.deepEqual(res, [{result: "Cote d'Ivoir"}]);

var res2 = alasql("SELECT 'Cote d''Ivoir' AS result");
assert.deepEqual(res2, [{result: "Cote d'Ivoir"}]);
});
});