Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SQLite3::Database#execute_batch fails with non-ASCII characters #524

Closed
Math2 opened this issue Apr 17, 2024 · 4 comments · Fixed by #526
Closed

SQLite3::Database#execute_batch fails with non-ASCII characters #524

Math2 opened this issue Apr 17, 2024 · 4 comments · Fixed by #526

Comments

@Math2
Copy link

Math2 commented Apr 17, 2024

Before v2.0.0, the following used to work:

require 'sqlite3'

db = SQLite3::Database.new ':memory:'

db.execute_batch %{
  CREATE TABLE test(t TEXT);
  INSERT INTO test VALUES ('héhé');
}

But now fails with:

/usr/home/mathieu/.gem/ruby/3.2/gems/sqlite3-2.0.0/lib/sqlite3/statement.rb:31:in `encode': "\xC3" from ASCII-8BIT to UTF-8 (Encoding::UndefinedConversionError)
        from /usr/home/mathieu/.gem/ruby/3.2/gems/sqlite3-2.0.0/lib/sqlite3/statement.rb:31:in `initialize'
        from /usr/home/mathieu/.gem/ruby/3.2/gems/sqlite3-2.0.0/lib/sqlite3/database.rb:166:in `new'
        from /usr/home/mathieu/.gem/ruby/3.2/gems/sqlite3-2.0.0/lib/sqlite3/database.rb:166:in `prepare'
        from /usr/home/mathieu/.gem/ruby/3.2/gems/sqlite3-2.0.0/lib/sqlite3/database.rb:250:in `execute_batch'
        from ./test.rb:5:in `<main>'
@flavorjones
Copy link
Member

@Math2 Thanks for reporting this. I'll take a look!

@flavorjones
Copy link
Member

Bisecting indicates a9a89f8 is the commit that introduced this behavior. It looks like the "tail" string comes back from #prepare as ASCII-8BIT because we're doing the encoding slightly differently.

I've got a fix, just writing up some tests and I'll create the PR shortly.

flavorjones added a commit that referenced this issue Apr 19, 2024
@flavorjones
fix: Database#execute_batch properly handles non-ASCII strings
3c67c7b 
Fix a regression in v2.0.0 that caused `Database#execute_batch` to
raise an encoding exception when passed some non-ASCII strings. As a
result of this fix, `Database#prepare` now ensures the "remainder"
string will always be encoded as UTF-8.

Closes #524
@flavorjones flavorjones mentioned this issue Apr 19, 2024
Merged
@flavorjones
Copy link
Member

See #526 for the proposed fix.

flavorjones added a commit that referenced this issue Apr 20, 2024
@flavorjones
fix: Database#execute_batch properly handles non-ASCII strings
f4c326e 
Fix a regression in v2.0.0 that caused `Database#execute_batch` to
raise an encoding exception when passed some non-ASCII strings. As a
result of this fix, `Database#prepare` now ensures the "remainder"
string will always be encoded as UTF-8.

Closes #524
@flavorjones
Copy link
Member

Fix released in https://github.com/sparklemotion/sqlite3-ruby/releases/tag/v2.0.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: Database#execute_batch properly handles non-ASCII strings sparklemotion/sqlite3-ruby
2 participants
@flavorjones @Math2