Change the string token parsing to bump by total_len
, instead of remainder[0..total_len].as_bytes().len()
#681
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi, this is my first PR here. I was looking through how
taplo
parses inputs, and saw the string parsing functions could be slightly optimized.To explain my thought process for this change:
taplo
iterates over each character, addingc.len_utf8()
to a variable calledtotal_len
.taplo/crates/taplo/src/syntax.rs
Line 136 in 6694969
char::len_utf8()
returns the number of bytes that character would need in UTF-8. https://doc.rust-lang.org/1.70.0/std/primitive.char.html#method.len_utf8remainder[0..total_len].as_bytes().len()
bytes.taplo/crates/taplo/src/syntax.rs
Line 144 in 6694969
impl SliceIndex<str> for Range<usize>
takes in a start and end point in bytes. https://doc.rust-lang.org/1.70.0/std/primitive.str.html#impl-SliceIndex%3Cstr%3E-for-Range%3Cusize%3Eremainder[0..total_len]
in bytes will always betotal_len
. Thus, we can just replaceremainder[0..total_len].as_bytes().len()
withtotal_len
.I was not able to test this PR locally, as
cargo test
would fail due to issues compilingpprof
on my Windows machine, but I did use a separate branch in my fork to run the GitHub workflows, which you can see the results of here: https://github.com/LikeLakers2/taplo/actions/runs/10974635267If I missed anything, or if my thought process is wrong, please let me know. :) Thank you in advance for reviewing my PR!