Skip to content

Tokenizer update #2410

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
ajtribick wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Tokenizer update #2410

ajtribick wants to merge 4 commits into from

Conversation

@ajtribick
Copy link
Collaborator

@ajtribick ajtribick commented Nov 29, 2025 •
edited
Loading

  • Use the new BufferedFile class to simplify the logic
  • Templatize getNumberValue to handle any numeric type, remove getIntegerValue
  • Do not unescape strings unless the value is read
  • Normalize line-endings when reading strings

@ajtribick ajtribick marked this pull request as ready for review November 29, 2025 22:08
@sonarqubecloud
Copy link

sonarqubecloud bot commented Nov 30, 2025

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@ajtribick ajtribick requested a review from a team December 2, 2025 05:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@ajtribick