feat : added scientific notation support in TSS tokenizer - #3200
feat : added scientific notation support in TSS tokenizer#3200tmdeveloper007 wants to merge 7 commits into
Conversation
…cate active property
|
Warning Review limit reached
Next review available in: 44 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (9)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary of What Has Been Done
Extended the TSS tokenizer in
packages/tss/src/tokenizer.tsto recognize scientific notation (e.g.,1e5,1.5e-2,2E+3) as valid numeric tokens.Previously, the number tokenizer only accepted digits and decimal points, so values like
font-size: 1.5e-2would incorrectly tokenize thee-2portion as a separateIdenttoken.Changes Made
packages/tss/src/tokenizer.ts
eorE, optionally consumes+or-, then consumes consecutive digits.packages/tss/src/tokenizer.test.ts
1e5)1.5e-2)2E+3)15 tokenizer tests pass.
Impact it Made
Enables TSS files to use scientific notation for numeric values, which is valid CSS syntax. Developers can now use compact scientific notation like
opacity: 1e-3;instead of spelling out0.001.Closes #3195
**Note: Please assign this PR to the
tmdeveloper007account.