refactor(token-estimation): use nullish checks

[steebchen]

Summary

• Refactors the estimateTokens function to replace falsy checks with explicit null and undefined checks
• Improves accuracy and clarity when estimating missing prompt and completion tokens

Changes

Token Estimation Logic

• Replaced if (!promptTokens || !completionTokens) with explicit checks for null and undefined
• Updated conditions to check promptTokens === null || promptTokens === undefined and similarly for completionTokens
• Ensured token estimation for prompt and completion tokens only triggers when values are truly missing (null or undefined), avoiding false negatives for zero or other falsy values
• Maintained existing logic for encoding messages and content to estimate tokens when necessary

Test plan

• Verified that token estimation triggers correctly when promptTokens or completionTokens are null or undefined
• Confirmed no regression in token counting accuracy
• Tested with various message inputs and content to ensure proper token calculation

🌿 Generated by Terry

---

ℹ️ Tag @terragon-labs to ask questions and address PR feedback

📎 Task: https://www.terragonlabs.com/task/7ff2611b-57c3-462f-9ee0-9efe344e0bec

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches

🧪 Generate unit tests

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch terragon/refactor-token-estimation-nullish-checks

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}