fix(categorizer): normalize LLM category output for consistent casing - #1097
fix(categorizer): normalize LLM category output for consistent casing#1097Saumya-b10 wants to merge 2 commits into
Conversation
|
@Saumya-b10 is attempting to deploy a commit to the Karan Mani Tripathi 's projects Team on Vercel. A member of the Team first needs to authorize it. |
🤖 CodeAnt AI — Review Status
|
Thanks for using CodeAnt! 🎉We're free for open-source projects. if you're enjoying it, help us grow by sharing. Share on X · |
github-actions
Bot
added
gssoc'26
WalkthroughThe categorizer now normalizes model-generated labels by trimming whitespace, removing punctuation, collapsing spacing, title-casing words, and preserving likely acronyms. Empty normalized results fall back to ChangesCategory normalization
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
codeant-ai
Bot
added
the
size:M
|
@coderabbitai review |
github-actions
Bot
added
size/m
and removed
size:M
![codeant-ai[bot]](https://avatars.githubusercontent.com/in/646884?s=60&v=4){kind=small}
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/lib/ai/categorizer.ts`:
- Line 11: Expand the trailing-punctuation normalization in the categorizer to
follow an explicit supported punctuation policy, including closing delimiters,
Unicode ellipsis, and punctuation separated by spaces such as “Recursion . .”.
Add representative tests covering these cases and confirm normalized values are
stored as the same category.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: a224c918-52cc-43e4-895a-389beb33e2b2
📒 Files selected for processing (2)
src/__tests__/lib/categorizer.test.tssrc/lib/ai/categorizer.ts
There was a problem hiding this comment.
🧹 Nitpick comments (1)
src/__tests__/lib/categorizer.test.ts (1)
89-92: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winCover missing model content as well as empty content.
The new fallback coverage only returns
content: ''. Add a mock case with missingmessage.content(or an emptychoicesarray) and assert thatcategorizeDoubtstill returnsGeneral, matching the upstream fallback contract.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/__tests__/lib/categorizer.test.ts` around lines 89 - 92, Add a test case in the categorizer tests covering an upstream response with missing message.content or an empty choices array, and assert that categorizeDoubt returns General. Keep the existing quoted-content test unchanged and reuse the established mock setup for the fallback response.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@src/__tests__/lib/categorizer.test.ts`:
- Around line 89-92: Add a test case in the categorizer tests covering an
upstream response with missing message.content or an empty choices array, and
assert that categorizeDoubt returns General. Keep the existing quoted-content
test unchanged and reuse the established mock setup for the fallback response.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 4d91de6a-1cfb-499a-875a-0bfb2e3ddcb1
📒 Files selected for processing (2)
src/__tests__/lib/categorizer.test.tssrc/lib/ai/categorizer.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- src/lib/ai/categorizer.ts
1 participant
User description
PR: Normalize LLM categorization output for consistent casing/punctuation
Problem
categorizeDoubt stores the raw string returned by the LLM directly as the doubt's category, relying entirely on the system prompt instruction ("Respond ONLY with the sub-topic name. No punctuation or explanation") for formatting consistency. LLMs don't reliably follow output-format instructions on every call — casing and trailing punctuation can vary between otherwise-identical requests, even at low temperature (0.1).
This means the same underlying topic can silently be stored as multiple distinct strings, e.g.:
"Recursion"
"recursion"
"Recursion."
" Recursion "
Since this category is used for grouping/analytics (e.g. teacher dashboards showing which sub-topics students struggle with most), these get treated as separate categories instead of one — fragmenting the data. There's no error or crash, so this goes unnoticed until someone inspects a report and the numbers look wrong.
Change
Added a normalizeCategory() helper that deterministically cleans the model's raw output before it's returned/stored:
typescript
function normalizeCategory(raw: string): string {
return raw
.trim()
.replace(/[.,;:!?"'`]+$/g, "") // strip trailing punctuation
.replace(/\s+/g, " ") // collapse internal whitespace
.split(" ")
.filter(Boolean)
.map(w => w.charAt(0).toUpperCase() + w.slice(1).toLowerCase())
.join(" ");
}
Applied in categorizeDoubt:
typescript
const raw = completion.choices[0]?.message?.content?.trim() || "";
return normalizeCategory(raw) || "General";
The existing "General" fallback behavior is preserved for empty/missing responses.
Closes #994
CodeAnt-AI Description
Normalize AI-generated doubt categories for consistent grouping
What Changed
GeneralImpact
✅ Consistent category grouping✅ Cleaner analytics labels✅ Reliable fallback for empty responses💡 Usage Guide
Checking Your Pull Request
Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.
Talking to CodeAnt AI
Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:
This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.
Example
Preserve Org Learnings with CodeAnt
You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:
This helps CodeAnt AI learn and adapt to your team's coding style and standards.
Example
Retrigger review
Ask CodeAnt AI to review the PR again, by typing:
Check Your Repository Health
To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.
Summary by CodeRabbit