-
Notifications
You must be signed in to change notification settings - Fork 776
feat: implement AI-powered resume scoring to replace hardcoded endpoint #4377
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,109 @@ | ||||||||||||||
| import { getDefaultProvider } from '../config/aiProviders.js'; | ||||||||||||||
| import { ApiError } from '../middleware/errorHandler.js'; | ||||||||||||||
|
|
||||||||||||||
| /** | ||||||||||||||
| * Analyzes a resume using AI and returns a structured score with feedback | ||||||||||||||
| * @param {string} resumeText - The resume text to analyze | ||||||||||||||
| * @returns {Promise<Object>} - The analysis result with scores and feedback | ||||||||||||||
| */ | ||||||||||||||
| export async function analyzeResume(resumeText) { | ||||||||||||||
| if (!resumeText || !resumeText.trim()) { | ||||||||||||||
| throw new ApiError(400, 'Resume text is required'); | ||||||||||||||
| } | ||||||||||||||
|
Comment on lines
+10
to
+12
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Suggestion: The input guard assumes the value is a string and calls Severity Level: Major
|
||||||||||||||
|
|
||||||||||||||
| try { | ||||||||||||||
| const provider = getDefaultProvider(); | ||||||||||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Suggestion: This always forces the global default provider and ignores per-request AI provider/key selection used elsewhere via middleware, so BYOK/request-specific configuration cannot work for this flow. Accept an injected provider (or request context) instead of hardwiring default provider resolution in the service. [api mismatch] Severity Level: Major
|
||||||||||||||
|
|
||||||||||||||
| const prompt = `You are an expert resume analyst. Analyze the following resume and provide a detailed evaluation. | ||||||||||||||
|
|
||||||||||||||
| Return ONLY a valid JSON object with this exact structure: | ||||||||||||||
| { | ||||||||||||||
| "overallScore": number (0-100), | ||||||||||||||
| "sections": { | ||||||||||||||
| "summary": { | ||||||||||||||
| "score": number (0-100), | ||||||||||||||
| "feedback": "string (brief feedback)" | ||||||||||||||
| }, | ||||||||||||||
| "skills": { | ||||||||||||||
| "score": number (0-100), | ||||||||||||||
| "feedback": "string (brief feedback)" | ||||||||||||||
| }, | ||||||||||||||
| "experience": { | ||||||||||||||
| "score": number (0-100), | ||||||||||||||
| "feedback": "string (brief feedback)" | ||||||||||||||
| }, | ||||||||||||||
| "education": { | ||||||||||||||
| "score": number (0-100), | ||||||||||||||
| "feedback": "string (brief feedback)" | ||||||||||||||
| }, | ||||||||||||||
| "projects": { | ||||||||||||||
| "score": number (0-100), | ||||||||||||||
| "feedback": "string (brief feedback)" | ||||||||||||||
| } | ||||||||||||||
| }, | ||||||||||||||
| "topSuggestions": [ | ||||||||||||||
| "string (actionable suggestion)", | ||||||||||||||
| "string (actionable suggestion)", | ||||||||||||||
| "string (actionable suggestion)" | ||||||||||||||
| ] | ||||||||||||||
| } | ||||||||||||||
|
|
||||||||||||||
| Scoring criteria: | ||||||||||||||
| - Overall score should reflect the resume's overall quality | ||||||||||||||
| - Each section should be scored independently (0-100) | ||||||||||||||
| - Feedback should be specific and actionable | ||||||||||||||
| - Top suggestions should be the most important improvements needed | ||||||||||||||
| - Be honest but constructive in your assessment | ||||||||||||||
|
|
||||||||||||||
| Resume text: | ||||||||||||||
| ${resumeText}`; | ||||||||||||||
|
|
||||||||||||||
| const result = await provider.generateContent(prompt); | ||||||||||||||
|
Comment on lines
+15
to
+61
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift 🧩 Analysis chain🏁 Script executed: #!/bin/bash
# Inspect generateContent implementations for timeout handling
ast-grep run --pattern 'generateContent($$$) { $$$ }' --lang javascript backend/src/config/aiProviders.js
rg -n -A10 'generateContent' backend/src/config/aiProviders.jsRepository: anurag3407/career-pilot Length of output: 4200 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "== backend/src/config/aiProviders.js (relevant sections) =="
nl -ba backend/src/config/aiProviders.js | sed -n '1,230p'
echo
echo "== timeout / abort / race search across backend/src =="
rg -n "Promise\.race|AbortController|timeout|timed out|setTimeout\(" backend/src || true
echo
echo "== resume service call site =="
nl -ba backend/src/services/resumeService.js | sed -n '1,120p'Repository: anurag3407/career-pilot Length of output: 261 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "== backend/src/config/aiProviders.js =="
sed -n '1,240p' backend/src/config/aiProviders.js
echo
echo "== timeout/abort/race references in backend/src =="
rg -n "Promise\.race|AbortController|setTimeout\(|timeout|timed out" backend/src || true
echo
echo "== backend/src/services/resumeService.js =="
sed -n '1,140p' backend/src/services/resumeService.jsRepository: anurag3407/career-pilot Length of output: 18960 Add a timeout around
🤖 Prompt for AI Agents |
||||||||||||||
|
|
||||||||||||||
| // Strip markdown code blocks if present | ||||||||||||||
| let jsonText = result.text.trim(); | ||||||||||||||
| if (jsonText.startsWith('```')) { | ||||||||||||||
| jsonText = jsonText.replace(/^```(?:json)?\n?/, '').replace(/\n?```$/, '').trim(); | ||||||||||||||
| } | ||||||||||||||
|
|
||||||||||||||
| // Parse the JSON response | ||||||||||||||
| const analysisData = JSON.parse(jsonText); | ||||||||||||||
|
|
||||||||||||||
| // Validate the structure | ||||||||||||||
| if (!analysisData.overallScore || typeof analysisData.overallScore !== 'number') { | ||||||||||||||
| throw new Error('Invalid response: missing or invalid overallScore'); | ||||||||||||||
| } | ||||||||||||||
|
Comment on lines
+73
to
+75
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win Falsy check rejects a legitimate
🐛 Proposed fix- if (!analysisData.overallScore || typeof analysisData.overallScore !== 'number') {
+ if (typeof analysisData.overallScore !== 'number' || Number.isNaN(analysisData.overallScore)) {
throw new Error('Invalid response: missing or invalid overallScore');
}📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents
Comment on lines
+73
to
+75
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Suggestion: The numeric validation incorrectly treats zero as missing. Because the condition uses a falsy check before the type check, a valid score of 0 will always be rejected and converted into a 500 error. Check for number type first and then validate numeric range explicitly. [falsy zero check] Severity Level: Major
|
||||||||||||||
|
|
||||||||||||||
| if (!analysisData.sections || typeof analysisData.sections !== 'object') { | ||||||||||||||
| throw new Error('Invalid response: missing or invalid sections'); | ||||||||||||||
| } | ||||||||||||||
|
|
||||||||||||||
| const requiredSections = ['summary', 'skills', 'experience', 'education', 'projects']; | ||||||||||||||
| for (const section of requiredSections) { | ||||||||||||||
| if (!analysisData.sections[section] || typeof analysisData.sections[section].score !== 'number') { | ||||||||||||||
| throw new Error(`Invalid response: missing or invalid ${section} section`); | ||||||||||||||
| } | ||||||||||||||
| } | ||||||||||||||
|
|
||||||||||||||
| if (!analysisData.topSuggestions || !Array.isArray(analysisData.topSuggestions)) { | ||||||||||||||
| throw new Error('Invalid response: missing or invalid topSuggestions'); | ||||||||||||||
| } | ||||||||||||||
|
|
||||||||||||||
| return analysisData; | ||||||||||||||
| } catch (error) { | ||||||||||||||
| console.error('Error analyzing resume:', error); | ||||||||||||||
|
|
||||||||||||||
| // If JSON parsing failed, it's likely the AI didn't return valid JSON | ||||||||||||||
| if (error instanceof SyntaxError) { | ||||||||||||||
| throw new ApiError(500, 'Failed to analyze resume: AI returned invalid JSON format'); | ||||||||||||||
| } | ||||||||||||||
|
|
||||||||||||||
| // Re-throw ApiError instances | ||||||||||||||
| if (error instanceof ApiError) { | ||||||||||||||
| throw error; | ||||||||||||||
| } | ||||||||||||||
|
|
||||||||||||||
| // Generic error | ||||||||||||||
| throw new ApiError(500, 'Failed to analyze resume'); | ||||||||||||||
| } | ||||||||||||||
| } | ||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Suggestion: This line enables a paid AI call in an endpoint that is not protected by authentication or AI rate limiting, unlike other AI routes. That allows anonymous abuse and uncontrolled provider-cost consumption. Add the same auth/rate-limit middleware chain used by other AI endpoints before invoking the analysis. [security]
Severity Level: Critical 🚨
Steps of Reproduction ✅
(Use Cmd/Ctrl + Click for best experience)
Prompt for AI Agent 🤖