docs(prompts): expand all template instructions with detailed extraction rules and validation - #394
Open
ThinkerDesigns wants to merge 1 commit into
Open
Conversation
…ion rules and validation - system_message: added parser persona, date normalization, title standardization, dedup rules - basics: per-field extraction for name/email/phone/url/summary/location/profiles with canonical naming - education: full 6-field schema, sorting, multi-entry handling, date inference - work: employment type classification, role segmentation, remote detection, startup indicators - projects: boundary detection, shared tech propagation, technology mapping table, complexity signals - skills: category classification, canonical naming table, cross-resume crawling, level consistency - awards: award vs non-award classification, tier handling, competition scoring, scholarship detection - github_project_selection: tighter commit-tier logic, explicit project type rules, stronger reasoning requirements - resume_evaluation_system_message: restructured with clear sections, per-category guidance, evidence standards Net gain: ~12 to ~670 lines across 9 prompt files
There was a problem hiding this comment.
Pull request overview
This PR expands the resume parsing and evaluation prompt templates with detailed, structured extraction rules (dates, canonical naming, validation, deduplication, scoring bands) to reduce LLM ambiguity and improve output consistency across the pipeline.
Changes:
- Added per-field extraction requirements and validation checks to section templates (basics/work/education/skills/projects/awards).
- Restructured GitHub project selection prompt with explicit filtering, ranking tiers, and reasoning requirements.
- Reworked resume evaluation system message into sectioned scoring guidance with bonuses/deductions and strict output constraints.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| prompts/templates/awards.jinja | Expands award extraction rules, sorting/dedup guidance, and validation. |
| prompts/templates/basics.jinja | Adds detailed per-field contact/profile extraction and normalization rules. |
| prompts/templates/education.jinja | Adds detailed education schema mapping, ordering, and date handling guidance. |
| prompts/templates/github_project_selection.jinja | Adds structured selection methodology, commit-tiering, and reasoning requirements. |
| prompts/templates/projects.jinja | Adds project boundary detection, URL/tech normalization, and validation guidance. |
| prompts/templates/resume_evaluation_system_message.jinja | Adds detailed scoring bands, bonuses/deductions, and strict JSON output constraints. |
| prompts/templates/skills.jinja | Adds category logic, canonical naming table, and whole-resume skill crawling rules. |
| prompts/templates/system_message.jinja | Adds global extraction rules (dates, dedup, mapping, formatting, error handling). |
| prompts/templates/work.jinja | Adds structured work schema mapping, date rules, ordering, and validation checks. |
Suppressed comments (4)
prompts/templates/work.jinja:31
- The template instructs endDate="present" (lowercase) and includes an education-specific phrase ("graduations") in a work context. The codebase’s existing transformations emit "Present" (capitalized), so this will likely create inconsistent endDate values across the pipeline.
5. **endDate**:
- Use the format `YYYY-MM` or `YYYY`.
- For ongoing/current employment: `"present"` — always lowercase.
- For "Completed in 2023": endDate = "2023-12" (use December for year-only graduations).
- If no end date is given and the role is clearly past, infer from surrounding context or set to `null`.
prompts/templates/work.jinja:72
- Validation references the literal string "present" (lowercase), but other parts of the pipeline output "Present" (capitalized). Keeping this aligned avoids inconsistent endDate values downstream.
- All dates must be valid YYYY-MM format strings or the literal string "present".
prompts/templates/system_message.jinja:19
- This section uses lowercase "present" while other parts of the pipeline (e.g., transform.py) use "Present". Keeping a single canonical casing avoids inconsistent values across extracted sections.
3. **DATE RANGE HANDLING**:
- For education: use the full duration of the degree program (enrollment to graduation, or enrollment to present).
- For work experience: use employment start date to end date (or "present" if currently employed).
prompts/templates/education.jinja:36
- This template uses endDate="present" (lowercase). Elsewhere in the pipeline ongoing dates are represented as "Present" (capital P), so aligning casing here avoids inconsistent outputs.
- For ongoing education: `"present"`.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| - If no summary paragraph exists (only bullets), return `null`. | ||
|
|
||
| 4. **startDate**: | ||
| - Use the format `YYYY-MM` or `YYYY` if only the year is available. |
Comment on lines
+12
to
+15
| 2. **DATE NORMALIZATION**: Convert all date formats to `YYYY-MM` format (ISO 8601). | ||
| - "Jan 2023" → "2023-01", "March 2024" → "2024-03", "2020 – Present" → "2020-01" (start) with endDate as `"present"` | ||
| - "Summer 2023", "Spring 2024" → use the first month of that season (e.g., "2023-06", "2024-03") | ||
| - If only year is available (e.g., "2023"), set both startDate and endDate to `"2023-01"` (normalize to January). |
| - Remove filler phrases like "Proficient in:", "Familiar with:", "Expertise in:". | ||
| - Keep the skill name clean: "Python" not "Python (Django, Flask)", extract those as separate keywords if listed separately. | ||
|
|
||
| 6. **DESCRIPION HANDLING**: For description/project/bullet-point fields: |
|
|
||
| 4. **startDate**: | ||
| - Use the format `YYYY-MM` or `YYYY` if only the year is available. | ||
| - For "2021 – Present" or "2021-Present": startDate = "2021-01", endDate = `"present"`. |
Comment on lines
+57
to
+60
| **VALIDATION CHECKS:** | ||
| - Title must never be null or empty — if an entry has no clear award title, skip it. | ||
| - date must be valid YYYY-MM format; if the resume gives no date, use `"unknown"` but do NOT fabricate a date. | ||
| - awarder must never be null — at minimum, infer from context (e.g., the university name for academic awards). |
Comment on lines
+21
to
+23
| ### 1. Author Commit Count (highest priority) | ||
| Projects where the candidate made significant contributions rank above all else. A project with 20 commits from the candidate is more impressive than a popular repo with only 2 of their commits. | ||
| - Prioritize: 15+ → 5–14 → exactly 4. |
| - **Tier 3 (10–99 stars)**: Smaller but active projects — interesting for context but lower weight. | ||
| - **Tier 4 (<10 stars)**: Barely known projects — rely on other criteria. | ||
|
|
||
| A small contribution to a Tier 1 project (e.g., fixing a bug in React's codebase) is often more impressive than building an entire personal project from scratch. Factor this into your ranking heavily. |
Comment on lines
+38
to
+41
| - If no end date is given and it's a past degree, infer from typical duration: | ||
| - Bachelor's: start + 4 years | ||
| - Master's: start + 2 years | ||
| - But ONLY if the resume indicates completion. If ambiguous, leave as `null`. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Expands all prompt templates with significantly more detailed instructions, reducing LLM ambiguity. Each template grew from ~12 lines (a single sentence + schema) to 80–130+ lines of structured guidance.
What changed
system_message.jinjabasics.jinjaeducation.jinjawork.jinjaprojects.jinjaskills.jinjaawards.jinjagithub_project_selection.jinjaopen_sourcevsself_project), stronger reasoning requirements per selectionresume_evaluation_system_message.jinjaNet change: 9 files, +878 / -248 lines. Total prompt instructions grew from ~12 lines to ~670+ lines.
Why this matters
The old prompts were essentially "extract X and return JSON" with no guardrails on how — leaving everything to LLM inference, which is the source of inconsistent parsing and scoring variance. These expanded instructions give the model explicit rules for:
Files changed
How to test
Run a couple of real resumes through the pipeline:
Check that section extraction JSON matches expected output and scoring is consistent across multiple runs.