Skip to content

docs(prompts): expand all template instructions with detailed extraction rules and validation - #394

Open
ThinkerDesigns wants to merge 1 commit into
interviewstreet:mainfrom
ThinkerDesigns:improve/prompt-details
Open

docs(prompts): expand all template instructions with detailed extraction rules and validation#394
ThinkerDesigns wants to merge 1 commit into
interviewstreet:mainfrom
ThinkerDesigns:improve/prompt-details

Conversation

@ThinkerDesigns

Copy link
Copy Markdown

Summary

Expands all prompt templates with significantly more detailed instructions, reducing LLM ambiguity. Each template grew from ~12 lines (a single sentence + schema) to 80–130+ lines of structured guidance.

What changed

Template Before After Key additions
system_message.jinja 1-line directive ~75 lines Parser persona, date normalization, title standardization, dedup rules, error handling
basics.jinja 2-line directive + URL hints ~80 lines Per-field extraction (name/email/phone/url/summary/location/profiles), validation checks, canonical naming examples
education.jinja 1-line directive ~95 lines 6-field schema with inference rules, reverse-chronological ordering, multi-degree handling, what to skip
work.jinja Basic work extraction + date rules ~105 lines Employment type classification (FT/intern/contract/fellowship/founder), role segmentation for promotions, remote detection
projects.jinja 1-line directive ~100 lines Project boundary detection, shared-technology propagation, technology mapping table, complexity signal guidance
skills.jinja 1-line directive ~105 lines Category classification logic, canonical naming table, cross-resume crawling rules (don't limit to skills section), level consistency
awards.jinja 1-line directive ~85 lines Award vs non-award classification, tier/rank handling, competition/scholarship detection, deduplication
github_project_selection.jinja Detailed but unstructured Restructured Tighter commit-tier tiers with explicit scoring bands, project-type classification rules (open_source vs self_project), stronger reasoning requirements per selection
resume_evaluation_system_message.jinja Dense 50-line wall of text ~130 lines Clear sectioned layout, per-category band guidance with evidence standards, bonus/audit logic organized step-by-step

Net change: 9 files, +878 / -248 lines. Total prompt instructions grew from ~12 lines to ~670+ lines.

Why this matters

The old prompts were essentially "extract X and return JSON" with no guardrails on how — leaving everything to LLM inference, which is the source of inconsistent parsing and scoring variance. These expanded instructions give the model explicit rules for:

  • Date normalization (YYYY-MM format, partial dates, season → month)
  • Canonical naming (JS → JavaScript, Node.js not nodejs, etc.)
  • Edge-case handling (empty fields → null vs empty string, deduplication, remote work indicators)
  • What to skip (classroom demos, placeholder text, implicit soft skills)

Files changed

prompts/templates/awards.jinja
prompts/templates/basics.jinja
prompts/templates/education.jinja
prompts/templates/github_project_selection.jinja
prompts/templates/projects.jinja
prompts/templates/resume_evaluation_system_message.jinja
prompts/templates/skills.jinja
prompts/templates/system_message.jinja
prompts/templates/work.jinja

How to test

Run a couple of real resumes through the pipeline:

python score.py ./resume/sample.pdf --role software_engineering_intern

Check that section extraction JSON matches expected output and scoring is consistent across multiple runs.

…ion rules and validation

- system_message: added parser persona, date normalization, title standardization, dedup rules
- basics: per-field extraction for name/email/phone/url/summary/location/profiles with canonical naming
- education: full 6-field schema, sorting, multi-entry handling, date inference
- work: employment type classification, role segmentation, remote detection, startup indicators
- projects: boundary detection, shared tech propagation, technology mapping table, complexity signals
- skills: category classification, canonical naming table, cross-resume crawling, level consistency
- awards: award vs non-award classification, tier handling, competition scoring, scholarship detection
- github_project_selection: tighter commit-tier logic, explicit project type rules, stronger reasoning requirements
- resume_evaluation_system_message: restructured with clear sections, per-category guidance, evidence standards

Net gain: ~12 to ~670 lines across 9 prompt files
Copilot AI review requested due to automatic review settings August 4, 2026 06:44

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR expands the resume parsing and evaluation prompt templates with detailed, structured extraction rules (dates, canonical naming, validation, deduplication, scoring bands) to reduce LLM ambiguity and improve output consistency across the pipeline.

Changes:

  • Added per-field extraction requirements and validation checks to section templates (basics/work/education/skills/projects/awards).
  • Restructured GitHub project selection prompt with explicit filtering, ranking tiers, and reasoning requirements.
  • Reworked resume evaluation system message into sectioned scoring guidance with bonuses/deductions and strict output constraints.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
prompts/templates/awards.jinja Expands award extraction rules, sorting/dedup guidance, and validation.
prompts/templates/basics.jinja Adds detailed per-field contact/profile extraction and normalization rules.
prompts/templates/education.jinja Adds detailed education schema mapping, ordering, and date handling guidance.
prompts/templates/github_project_selection.jinja Adds structured selection methodology, commit-tiering, and reasoning requirements.
prompts/templates/projects.jinja Adds project boundary detection, URL/tech normalization, and validation guidance.
prompts/templates/resume_evaluation_system_message.jinja Adds detailed scoring bands, bonuses/deductions, and strict JSON output constraints.
prompts/templates/skills.jinja Adds category logic, canonical naming table, and whole-resume skill crawling rules.
prompts/templates/system_message.jinja Adds global extraction rules (dates, dedup, mapping, formatting, error handling).
prompts/templates/work.jinja Adds structured work schema mapping, date rules, ordering, and validation checks.
Suppressed comments (4)

prompts/templates/work.jinja:31

  • The template instructs endDate="present" (lowercase) and includes an education-specific phrase ("graduations") in a work context. The codebase’s existing transformations emit "Present" (capitalized), so this will likely create inconsistent endDate values across the pipeline.
5. **endDate**:
   - Use the format `YYYY-MM` or `YYYY`.
   - For ongoing/current employment: `"present"` — always lowercase.
   - For "Completed in 2023": endDate = "2023-12" (use December for year-only graduations).
   - If no end date is given and the role is clearly past, infer from surrounding context or set to `null`.

prompts/templates/work.jinja:72

  • Validation references the literal string "present" (lowercase), but other parts of the pipeline output "Present" (capitalized). Keeping this aligned avoids inconsistent endDate values downstream.
- All dates must be valid YYYY-MM format strings or the literal string "present".

prompts/templates/system_message.jinja:19

  • This section uses lowercase "present" while other parts of the pipeline (e.g., transform.py) use "Present". Keeping a single canonical casing avoids inconsistent values across extracted sections.
3. **DATE RANGE HANDLING**:
   - For education: use the full duration of the degree program (enrollment to graduation, or enrollment to present).
   - For work experience: use employment start date to end date (or "present" if currently employed).

prompts/templates/education.jinja:36

  • This template uses endDate="present" (lowercase). Elsewhere in the pipeline ongoing dates are represented as "Present" (capital P), so aligning casing here avoids inconsistent outputs.
   - For ongoing education: `"present"`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- If no summary paragraph exists (only bullets), return `null`.

4. **startDate**:
- Use the format `YYYY-MM` or `YYYY` if only the year is available.
Comment on lines +12 to +15
2. **DATE NORMALIZATION**: Convert all date formats to `YYYY-MM` format (ISO 8601).
- "Jan 2023" → "2023-01", "March 2024" → "2024-03", "2020 – Present" → "2020-01" (start) with endDate as `"present"`
- "Summer 2023", "Spring 2024" → use the first month of that season (e.g., "2023-06", "2024-03")
- If only year is available (e.g., "2023"), set both startDate and endDate to `"2023-01"` (normalize to January).
- Remove filler phrases like "Proficient in:", "Familiar with:", "Expertise in:".
- Keep the skill name clean: "Python" not "Python (Django, Flask)", extract those as separate keywords if listed separately.

6. **DESCRIPION HANDLING**: For description/project/bullet-point fields:

4. **startDate**:
- Use the format `YYYY-MM` or `YYYY` if only the year is available.
- For "2021 – Present" or "2021-Present": startDate = "2021-01", endDate = `"present"`.
Comment on lines +57 to +60
**VALIDATION CHECKS:**
- Title must never be null or empty — if an entry has no clear award title, skip it.
- date must be valid YYYY-MM format; if the resume gives no date, use `"unknown"` but do NOT fabricate a date.
- awarder must never be null — at minimum, infer from context (e.g., the university name for academic awards).
Comment on lines +21 to +23
### 1. Author Commit Count (highest priority)
Projects where the candidate made significant contributions rank above all else. A project with 20 commits from the candidate is more impressive than a popular repo with only 2 of their commits.
- Prioritize: 15+ → 5–14 → exactly 4.
- **Tier 3 (10–99 stars)**: Smaller but active projects — interesting for context but lower weight.
- **Tier 4 (<10 stars)**: Barely known projects — rely on other criteria.

A small contribution to a Tier 1 project (e.g., fixing a bug in React's codebase) is often more impressive than building an entire personal project from scratch. Factor this into your ranking heavily.
Comment on lines +38 to +41
- If no end date is given and it's a past degree, infer from typical duration:
- Bachelor's: start + 4 years
- Master's: start + 2 years
- But ONLY if the resume indicates completion. If ambiguous, leave as `null`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants