Skip to content

fix: detect merged PRs to externally-owned repos for open source scoring - #376

Open
janampatel wants to merge 1 commit into
interviewstreet:mainfrom
janampatel:fix/detect-external-oss-contributions
Open

fix: detect merged PRs to externally-owned repos for open source scoring#376
janampatel wants to merge 1 commit into
interviewstreet:mainfrom
janampatel:fix/detect-external-oss-contributions

Conversation

@janampatel

Copy link
Copy Markdown

Problem

Open source scoring only ever looks at repos the candidate owns (GET /users/{username}/repos), and classifies open_source vs self_project by whether other people contributed to those repos. It never checks whether the candidate contributed to someone else's repo, which is what open source contribution actually means for most people, since it's usually done via fork + PR rather than owning a popular repo.

I hit this scoring my own resume: I have done several OS contributions, via a personal fork. The scorer gave me 5/35 on Open Source with "no contributions to other projects" as the evidence. Two reasons:

  1. The fork itself gets filtered out by if repo.get("fork") and repo.get("forks_count", 0) < 5: continue — a personal contribution fork will basically always have 0 forks of its own.
  2. Even without that filter, the tool only checks contributors on repos I own — never PRs I have made elsewhere.

Fix

  • github.py: new fetch_external_pr_contributions(username), using the Search API (author:{username} type:pr is:merged) to find merged PRs, filtered to repos owned by someone else. Added to fetch_and_display_github_info() as external_contributions.
  • transform.py: renders external_contributions into the === GITHUB DATA === block so the evaluator actually sees it.
  • roles/software_engineering_intern/criteria.jinja: treat this as the primary open-source signal, and only fall back to the old "self_project-only → cap at 10" rule when there's nothing here.

Verification

Ran python score.py resume/sample.pdf --role software_engineering_intern against my own GitHub profile, before and after:

  • Before: Open Source 5/35, evidence only referenced owned repos.
  • After: Open Source 25/35, evidence: "N merged PRs across M repositories".

Cross-checked the merged-PR count against the GitHub Search API directly. black github.py transform.py --check passes.

Note

Only counts merged PRs (is:merged) — open/unreviewed PRs aren't counted, on purpose.

GitHub enrichment only ever listed repos owned by the candidate
(GET /users/{username}/repos) and classified open_source vs
self_project by whether *other people* contributed to those owned
repos. It had no way to see PRs the candidate merged into someone
else's repo -- the actual definition of open source contribution --
so genuine external contributions made via the standard fork-and-PR
workflow were invisible to the scorer.

Add fetch_external_pr_contributions() using the Search API
(author:{username} type:pr is:merged), keep only PRs merged into
repos owned by someone else, and surface them as a new
"External Open Source Contributions" section in the GitHub data fed
to the evaluator. Update the open_source rubric to treat this as the
primary signal, ahead of the owned-repo contributor-count heuristic.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant