NeurIPS 2025 Metadata Pipeline

This folder contains the exact scripts and default outputs used to build the three public CSVs:

• neurips2025_accepted.csv
• neurips2025_arxiv_matches.csv
• neurips2025_semanticscholar_matches.csv

The code fetches accepted submissions from OpenReview, enriches the data with arXiv metadata, and then hydrates matches from the Semantic Scholar Graph API. Default outputs are checked in under outputs/ so the repository can be shared immediately, while build_outputs.py lets you regenerate everything end-to-end.

Layout

• neurips2025_export.py – downloads accepted NeurIPS 2025 papers from OpenReview.
• neurips2025_arxiv_match_all.py & neurips_arxiv_match.py – query arXiv (title + author match with a ≥3 s inter-request delay).
• neurips2025_semanticscholar_match_all.py & get_semanticscholar_data.py – hydrate Semantic Scholar metadata (default 3 s delay, exponential backoff for 429/5xx).
• build_outputs.py – orchestration script that runs all steps and produces outputs/neurips2025_papers.csv, a convenience merge of the three CSVs.
• requirements.txt – minimal Python dependencies.
• outputs/ – default CSV artifacts ready for sharing or verification.

Prerequisites

1. Python 3.10+.

2. Install dependencies:

   pip install -r requirements.txt

3. OpenReview credentials via environment variables or CLI flags:

   • OPENREVIEW_USERNAME and OPENREVIEW_PASSWORD, or
   • OPENREVIEW_TOKEN (for the API v2 token flow).

4. (Optional but recommended) Semantic Scholar API key set as SEMANTIC_SCHOLAR_API_KEY to raise rate limits.

Regenerating the CSVs

python build_outputs.py

The script:

1. Calls neurips2025_export.py to refresh outputs/neurips2025_accepted.*.
2. Enriches arXiv data with a 3 s pause between requests (--arxiv-sleep).
3. Fetches Semantic Scholar summaries with 3 s between calls (--semanticscholar-sleep).
4. Produces the combined outputs/neurips2025_papers.csv.

Flags you may find useful:

• --limit N – run a quick dry-run on the first N papers.
• --skip-export/--skip-arxiv/--skip-semanticscholar – reuse existing artifacts and only run later stages.
• --out-dir PATH – store outputs in a different directory.

ArXiv strongly recommends ~3 s between requests, and Semantic Scholar publishes similar guidance; these values are the defaults. Feel free to increase them further if you expect to re-run often.