Skip to content

PDF year extraction grabs first body year, not publication year #70

Description

@kytmanov

Symptom

~/my-wikiV06-4/raw/2606-00164v1-d00ee217.md frontmatter has year: 1946 for an arXiv June-2026 HST paper ("Astronomical Advantages of a Boost Mission to Facilitate HST Science into the 2030s…"). The other three ingested papers extracted correct years (2026/2025/2026).

Root cause

src/synto/extractors/pdf.py:348

year_match = re.search(r"\b(19|20)\d{2}\b", first_page_md)  # takes FIRST year on page 1

Page 1 prose: "…homage to Lyman Spitzer's 1946 RAND Project report …", so the regex matches 1946. The PDF creationDate is consulted only when the body yields no year at all (pdf.py:351-355) — backwards for papers, which almost always cite some year in prose before the publication year appears.

Blast radius

Low. The value lives only in raw/*.md frontmatter. It is not stored in source_documents (metadata_json is empty), not rendered on wiki/sources/* pages, and not fed into compile prompts (compile uses source_title, which extracted correctly). Not a v0.6.0 release blocker, but it produces visibly wrong metadata.

Suggested fix

Prefer the PDF creationDate (and/or an arXiv-id-derived date) as the primary signal, with the body regex as fallback — or anchor the regex to a date / "submitted on" context near the title block rather than first-match-on-page.

Repro

Ingest 2606.00164v1.pdf; inspect the generated raw/<id>.md frontmatter year.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions