Skip to content

fix(scraper): parse Norrport's current calendar markup - #441

Merged
ranveeraggarwal merged 1 commit into
mainfrom
claude/lucid-cray-oa2aui
Oct 26, 2026
Merged

ranveeraggarwal merged 1 commit into
mainfrom
claude/lucid-cray-oa2aui

Conversation

@ranveeraggarwal

Copy link
Copy Markdown
Owner

The Norrport scraper was finding 0 events on norrport.se. That is why the 29 Oct Old Boy Stompers show (#440) had to be added by hand.

What changed on the site

  1. Date. The booking button used to carry the date in onclick="openBookingForEvent('2026-06-18', …)". It now carries it as data-date="2026-10-29". The parser read the onclick, so every card was dropped.
  2. End time. Cards used to read kl 19:00 - 22:00. Most now read kl 19:00, and the parser required a range.

What this PR does

  • Reads the date from data-date.
  • Accepts a start-only time. A missing end is filled as start + 3h, wrapping past midnight. end can't be left empty: ics.ts and jsonld.ts both compute from it, and no row in oneoffs.csv has an empty one. All 25 existing Norrport rows are exactly 3h (19:00–22:00 or 20:00–23:00). An explicit end on the card still wins.
  • Strips emoji and a trailing ! or : from band names. Old Boy Stompers! 🎉 was being written to the CSV with the emoji.
  • Refreshes fixtures/norrport.html to the live page and rewrites the Norrport tests around it. The new tests fail on the old parser (11 of 17) and pass now. They include a guard that the parser reads every bookable card, so a markup change that makes it return 0 events fails the build.
  • Updates the Norrport row in docs/architecture/SCRAPERS.md.

Private "Abonnerat" events (disabled button, no date) and STÄNGT days are still skipped. ABONNERAT is also already swing=no in bands.csv.

Checked

  • npm test passes (345 tests), eslint scripts/scrapers is clean, npm run validate:data passes.
  • node scripts/scrape.mjs --dry-run against the live site: Norrport yields 9 trusted events, of which 8 are new proposals and the 29 Oct show is skipped as already on the calendar (the hand-added row has the same norrport-2026-10-29 id). 9 unknown acts are surfaced for the bands PR. The other S:ta Clara, Chicago and Årstaliden fetches failed in my sandbox because of its network policy, so I did not verify those parsers.

Worth knowing

  • The 3h end is an assumption from the venue's pattern, not something the page says, and the scrape report doesn't flag it. If a human corrects an end time on a scraper-owned row, the runner will propose reverting it each night until the page changes. That is the existing behaviour tracked in Scraper re-proposes reverts on human-edited scraper-owned rows (needs field-level provenance) #66, and I didn't add a guard for it here.
  • The fixture is a raw snapshot of the live page, like the other fixtures, so the diff is large.

Generated by Claude Code

The scraper found 0 events on norrport.se. Two things on the page changed:

- The booking button now carries the date as `data-date`; the
  `openBookingForEvent('…')` onclick the parser read is gone, so every
  card was dropped.
- Cards list a start time only ("kl 19:00"); the parser required a
  "19:00 - 22:00" range. A missing end is filled as start + 3h, the slot
  every existing Norrport row uses, because `end` is required downstream.

Titles like "Old Boy Stompers! 🎉" also had the emoji and "!" written into
the band name; they are stripped now.

The fixture is refreshed to the live page, and the tests cover the
failure that shipped (0 events) plus explicit ends and midnight wrap.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tu5y7PyLZGtzXKELGRAEKq
@vercel

vercel Bot commented Oct 26, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
swsthlm Ready Ready Preview Oct 26, 2026 8:10pm UTC

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-26T20:11:47.531830Z 5eeb192 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@ranveeraggarwal
ranveeraggarwal merged commit a532ece into main Oct 26, 2026
4 checks passed
@ranveeraggarwal
ranveeraggarwal deleted the claude/lucid-cray-oa2aui branch October 26, 2026 20:11

This branch was successfully deployed

1 active deployment
Preview — 5eeb1927 Deployed Oct 26, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants