Open
Conversation
Scrapes homepage carousel tiles for all 20 O2 Academy venues and Edinburgh Corn Exchange, surfaces outdated and expiring-soon tiles on a live dashboard, and sends alerts via email, Slack, and/or webhook when tiles need replacing. - server.js: Express server with axios+cheerio scraping, hourly cron job, SSE real-time updates, and nodemailer/Slack/webhook notifications - public/index.html: Dashboard with summary bar, filter buttons, and venue cards - public/app.js: SSE client, browser notifications, filter/refresh UI - public/style.css: Colour-coded status indicators (red/orange/green) - .env.example: SMTP, Slack, and webhook configuration template https://claude.ai/code/session_01JoKM72wEiTc7vswPJGxsSM
Replaces the Node.js server with a standalone index.html that works by opening directly in any browser — no install or server required. - Uses allorigins.win as a CORS proxy to fetch venue pages client-side - DOMParser replaces cheerio for HTML parsing (same selectors) - Skeleton cards render immediately while venues load in batches of 4 - Browser Notifications API for desktop alerts - Slack and generic webhook support via settings panel (localStorage) - Configurable warning threshold (default 7 days) - Auto-refreshes every hour via setTimeout https://claude.ai/code/session_01JoKM72wEiTc7vswPJGxsSM
Replaces the single allorigins.win proxy with an ordered fallback chain: corsproxy.io → allorigins (JSON) → codetabs. Each venue request tries all three in sequence before failing. Added AbortController timeout (18 s per attempt) and a custom proxy URL option in Settings. https://claude.ai/code/session_01JoKM72wEiTc7vswPJGxsSM
parseAMG now scans both carousel-slide elements AND the module-content-list-large/medium/small sections that carry the full event listing (16 events on O2 Ritz vs 6 from the carousel alone). Items with no parseable date are skipped to filter out navigation links that share the same selector. Deduplication prevents double-counting events that appear in both sections. https://claude.ai/code/session_01JoKM72wEiTc7vswPJGxsSM
…d more
Features added:
- Sort tiles by date ascending within every venue card
- Per-venue ↺ refresh button (event delegation, skips full scan)
- Cross-venue Events tab — flat date-sorted table of all events across
all venues, filtered by the same status pills and search box
- Artist/venue search — live-filters both card and events views
- Configurable auto-refresh interval (off / 15 min / 30 min / 1h / 2h)
replacing the hardcoded 1-hour timer; footer label updates dynamically
- Change detection — compares each scan against the previous (persisted
in localStorage); NEW badge on new tiles, REMOVED tiles shown
struck-through, changes summary in card footer
- Relative scan-age per card ("Checked 4m ago"), turns amber after 3h
and refreshes every 30s without re-rendering
- Export CSV button (header + footer) — downloads all events with venue,
artist, date, status, days-until, and is-new columns
- Total Events counter added to summary bar
- Settings modal now includes auto-refresh interval select
https://claude.ai/code/session_01JoKM72wEiTc7vswPJGxsSM
These are automatic content feeds, not manually maintained carousel tiles, so flagging them as outdated would be misleading. Added buildExcludedItems(doc): scans the parsed DOM for h1-h4 headings matching "What's New" or "Coming up" (handles straight and curly apostrophes), walks up to find the containing module, and marks every module-content-list-item inside it as excluded. Both parseAMG and parseECE now call buildExcludedItems and skip any item in the returned Set before adding a tile. https://claude.ai/code/session_01JoKM72wEiTc7vswPJGxsSM
- Add user authentication with bcrypt password hashing and express-session - Add SQLite database for persistent storage (users, venues, catalogue) - Add venue catalogue with add/remove/manage functionality - Add Server-Sent Events (SSE) for real-time scraping progress updates - Add db/ directory to .gitignore - Update dependencies: bcryptjs, better-sqlite3, express-session https://claude.ai/code/session_01JoKM72wEiTc7vswPJGxsSM
Adds Node.js engines field so Railpack detects the app correctly, and nixpacks.toml to provide gcc/python build tools for better-sqlite3. https://claude.ai/code/session_01JoKM72wEiTc7vswPJGxsSM
railway-app
bot
temporarily deployed
to
honest-fascination / production
Railway's nginx proxy buffers SSE responses by default, causing the dashboard to stay stuck on "Starting up..." after login. This header disables proxy buffering so events flow through immediately. https://claude.ai/code/session_01JoKM72wEiTc7vswPJGxsSM
railway-app
bot
temporarily deployed
to
honest-fascination / production
The page was calling the old client-side CORS proxy scraper (startScrape) at init, which conflicted with the server-side SSE system. Both shared the isScraping flag, causing the dashboard to stay stuck on "Starting up..." after login. Now the page calls initApp() which checks auth and connects to the server SSE, and auto-refresh uses triggerRefresh() (server API) instead of the client-side scraper. https://claude.ai/code/session_01JoKM72wEiTc7vswPJGxsSM
railway-app
bot
temporarily deployed
to
honest-fascination / production
SSE on Railway can be delayed or dropped before the first event arrives, leaving the dashboard stuck on "Starting up...". After login (and on page load if already authenticated), immediately fetch /api/status via a normal HTTP request to render the current state, then use SSE only for live real-time updates. https://claude.ai/code/session_01JoKM72wEiTc7vswPJGxsSM
railway-app
bot
temporarily deployed
to
honest-fascination / production
In-memory sessions are lost on every process restart (deploy, crash).
Replace the default MemoryStore with a minimal SQLite-backed store using
the already-available better-sqlite3 connection, so sessions survive
restarts. Also add app.set('trust proxy', 1) so Express correctly reads
X-Forwarded-Proto from Railway's edge proxy.
On the client, show an error banner if /api/status fails instead of
silently swallowing the error and leaving the grid stuck on "Starting up...".
https://claude.ai/code/session_01JoKM72wEiTc7vswPJGxsSM
railway-app
bot
temporarily deployed
to
honest-fascination / production
Railway's HAProxy terminates long-lived SSE connections. After 3 consecutive SSE errors the client now switches to polling /api/status every 20 s instead of showing a permanent "Connection lost" banner. SSE ping interval also reduced from 25 s to 15 s to stay well under Railway's 60-second idle timeout so SSE works if the proxy allows it. https://claude.ai/code/session_01JoKM72wEiTc7vswPJGxsSM
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.