Skip to content

feat: M4 content — connectors, library, transcript player and local search - #4

Closed
EliseuSantos wants to merge 3 commits into
feat/m3-plannerfrom
feat/m4-content
Closed

feat: M4 content — connectors, library, transcript player and local search#4
EliseuSantos wants to merge 3 commits into
feat/m3-plannerfrom
feat/m4-content

Conversation

@EliseuSantos

Copy link
Copy Markdown
Owner

Stacked on #3 (M3 planner). Merge order: #1#2#3 → this.

What

Milestone 4 (content), complete — search a subject, attach a video to a topic, open it with a synced transcript, and find anything through local full-text search:

  • Connectors (packages/connectors): Connector interface + registry; Wikipedia and Stack Exchange run client-side (CORS-friendly APIs), YouTube goes through the Worker proxy. All fetch-injectable, ≤10 results, failures degrade to empty — never throw. Plus a parseTimedText transcript parser (entity-decoding, attribute-order agnostic).
  • Worker proxy: /proxy/youtube/search (Data API key as optional secret — 503 when absent), /proxy/youtube/transcript (pt → en fallback), /proxy/rss (https-only, private-host blocklist, 5s timeout). All bearer-auth'd and cached via a testable Cache API seam (6h/24h/1h).
  • Local FTS5 search (packages/db/search.ts): search_index virtual table created client-side only (D1 lacks FTS5, shared migrations untouched), rebuilt via atomic reindexAll, queried with per-token prefix + bm25 + snippets; hostile input sanitized.
  • UI: /library with parallel unified search, source filter chips and an attach-to-topic picker; /library/watch/[id] — youtube-nocookie embed with clickable transcript cues (seek via postMessage) and follow mode; content list on the topic view; global search combobox in the header (keyboard navigable, reindex debounced on table changes).

Tests

  • 143 bun tests (connectors fixtures, proxy incl. cache-hit/503/private-host cases, content repo, FTS matching/sanitization/bm25 order)
  • Cypress 16 specs green — new library-loop.cy.ts with fully intercepted network: stubbed sources → attach → topic content link → transcript renders → global FTS finds topic and content

🤖 Generated with Claude Code

EliseuSantos and others added 3 commits July 10, 2026 12:36
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…earch

- connectors: wikipedia/stackexchange/youtube (fetch-injectable, <=10
  results, failures return empty) + timedtext transcript parser
- worker: /proxy/youtube/search|transcript and /proxy/rss with Cache API
  seam (testable), optional YOUTUBE_API_KEY (503 when absent)
- db: content repo (attach/list/delete via oplog) + local-only FTS5
  search_index (ensure/reindex/searchLocal with bm25 + snippet)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…earch

- /library: parallel connector search (wikipedia, stack exchange, youtube
  via authed proxy), source filter chips, attach-to-topic picker
- /library/watch/[id]: youtube-nocookie embed + transcript panel with
  click-to-seek and follow mode (postMessage api)
- topic content list on track view; global search combobox in the header
  over the local FTS5 index (reindex debounced on table changes)
- e2e: library-loop.cy.ts with fully intercepted network; ignore cypress
  downloads dir

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant