Add comprehensive documentation for architecture and data systems

README.md

@@ -4,6 +4,8 @@
 
 A fully featured forum site powered entirely by **GitHub Discussions** — no backend, no database. It builds to plain static files and runs on **GitHub Pages**. Built with **SvelteKit** (Svelte 5) and **Tailwind CSS v4**.
 
+Deep-dive documentation lives in [`docs/`](docs/README.md) — architecture, [data syncing](docs/data-sync.md) (with flow diagram), [reputation](docs/reputation.md), and the [read-only archive](docs/archive.md).
+
 ## How it works
 
 - **Topics** are your repository's Discussion categories.

docs/README.md

@@ -0,0 +1,10 @@
+# Discussion Kit — documentation
+
+Deep-dives into how the system works. For setup and configuration, start with the [project README](../README.md).
+
+| Document | Covers |
+| --- | --- |
+| [Architecture](architecture.md) | The big picture: static SPA, GitHub Discussions as the database, auth, caching |
+| [Data syncing](data-sync.md) | The `data` branch, the sync workflow, and the full trigger → process → storage → fetch flow (with diagram) |
+| [Reputation system](reputation.md) | How rep is earned, computed, displayed, and enforced |
+| [Read-only archive](archive.md) | The cold-store archive and anonymous browsing mode |

docs/architecture.md

@@ -0,0 +1,35 @@
+# Architecture
+
+Discussion Kit is a **fully client-side SvelteKit SPA with no backend and no database**. GitHub Discussions is the data store, the GitHub GraphQL API is the data layer, and GitHub Pages serves the static build.
+
+```
+┌────────────────────┐         GraphQL (auth required)        ┌──────────────────┐
+│  Browser (SPA)     │ ─────────────────────────────────────► │  GitHub API      │
+│  SvelteKit static  │                                        │  (Discussions)   │
+│  build on Pages    │ ◄───────────────────────────────────── │                  │
+└────────────────────┘         raw file fetch (no auth)       └──────────────────┘
+          ▲                                                            ▲
+          │              ┌──────────────────┐   scans + writes         │
+          └───reads──────│  `data` branch   │◄────── GitHub Actions ───┘
+                         │  (machine-owned) │        (data-sync.yml)
+                         └──────────────────┘
+```
+
+## Core pieces
+
+- **Routing / pages** (`src/routes/`) — home feed, topic pages (`/t/[slug]`), threads (`/d/[number]`), user profiles (`/u/[login]`), search, new-post composer. The build is a static SPA (`adapter-static` with a `404.html` fallback), so dynamic routes work on GitHub Pages via the client router.
+- **API layer** (`src/lib/github/api.ts`) — every GraphQL query/mutation the app makes. Discussions, comments, reactions, upvotes, search, markdown rendering, and user profiles. Responses GitHub can't be fully trusted on (per-user activity scoping) are re-filtered client-side.
+- **Auth** (`src/lib/github/auth.svelte.ts`) — each visitor authenticates with *their own* GitHub account, either by pasting a fine-grained PAT or via OAuth through a tiny Cloudflare Worker proxy (`oauth-proxy/`) that holds the client secret. Tokens live in `localStorage` only and are sent solely to `api.github.com`.
+- **Cache** (`src/lib/cache.ts`) — stale-while-revalidate over `localStorage`: pages paint instantly from the last known data while a background request refreshes them. Entries are versioned, scoped per signed-in user, and wiped on sign-out.
+- **Config** (`forum.config.ts` + `src/lib/config/schema.ts`) — everything is optional and deep-merged over defaults; the repo can be auto-detected from the Actions build environment so forks deploy with zero code changes.
+- **Machine-written data** (the `data` branch) — reputation files and the cold-store archive, maintained by GitHub Actions and read by the SPA over raw URLs. See [Data syncing](data-sync.md).
+
+## Trust model
+
+- GitHub enforces all permissions server-side (posting, editing, deleting, moderation) — the UI only mirrors them.
+- Post bodies are rendered by GitHub (`bodyHTML`), so markdown rendering and HTML sanitisation are GitHub's, not ours.
+- Anything the workflow writes to the `data` branch is derived and recomputable; nothing on it is hand-edited or authoritative beyond the last sync.
+
+## Why sign-in is required for live data
+
+The GraphQL API refuses anonymous calls and there is no server to hold a shared token. Signed-out visitors either see a sign-in prompt or — when the [archive](archive.md) is enabled — browse a read-only snapshot served from the `data` branch.

docs/archive.md

@@ -0,0 +1,46 @@
+# Read-only archive (anonymous mode)
+
+Optional (off by default). Solves Discussion Kit's biggest UX limitation: the GitHub API refuses anonymous calls, so without this feature signed-out visitors can read **nothing**.
+
+```ts
+archive: { enabled: true }
+```
+
+With it enabled, the [data-sync workflow](data-sync.md) snapshots every discussion to the `data` branch, and **signed-out visitors browse that snapshot read-only** — served straight from `raw.githubusercontent.com`, which allows anonymous, CORS-open fetches on public repositories.
+
+## What's stored
+
+```
+data
+├── meta.json                    # categories, pinned posts, archivedAt
+└── posts/
+    ├── index.json               # list rows for the home/topic feeds
+    └── <number>/content.json    # one full thread
+```
+
+Every file is shaped **exactly like the SPA's own API types** (`Discussion`, `DiscussionListItem`), so the existing pages and components render archive data unchanged — anonymous mode is a different *data source*, not a different UI. Threads store GitHub-rendered `bodyHTML` (GFM, syntax highlighting, sanitisation all done by GitHub), and viewer-specific fields (`viewerHasReacted`, `viewerHasUpvoted`) are neutralised since the snapshot is identical for everyone.
+
+## What a signed-out visitor gets
+
+| Works | Doesn't (routes to sign-in) |
+| --- | --- |
+| Home feed, category cards, pinned posts | Posting, commenting, replying |
+| Topic pages with post/article filtering | Reactions and upvotes |
+| Full threads: rendered bodies, comments, replies, reaction counts | Search (API-only) |
+| Lightweight profiles: archived posts, rep, comment count | Profile READMEs, comment details, GitHub join date |
+
+Plus a persistent banner — *"Read-only snapshot — sign in to post, comment, and react."* — and a "not yet archived" notice on posts created after the last sync. Signing in mid-session re-boots the app onto the live API transparently.
+
+## How the SPA decides which source to use
+
+`archiveMode()` in [`src/lib/ui.svelte.ts`](../src/lib/ui.svelte.ts) is true when `archive.enabled && !signedIn`. On boot:
+
+- **Signed in** → the normal authenticated GraphQL bootstrap (categories, permission, home feed) with SWR caching.
+- **Signed out + archive enabled** → fetch `meta.json` + `posts/index.json`, apply the same topic include/exclude filtering, and mark the session as archive-booted. Thread and profile pages then fetch their archive files on demand.
+- **Signed out + no archive yet** (workflow never ran) → the classic sign-in prompt, unchanged.
+
+## Constraints
+
+- **Public repositories only.** Raw URLs require auth on private repos, which would defeat the purpose.
+- **Freshness**: a few minutes behind live (workflow runtime + ~5 min CDN cache); reactions/upvotes refresh on the 6-hourly cron. See [Data syncing → Freshness](data-sync.md#freshness).
+- The archive is a *mirror*, not a backup with history — each sync replaces the previous snapshot entirely.

docs/data-sync.md

@@ -0,0 +1,80 @@
+# Data syncing
+
+One GitHub Actions workflow — [`data-sync.yml`](../.github/workflows/data-sync.yml) running [`data-sync.ts`](../.github/scripts/data-sync.ts) — maintains everything on the machine-owned `data` branch: the [reputation](reputation.md) files and the [read-only archive](archive.md). This page walks the full pipeline from a user action to the data landing back in someone's browser.
+
+## The flow
+
+```mermaid
+flowchart TD
+    subgraph triggers ["1 · Triggers"]
+        U1["User posts / comments / replies<br/>(forum UI or github.com)"]
+        U2["Post or comment edited,<br/>deleted, moved, answered"]
+        CRON["Cron — every 6 h<br/>(reactions have no webhook)"]
+        MANUAL["Manual run<br/>(workflow_dispatch)"]
+    end
+
+    U1 --> EV["GitHub event<br/>discussion / discussion_comment"]
+    U2 --> EV
+    EV --> Q
+    CRON --> Q
+    MANUAL --> Q
+
+    subgraph workflow ["2 · Data-sync workflow (GitHub Actions)"]
+        Q["Concurrency queue<br/>(one run at a time, no races)"]
+        SCAN["Scan ALL discussions via GraphQL<br/>posts · comments · replies ·<br/>answers · reactions · categories · pins"]
+        COMPUTE["Recompute from scratch<br/>rep ledger (engine.ts: gains, daily caps)<br/>+ archive JSON (bodyHTML, neutralised)"]
+        ENFORCE{"Under-rep post in<br/>a gated topic?"}
+        MOD["Moderate via API:<br/>move / lock / delete + bot comment"]
+        SNAP["Write snapshot tree to OUT_DIR"]
+
+        Q --> SCAN --> COMPUTE --> ENFORCE
+        ENFORCE -- yes --> MOD --> SNAP
+        ENFORCE -- no --> SNAP
+    end
+
+    subgraph storage ["3 · Storage — data branch (orphan, machine-owned)"]
+        PUSH["git force-push single fresh commit<br/>(creates the branch if missing)"]
+        TREE["meta.json<br/>posts/index.json · posts/&lt;n&gt;/content.json<br/>profiles/index.json · profiles/&lt;login&gt;/rep.json"]
+        PUSH --> TREE
+    end
+
+    SNAP --> PUSH
+
+    subgraph fetching ["4 · Fetching — browser (SPA)"]
+        CDN["raw.githubusercontent.com<br/>(no auth · CORS * · ~5 min CDN cache)"]
+        ANON["Signed-out visitor<br/>browses archive read-only:<br/>home · topics · threads · profiles"]
+        AUTHED["Signed-in visitor<br/>live GraphQL data +<br/>rep badges & topic gating from profiles/"]
+    end
+
+    TREE --> CDN
+    CDN --> ANON
+    CDN --> AUTHED
+```
+
+## Why "recompute from scratch"?
+
+Every run rescans the whole forum and rebuilds every file, instead of incrementally patching:
+
+- **Idempotent** — two runs over the same forum state produce byte-identical output, so concurrent or repeated triggers can never corrupt anything; the concurrency queue just serialises pushes.
+- **Lossless under load** — GitHub keeps at most one queued run per concurrency group. With incremental updates a superseded run would mean lost events; with recompute, whichever run lands last has everything.
+- **Self-healing** — deleted posts stop being counted and archived automatically; a failed run is fully repaired by the next one.
+
+## Why snapshot force-pushes?
+
+The branch holds only derived, recomputable data, so its git history has no value. Each run commits one fresh snapshot and force-pushes it:
+
+- The branch stays at **one commit forever** — no repo bloat from thousands of tiny data commits.
+- Stale files (deleted posts, renamed users) vanish because the whole tree is rewritten.
+- A force-push to a branch that doesn't exist **creates it** — which is the entire bootstrap story. No manual setup, no seed commit.
+
+## Interaction with branch protection
+
+`main` keeps its rulesets (PRs, tests, 100 % coverage) untouched — the workflow never pushes there. The `data` branch is an orphan with no shared history, so the two can never be merged into each other accidentally. If your rulesets target *all* branches rather than `main`, exclude the data branch or add an Actions bypass, otherwise the workflow's push is rejected.
+
+## Freshness
+
+| Path | Latency |
+| --- | --- |
+| Post/comment/edit events | workflow runtime (~30–60 s) + raw-URL CDN cache (~5 min) |
+| Reactions & upvotes | next 6-hourly cron (no webhook exists for them) |
+| Manual dispatch | same as events |

docs/reputation.md

@@ -0,0 +1,57 @@
+# Reputation system
+
+Optional (off by default). Users earn rep for forum activity; topics can require a minimum rep to post — a middle ground between fully open topics and maintainer-only announcements. Enabled and tuned entirely from [`forum.config.ts`](../forum.config.ts):
+
+```ts
+rep: {
+	enabled: true,
+	gains: { post: 5, comment: 2, answerAccepted: 15 },
+	dailyCaps: { post: 25, comment: 10 },  // rep per UTC day, 0 = uncapped
+	topics: { showcase: 50 },              // slug → min rep to post
+	onViolation: 'move',                   // 'move' | 'lock' | 'delete'
+	fallbackTopic: 'general'
+}
+```
+
+## Earning rep
+
+| Action | Default rep | Daily cap |
+| --- | --- | --- |
+| Creating a post | +5 | 25/day |
+| Commenting or replying | +2 | 10/day |
+| Having an answer accepted | +15 | never capped |
+
+Design notes:
+
+- **Rep is a pure function of forum activity.** The [data-sync workflow](data-sync.md) recomputes the entire ledger from scratch on every run ([`src/lib/rep/engine.ts`](../src/lib/rep/engine.ts) — unit-tested at 100 % coverage). There are no increments to lose or corrupt, and deleting content automatically removes the rep it granted.
+- **Daily caps** bucket per user, per UTC day, per action type, applied in chronological order so the result is deterministic regardless of scan order. They exist to blunt farming: past the cap, more spam earns nothing.
+- **Accepted answers are uncapped** because someone *else* grants them — the one signal that can't be self-farmed.
+- Logins are lowercased everywhere (GitHub treats them case-insensitively).
+
+## Where rep is stored and shown
+
+The workflow writes to the `data` branch:
+
+- `profiles/index.json` — compact `login → rep` map. One fetch powers every rep chip next to usernames in lists, comments, and threads.
+- `profiles/<login>/rep.json` — per-user detail: total rep plus a raw action breakdown (`post` / `comment` / `answerAccepted` counts).
+
+The SPA shows rep next to usernames (beside badges), on profile pages (before the post/comment counters), and uses it to gate the topic picker. Before the first sync lands, everyone displays **0 rep**.
+
+## Gating and enforcement
+
+Two layers, one honest limitation:
+
+1. **UI gating (soft).** Gated topics disappear from the `/new` topic picker and show a "Requires N rep" chip on their topic page. This gate *fails open* while the rep data hasn't loaded — the UI is a mirror, not the enforcer.
+2. **Workflow enforcement (authoritative, but reactive).** GitHub has no pre-post hook, so a low-rep user can momentarily post into a gated topic straight on github.com. Each sync run sweeps posts from the **last 48 hours** in gated topics and moderates violations per `onViolation`:
+   - `move` (default): move to `fallbackTopic` + explanatory bot comment — non-destructive
+   - `lock`: comment + lock the thread
+   - `delete`: remove the discussion entirely
+
+   A post's own rep gain is excluded when checking its own gate (otherwise a `+5` post could clear a 5-rep threshold by existing).
+
+**Exemptions**: users with push access (`exemptMaintainers`, default on) and everyone in `admins.logins`.
+
+## Caveats
+
+- Rep display lags the CDN cache (~5 min) and the sweep window is 48 h — a maintainer moving a violating post *back* into a gated topic within that window will see the workflow move it out again.
+- Farming resistance comes from cap/weight tuning, not structure. If a community finds a loophole, tune `gains`/`dailyCaps` rather than expecting the engine to catch intent.