Skip to content

Move user-uploaded files from browser IndexedDB to R2/D1#165

Merged
pokle merged 11 commits into
masterfrom
claude/this-plan-gRKHe
May 17, 2026
Merged

Move user-uploaded files from browser IndexedDB to R2/D1#165
pokle merged 11 commits into
masterfrom
claude/this-plan-gRKHe

Conversation

@pokle
Copy link
Copy Markdown
Owner

@pokle pokle commented May 11, 2026

Summary

Replaces the browser-only IndexedDB storage for user IGC tracks, XCTSK tasks, and map annotations with server-side storage scoped per user. Account deletion now wipes the user's R2 prefix as well as their D1 rows.

  • Tracks → R2 at u/{userId}/track/{sha256}.igc.gz, metadata in new user_track table (D1)
  • Tasks → JSON in new user_task.xctsk_json (no R2 — same shape as comp task.xctsk)
  • Annotations → new user_annotation table, scoped to (user, track); public-by-link readable so anyone viewing a track sees the owner's strokes
  • Account deletion → auth-api now lists+deletes everything under u/{userId}/ in R2, then deletes the user row; FK cascade handles the D1 rows

Files are public-by-link by default — anyone with a share link can view, but there is no /api/u/:username/tracks (or …/tasks) listing endpoint, so libraries aren't discoverable. The dashboard surfaces this with a permanent banner above the tabs.

Per-user quotas: 500 tracks, 200 tasks, 200 MB total. Quota errors surface in the UI as toasts (analysis page) / alert (dashboard).

One-time client-side migration (web/frontend/src/auth/user-files-migration.ts) uploads any leftover IndexedDB tracks/tasks the first time the user signs in after this rollout; orphaned legacy annotations are dropped with a console warning (they had no track association in the old schema).

Public share-link routing

Analysis page now accepts ?u={username}&track={sha256} and ?u={username}&task={code}. Storage layer flips into "public namespace" mode and resolves reads against /api/u/:username/…; writes are blocked while in public mode. Annotation layer is set readonly for non-owners.

Anonymous fallback

Anonymous users keep getting in-memory analysis. storage.isAvailable() short-circuits to false when signed out, so loadIGCFile/loadTask skip persistence; the annotation layer goes in-memory only. Drawing still works; nothing persists.

Test plan

  • bun run typecheck:all — all packages green
  • bun run --filter competition-api test — 248/248 pass (16 new user-files cases covering upload, get, list, delete, quotas, ownership, public reads, annotation cascade)
  • bun run --filter auth-api test — 3 new delete-account tests pass; the pre-existing two users do not see each other's preferences test occasionally times out at the harness default 5 s on slower runners (unrelated to this change, reproduces on baseline)
  • bun test --cwd . ./web/engine — 406/406 pass
  • Manual: drop an IGC on /analysis.html signed in → uploaded, dashboard lists it, URL becomes ?storedTrack=<id>
  • Manual: draw strokes, reload → strokes reappear from server
  • Manual: share ?u={username}&track={id} link → loads in another browser, no draw UI
  • Manual: delete account → R2 u/{userId}/ empty, D1 rows gone

https://claude.ai/code/session_01ASJpfj483tfd8XjkQ5MZxX


Generated by Claude Code

claude and others added 3 commits May 11, 2026 11:34
…ser to R2/D1

Tracks now live in R2 under u/{userId}/track/{sha256}.igc.gz with metadata in
the new user_track D1 table. Tasks live in user_task as JSON. Annotations live
in user_annotation scoped to (user, track) so they're visible to anyone viewing
the track via /analysis.html?u={username}&track={track_id}.

Files are public-by-link by default; the dashboard surfaces this with a
permanent banner. There is no listing endpoint under /api/u/:username/ — files
are not discoverable. Per-user quotas: 500 tracks, 200 tasks, 200 MB total.

Account deletion via auth-api now lists + deletes every R2 object under
u/{userId}/ before removing the user row; the FK cascade then wipes the D1
metadata. A one-time client-side migration uploads any leftover IndexedDB
items on the next signed-in page load.

https://claude.ai/code/session_01ASJpfj483tfd8XjkQ5MZxX
The new user-file download responses set Content-Disposition,
X-Filename, and X-Display-Name from user-supplied filenames and
IGC pilot names. WHATWG Headers values must be ASCII, so an IGC
with a non-ASCII pilot ("François") would 500 the download.
Add asciiHeaderSafe() and an RFC 5987 filename*=UTF-8'' builder;
also cap x-filename at 255 chars.

Vite's dev proxy only forwarded /api/auth and /api/comp; the new
/api/user and /api/u/ routes 404'd under `bun run dev` because
the Pages Functions only run in production. Add proxy entries
so manual testing works end-to-end.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@pokle pokle marked this pull request as ready for review May 17, 2026 00:40
pokle and others added 8 commits May 18, 2026 07:51
… e2e

On preview deploys, /api/user/* and /api/u/* (added in this PR) returned a
Cloudflare-level 404 with no CORS headers — the request never reached the
competition-api worker, so IGC uploads from the My Flights dashboard failed
with "HTTP 404". Locally `wrangler pages functions build` produces an
auto-generated _routes.json that includes both new prefixes, but the
deployed manifest had stranded them at 404. Check in an explicit
_routes.json so Pages stops relying on auto-generation for these.

Add an e2e spec covering IGC upload, XCTSK upload, delete (track + task),
and an anonymous public-link read of /api/u/:username/track/:sha — the
flows newly added/changed by the user-files migration.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI failed all upload tests except the first one — that test happened to
check #tracks-empty before setInputFiles, which incidentally synchronised
on dashboard.ts's first refreshLists() call. The other tests raced the
change-event listener attachment and the file selection was a no-op.

Hoist the wait into signInAndOnboard so every test sees a fully-mounted
dashboard before driving the upload inputs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI on 98c680c failed the pre-existing comp-creation test with a 60s
timeout waiting for the create-comp dialog to open after clicking the
button. comp.ts attaches the dialog-opening click handler only after
loadComps() resolves; the test was clicking the button before the
handler was bound, so the click was a no-op.

Wait for networkidle (initNav's /api/auth/me and loadComps' /api/comp
both completed) before sending the click.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The retry attempts on 59d2afd hit "Competition not found" — comp-detail.ts
called showNotFound() because /api/comp/:id raced one of the concurrent
/api/auth/me calls (initNav + preferences-sync bootstrap) and resolved
without a user, so the test-comp admin check returned 404.

Wait for networkidle after the redirect so all the page's startup
requests complete before we read #comp-title. If the GET is going to
succeed, it has by then; if it's going to fail, it's already failed,
so the assertion can read the populated title instead of the
intermediate empty state.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The next time GHA flakes a test we want a per-step waterfall (action +
network + console) to tell whether it's a real race in our code or just
slow infrastructure. retain-on-failure keeps the trace zip only when a
test fails, so passing runs stay cheap.

Traces land in test-results/ rather than playwright-report/, so the
upload-artifact step needed both paths. Open one locally with
`bunx playwright show-trace test-results/<dir>/trace.zip`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…500s

Hono's default error response is a plain "Internal Server Error" body,
which left a real bug invisible: the CI comp-creation flake on this
branch is a 500 from GET /api/comp/:id that the frontend catches and
renders as "Competition not found". With no error body, the playwright
trace just shows status=500 — no clue what threw.

Add an app.onError that logs the stack server-side and returns
{ error: <message> } with status 500. Stack stays server-side; the
message is enough for the trace to point at the real failure.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The comp-creation + user-files-upload specs share the local D1 instance,
and with Playwright's default 2 workers on GHA they race a 500 out of
GET /api/comp/:id roughly half the time. Forcing workers: 1 in CI keeps
the runs deterministic at the cost of a few seconds of wall time.
Local stays parallel (workers: undefined) so dev iteration is unaffected.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Same rationale as the competition-api change in dddc2c8: Hono's default
"Internal Server Error" body hides the real exception, so any 500 from
auth-api (e.g. Better Auth misconfiguration, D1 session race) shows up
in CI traces and wrangler tail as a stackless mystery. Logs the stack
server-side and returns { error: <message> } to the caller.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

Preview Deployment
https://b1f08f93.glidecomp.pages.dev
Commit: 5060d06

@pokle pokle merged commit 03760b4 into master May 17, 2026
7 checks passed
@pokle pokle deleted the claude/this-plan-gRKHe branch May 17, 2026 23:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants