diff --git a/README.md b/README.md index 6e1d81db8..d916fc20d 100644 --- a/README.md +++ b/README.md @@ -24,12 +24,13 @@ This is the recommended way to deploy it. Fork the repo, connect it to Bunny, an | `DB_URL` | Your Bunny database URL | | `DB_TOKEN` | Your Bunny database auth token | | `DB_ENCRYPTION_KEY` | 32-byte base64-encoded AES-256 key | + | `SCHEDULED_TASK_KEY` | Unique 32-byte base64url maintenance key | 5. **Add GitHub Actions secrets** to your repository: `BUNNY_SCRIPT_ID` and `BUNNY_ACCESS_KEY` Pushes to `main` trigger the deploy workflow automatically. The database schema auto-migrates on first request. Visit `/setup/` to set your admin password and currency. -For image uploads, also add `STORAGE_ZONE_NAME` and `STORAGE_ZONE_KEY` as Bunny secrets. See the [CONFIG_KEYS reference](https://chobbledotcom.github.io/tickets/doc.ts/~/CONFIG_KEYS.html) for all optional variables. +For image uploads, also add `STORAGE_ZONE_NAME` and `STORAGE_ZONE_KEY` as Bunny secrets. Configure an external monitor using the [scheduled maintenance guide](docs/scheduled-maintenance.md). See the [CONFIG_KEYS reference](https://chobbledotcom.github.io/tickets/doc.ts/~/CONFIG_KEYS.html) for all optional variables. --- @@ -355,7 +356,7 @@ Optional: | `ADMIN_EMAIL_ADDRESS` | Enables a superuser recovery account. The email local-part (before `@`) must be a valid username: 2–32 characters, letters, numbers, hyphens, and underscores only. Email delivery must be configured before the superuser can be enabled. | | `DEBUG_KEY` | Optional diagnostic key. `GET /health` returns a plain `Up :)` by default; a request carrying a matching `X-Debug-Key` header gets a small JSON payload (build commit, build timestamp, server time). Unset ⇒ the verbose response is disabled. | -**Database maintenance:** pruning of expired sessions, rate-limit rows, payment idempotency records and (optionally) orphaned attendees runs automatically while serving requests, self-gated to roughly once per `PRUNE_INTERVAL_HOURS` (default 24) per table — so a site with regular traffic needs no setup. To guarantee pruning on a quiet site, point a cron at `GET /scheduled` — a dynamic route that prunes on every hit (static asset URLs such as `/favicon.ico` are served before pruning runs, so they won't do). On a builder, `POST /scheduled` additionally pokes the least-recently-pruned built site (a plain request that triggers _its_ prune), so one cron on the master keeps quiet client sites pruned too — run it often enough to cover the fleet within `PRUNE_INTERVAL_HOURS` (e.g. hourly handles ~24 clients at the default). +**Database maintenance:** pruning of expired sessions, rate-limit rows, payment records, and optional orphan attendees runs automatically while serving requests. For quiet sites, configure an external monitor to send an authenticated `POST /scheduled` to each site at least every 15 minutes. Each site needs its own key. The builder does not contact child sites for the monitor. See the [scheduled maintenance guide](docs/scheduled-maintenance.md) for setup and CDN rules. **Backups:** every table is dumped to a single `.zip`, with table reads keyset-paginated so no single response trips libsqld's "Response is too large" payload cap (the server limit behind Bunny's databases). Backups run **out-of-band**, not inside the migration: a full dump of a ~31-table schema can't fit alongside a migration within one edge request's [50-subrequest budget](https://docs.bunny.net/scripting/limits), so migrations just migrate, and a backup is taken by GitHub Actions (or `deno task backup`) beforehand. To enforce that, **`/admin/update` and the per-site update button refuse to deploy unless a backup of that database was taken in the last hour.** diff --git a/TODO.md b/TODO.md index a38a05cfd..f5fd4db7d 100644 --- a/TODO.md +++ b/TODO.md @@ -448,7 +448,7 @@ look. ## Request performance: consolidate AsyncLocalStorage scopes -`src/features/index.ts` enters eleven nested request scopes for locale, client +`src/features/app/request.ts` enters eleven nested request scopes for locale, client IP, request ID, request cache, query logging, flash, session memoization, iframe mode, CSRF, saved form data, and settings auditing. Replace them with one typed `RequestContext` in one `AsyncLocalStorage`; retain domain methods where they add diff --git a/deno.json b/deno.json index 9da18d0b0..b220ccf2a 100644 --- a/deno.json +++ b/deno.json @@ -53,7 +53,7 @@ "#shared/": "./src/shared/", "#i18n": "./src/shared/i18n.ts", "#locales/": "./src/locales/", - "#routes": "./src/features/index.ts", + "#routes": "./src/features/app/request.ts", "#routes/": "./src/features/", "#templates/": "./src/ui/templates/", "#static/": "./src/ui/static/", diff --git a/docs/scheduled-maintenance.md b/docs/scheduled-maintenance.md new file mode 100644 index 000000000..c0dad1cb8 --- /dev/null +++ b/docs/scheduled-maintenance.md @@ -0,0 +1,49 @@ +# Scheduled maintenance + +Each site runs its own small database maintenance jobs. An external HTTPS +monitor must send one request to each site at least every 15 minutes: + +```http +POST /scheduled HTTP/1.1 +Authorization: Bearer +``` + +The request has no body. A successful run returns an empty `204`. A configured +site returns an empty `401` for a missing or wrong key. A site with no key, or a +non-`POST` request, returns an empty `404`. A system failure returns an empty +`503`. All responses use `Cache-Control: no-store`. + +## Set Up A Site + +Create a different 32-byte key for every independently deployed site: + +```bash +openssl rand -base64 32 | tr '+/' '-_' | tr -d '=' +``` + +Store it as the native `SCHEDULED_TASK_KEY` secret on that site. Never put the +key in a URL, monitor name, note, log, or plaintext database field. The owner +can read the local key on **Settings > Advanced**. + +The built-site manager creates a different key for every child. It stores the +key in the child's native secret and in the builder's encrypted site data. Use +the child's **Scheduled maintenance** tab to set up an older child. + +## Change A Child Key + +Coordinated one-click key rotation is intentionally deferred to the upcoming +Uptime Kuma integration, which will update the child and its monitor together. +Until then, a host operator can manually replace a compromised key on the child +and in the monitor. + +## CDN Rules + +Allow the monitor to reach `/scheduled` without a browser challenge, cached +response, redirect, or body rewrite. If the CDN has an allowlist, add the +monitor there. + +Rate-limit `/scheduled` at the CDN before requests reach the edge script. The +application deliberately does not keep a request counter for rejected calls: +doing so would let unauthenticated traffic consume database subrequests before +authentication. Keep the limit high enough for the monitor's retries, but far +below a general API traffic rate. diff --git a/scripts/build-site.ts b/scripts/build-site.ts index a2491c72f..4aef22e84 100644 --- a/scripts/build-site.ts +++ b/scripts/build-site.ts @@ -12,7 +12,7 @@ * BUNNY_API_KEY=... deno run --allow-all scripts/build-site.ts "My Event Site" */ -import { builderApi } from "#shared/builder.ts"; +import { buildRetainedSite } from "#shared/site-build.ts"; import { runBuildEdge } from "./run-build-edge.ts"; const [siteName] = Deno.args; @@ -41,7 +41,7 @@ console.log( `Bundle ready (${code.length} bytes). Provisioning site "${siteName}"…`, ); -const result = await builderApi.buildSite({ code, siteName }); +const { result } = await buildRetainedSite(siteName, { code, siteName }); if (!result.ok) { console.error(`Build failed: ${result.error}`); diff --git a/scripts/find-unused-src.ts b/scripts/find-unused-src.ts index 7df83b736..873ca8a53 100644 --- a/scripts/find-unused-src.ts +++ b/scripts/find-unused-src.ts @@ -205,7 +205,6 @@ const entryPoints = new Set([ "src/index.ts", // deno task start "src/edge.ts", // esbuild entry for Bunny CDN "src/fp.ts", // import map root alias - "src/features/index.ts", // import map root alias "src/doc.ts", // deno doc generation "src/shared/jsx/jsx-dev-runtime.ts", // jsxImportSource compiler config ]); diff --git a/scripts/mutation/equivalent-mutants.txt b/scripts/mutation/equivalent-mutants.txt index 9022ea2cd..0ce680129 100644 --- a/scripts/mutation/equivalent-mutants.txt +++ b/scripts/mutation/equivalent-mutants.txt @@ -53,10 +53,10 @@ src/shared/config.ts:177:33 ?? → || # getEnv(): string|undefined, only fal src/shared/qr-token.ts:60:18 ?? → || # input.date?: string, only falsy string "" === fallback "" src/shared/qr-token.ts:62:18 ?? → || # input.name?: string, only falsy string "" === fallback "" src/shared/app-forms.ts:71:18 ?? → || # config.auth is an AuthPolicy object when present, so it is always truthy -src/shared/site-assignment.ts:192:57 ?? → || # parseReadOnlyFromMs(): number|null, 0 ?? 0 === 0 || 0 -src/shared/site-assignment.ts:370:34 ?? → || # available[idx]: object|undefined, never falsy non-null -src/shared/site-assignment.ts:392:34 ?? → || # getEmailConfig(): EmailConfig|null -src/shared/site-assignment.ts:416:53 ?? → || # parseEmail(): ValidEmail|null, always truthy or null +src/shared/site-assignment.ts:161:57 ?? → || # parseReadOnlyFromMs(): number|null, 0 ?? 0 === 0 || 0 +src/shared/site-assignment.ts:332:34 ?? → || # available[idx]: object|undefined, never falsy non-null +src/shared/site-assignment.ts:354:34 ?? → || # getEmailConfig(): EmailConfig|null +src/shared/site-assignment.ts:378:53 ?? → || # parseEmail(): ValidEmail|null, always truthy or null src/shared/ledger/project.ts:20:41 ?? → || # allBalances: Map.get; the only falsy-non-null number is 0, and 0 ?? 0 === 0 || 0 src/shared/listing-parents-rules.ts:124:4 ?? → || # find()?.error(): an i18n message is never "", so only undefined reaches the fallback either way src/shared/ledger/project.ts:38:49 ?? → || # balanceOf: same Map.get fallback; 0 ?? 0 === 0 || 0 @@ -77,7 +77,21 @@ src/shared/accounting/rows.ts:262:58 ?? → || # selectById (await …)[0]: T src/shared/accounting/rows.ts:133:72 ?? → || # renderInsert guard?.args: InValue[]|undefined — an array is always truthy src/shared/accounting/manual-entries.ts:209:29 ?? → || # guard error message `transfer.kind ?? ""`: string|undefined, "" ?? "" === "" || "" src/shared/checkout-ledger.ts:35:72 ?? → || # find(...)?.amount: number|undefined; the only falsy-non-null amount is 0 and 0 ?? 0 === 0 || 0 -src/features/settings-bundles.ts:264:40 ?? → || # bundle is a readonly string[] or undefined; every present array, including [], is truthy +src/features/settings-bundles.ts:247:40 ?? → || # bundle is a readonly string[] or undefined; every present array, including [], is truthy +src/shared/scheduled-access.ts:16:20 ?? → || # the optional regex capture is non-empty because the capture uses +, so only undefined reaches the null fallback +src/shared/maintenance/runner.ts:113:40 ?? → || # wakePolicy is a non-empty MaintenanceWakePolicy value or undefined, so only undefined reaches the fallback +src/shared/builder.ts:201:44 ?? → || # dbToken is string|undefined and the fallback is "", so its only falsy string already equals the fallback +src/shared/builder.ts:202:35 ?? → || # dbProvider is a non-empty provider value or undefined, so only undefined reaches the fallback +src/shared/builder.ts:207:36 ?? → || # dbProvider is a non-empty provider value or undefined, so only undefined reaches the fallback +src/shared/builder.ts:287:26 ?? → || # hostingProvider is a non-empty provider value or undefined, so only undefined reaches the fallback +src/features/url.ts:58:35 ?? → || # URLSearchParams.get returns null or a string; its only falsy string is already the empty fallback +src/shared/storage.ts:376:18 application/octet-stream → "" # the pinned Bunny SDK normalizes an empty contentType to application/octet-stream, as the request-level upload test proves +src/shared/site-build.ts:39:21 = → += # retainedId starts at 0 and buildSite invokes retain exactly once, so both assignments store row.id +src/shared/db/built-site-scheduler.ts:11:36 ?? → || # scheduledTaskKey is null or a non-empty canonical key, so both operators select the candidate only when no key exists +src/shared/deno-deploy-api.ts:85:34 ?? → || # env_vars is a record object when present, and every object is truthy +src/shared/bunny-cdn.ts:477:42 ?? → || # DefaultHostname is string|undefined; its only falsy string is already the same empty fallback +src/shared/bunny-cdn.ts:529:50 ?? → || # Secrets is an array|null, and every present array is truthy while null takes the fallback either way +src/shared/subrequest-budget.ts:38:40 ?? → || # scoped counts is an object when present, and every object is truthy src/shared/db/attendees/balance.ts:223:22 ?? → || # attendee status ids are positive integers when present, so the value is always truthy or nullish src/shared/accounting/queries.ts:162:26 || → && # transferActivityBounds: MIN(occurred_at) and MAX(occurred_at) over one table are NULL together (both iff the table is empty), so either-null and both-null coincide src/shared/admin-features.ts:121:58 ?? → || # find(): AdminFeatureDefinition|undefined; a feature object is always truthy @@ -122,6 +136,12 @@ src/shared/db/table.ts:721:6 === → == # wrapNullable constrains T to exclud # Login attempt rows store an integer count. For undefined, null, zero, or any # non-zero integer, `(attempts ?? 0) + 1` and `(attempts || 0) + 1` agree. src/shared/db/login-attempts.ts:46:39 ?? → || +src/shared/db/users.ts:263:62 ?? → || # queryAll()[0] is a truthy UserAuthFields object or undefined, so both operators return the row or null +src/shared/db/query-log.ts:71:62 ?? → || # queryLogScope.current() is a truthy QueryLogState object or undefined +src/shared/db/query-log.ts:252:43 ?? → || # stored read counts start at 1 and remain positive, so only undefined reaches the zero fallback +src/shared/db/query-log.ts:292:22 ?? → || # store is a truthy QueryLogState object or undefined +src/shared/update.ts:131:45 ?? → || # the stored marker is a string or undefined and the fallback is empty, so both operators return the same string +src/shared/update.ts:156:45 ?? → || # the stored commit is a string or undefined and the fallback is empty, so both operators return the same string # Parent/child fold moved to fold-tree.ts in Phase 2a (see the fold-tree.ts # entries below); the ticket-payment.ts fold-internal equivalents went with it. @@ -273,11 +293,6 @@ src/features/admin/api.ts:549:19 ?? → || # groupIds is an array or undefine src/features/admin/api.ts:562:43 ?? → || # an existing listing id is positive and truthy, otherwise optional access yields undefined src/features/admin/api.ts:564:38 ?? → || # groupIds is an array or undefined; every array, including [], is truthy -# Settings registry: readOnly is optional but can only be true when present, so -# `"readOnly" in accessor && accessor.readOnly` and `... || accessor.readOnly` -# make the same if-condition decision for every valid accessor. -src/shared/settings/registry.ts:345:40 && → || # readOnly?: true; absent is falsy either way, present is true either way - # --- External order widget (src/ui/client/order.ts) ---------------------------- # URL matches and catalog entries are either non-empty strings/objects or absent, # so these nullish and truthy fallbacks have the same result. @@ -419,14 +434,11 @@ src/shared/images/resize.ts:75:19 = → += # out[o+1] = G into fresh-zeroed src/shared/images/resize.ts:76:19 = → += # out[o+2] = B into fresh-zeroed array; written once src/shared/images/resize.ts:78:17 = → += # out[o+3] = A into fresh-zeroed array; written once -# middleware.ts's missing Content-Type fallback is fed only to startsWith checks, -# so neither "" nor "mutated" can match an accepted MIME prefix. Once a tracking -# key is found, assigning true with `=` or `+=` is the same (`false + true` is 1, -# which is truthy), and continuing instead of breaking can only rediscover more -# tracking keys without changing the final boolean. -src/features/middleware.ts:163:63 → "mutated" # neither fallback matches an accepted Content-Type -src/features/middleware.ts:213:18 = → += # false + true is truthy, matching true -src/features/middleware.ts:214:7 break; → (removed) # later keys cannot reverse hasTracking +# middleware.ts's Content-Type fallback already equals the only falsy string. +# Once a tracking key is found, continuing instead of breaking can only +# rediscover more tracking keys without changing the final boolean. +src/features/middleware.ts:155:39 ?? → || # Content-Type is string|null; the only falsy non-null string is "", which is also the fallback +src/features/middleware.ts:217:7 break; → (removed) # later keys cannot reverse hasTracking # bundle-loader.ts assigns query/style/script values onto freshly-created URLs # and elements. Each left-hand value is empty (or false for defer), so `=` and @@ -969,8 +981,8 @@ src/features/admin/index.ts:28:21 ?? → || # `path.split("/")[2] ?? ""`: the src/features/admin/index.ts:45:43 ?? → || # areasBySegment values are arrays, so every present value is truthy and a miss is undefined # App prefix dispatch (app/routes.ts) — two provably-equivalent survivors. -src/features/app/routes.ts:239:14 → "mutated" # publicPagePath only receives the declared prefixes "", "listings", and "terms"; for "", both ternary arms return "/", and neither non-empty prefix equals "mutated", so no declared route path changes -src/features/app/routes.ts:444:59 ?? → || # route handlers return Response|null; every Response object is truthy, while null (and the mutation runner's return-undefined mutants) selects notFoundResponse under both operators +src/features/app/routes.ts:235:14 → "mutated" # publicPagePath only receives the declared prefixes "", "listings", and "terms"; for "", both ternary arms return "/", and neither non-empty prefix equals "mutated", so no declared route path changes +src/features/app/routes.ts:439:59 ?? → || # route handlers return Response|null; every Response object is truthy, while null (and the mutation runner's return-undefined mutants) selects notFoundResponse under both operators # Logger (logger.ts) — two provably-equivalent survivors from the run over the # logger suites. diff --git a/src/features/admin/builder.ts b/src/features/admin/builder.ts index 38df30cb8..050723e2f 100644 --- a/src/features/admin/builder.ts +++ b/src/features/admin/builder.ts @@ -22,10 +22,11 @@ import { isTursoEnabled, } from "#shared/config.ts"; import { logActivity } from "#shared/db/activityLog.ts"; +import { providerOrBunny } from "#shared/db/built-sites/types.ts"; import { builtSites, + builtSitesCrudTable, insertBuiltSite, - providerOrBunny, } from "#shared/db/built-sites.ts"; import { settings } from "#shared/db/settings.ts"; import { getEnv } from "#shared/env.ts"; @@ -158,14 +159,33 @@ const builderPost = createAuthedFormRoute({ const dbError = dbProviderConfigError(dbProviderVal, values.db_url); if (dbError) return errorRedirect(BUILDER_PATH, dbError); + let retainedSiteId: number | null = null; + const assignable = form.getFlag("assignable"); const result = await settings.withCurrentTask("builder", () => - builderApi.buildSite({ - ...(dbProviderVal === "manual" ? {} : { dbProvider }), - dbToken: values.db_token, - dbUrl: values.db_url, - hostingProvider, - siteName: values.site_name, - }), + builderApi.buildSite( + { + ...(dbProviderVal === "manual" ? {} : { dbProvider }), + dbToken: values.db_token, + dbUrl: values.db_url, + hostingProvider, + siteName: values.site_name, + }, + async (prepared) => { + const row = await insertBuiltSite( + values.site_name, + prepared.defaultHostname, + prepared.dbUrl, + prepared.dbToken, + false, + prepared.hostingId, + undefined, + prepared.hostingProvider, + prepared.dbProvider, + prepared.scheduledTaskKey, + ); + retainedSiteId = row.id; + }, + ), ); if (!result.ok) return errorRedirect(BUILDER_PATH, result.error); @@ -173,17 +193,12 @@ const builderPost = createAuthedFormRoute({ const buildResult = result.value; if (!buildResult.ok) return errorRedirect(BUILDER_PATH, buildResult.error); - await insertBuiltSite( - values.site_name, - buildResult.defaultHostname, - buildResult.dbUrl, - buildResult.dbToken, - form.getFlag("assignable"), - buildResult.hostingId, - undefined, - buildResult.hostingProvider, - buildResult.dbProvider, - ); + if (retainedSiteId === null) { + throw new Error("Built site was published before it was retained"); + } + if (assignable) { + await builtSitesCrudTable.update(retainedSiteId, { assignable: true }); + } await logActivity(`Built new site: ${values.site_name}`); return redirect( diff --git a/src/features/admin/built-site-action.ts b/src/features/admin/built-site-action.ts new file mode 100644 index 000000000..53e048b16 --- /dev/null +++ b/src/features/admin/built-site-action.ts @@ -0,0 +1,47 @@ +import { requireOwnerOr } from "#routes/auth.ts"; +import { requireCsrfForm } from "#routes/csrf.ts"; +import { createIdEntityHandler, type IdRouteHandler } from "#routes/entity.ts"; +import { errorRedirect, htmlResponse, redirect } from "#routes/response.ts"; +import type { BuiltSite } from "#shared/db/built-sites/types.ts"; +import { builtSitesCrudTable } from "#shared/db/built-sites.ts"; +import { builtSitePage } from "./built-site-page.tsx"; + +export type BuiltSiteTab = "maintenance" | "renewal" | "secrets" | "update"; + +export const builtSiteTabSuccess = ( + id: number, + tab: BuiltSiteTab, + message: string, +): Response => redirect(builtSitePage.path(id, tab), message, true); + +export const builtSiteTabError = ( + id: number, + tab: BuiltSiteTab, + message: string, +): Response => errorRedirect(builtSitePage.path(id, tab), message); + +export const builtSiteTabResult = + (tab: BuiltSiteTab, failure: (error: string) => string = (error) => error) => + (success: string) => + (id: number, result: { ok: true } | { ok: false; error: string }): Response => + result.ok + ? builtSiteTabSuccess(id, tab, success) + : builtSiteTabError(id, tab, failure(result.error)); + +export type BuiltSitePost = ( + site: BuiltSite, + form: { getString: (key: string) => string }, + id: number, +) => Promise; + +const builtSiteHandler = createIdEntityHandler( + builtSitesCrudTable.findById, +); + +export const builtSiteAction = (action: BuiltSitePost): IdRouteHandler => + builtSiteHandler(requireOwnerOr)(async (site, _session, request, { id }) => { + const csrf = await requireCsrfForm(request, () => + htmlResponse("CSRF token invalid", 403), + ); + return csrf.ok ? action(site, csrf.form, id) : csrf.response; + }); diff --git a/src/features/admin/built-site-page.tsx b/src/features/admin/built-site-page.tsx index 7a27e7973..f668d793e 100644 --- a/src/features/admin/built-site-page.tsx +++ b/src/features/admin/built-site-page.tsx @@ -9,11 +9,13 @@ import { submittedValueProps, } from "#routes/admin/entity-write-tab.ts"; import { requireOwnerOr } from "#routes/auth.ts"; -import { type BuiltSite, builtSitesCrudTable } from "#shared/db/built-sites.ts"; +import type { BuiltSite } from "#shared/db/built-sites/types.ts"; +import { builtSitesCrudTable } from "#shared/db/built-sites.ts"; import { loadSiteSecretsStatus } from "#shared/site-secrets.ts"; import { loadBuiltSiteUpdateState } from "#shared/site-update.ts"; import { BuiltSitesGuideFooter } from "#templates/admin/built-sites/list-parts.tsx"; import { + MaintenancePanel, renewalPanelFor, SecretsPanel, UpdatePanel, @@ -46,6 +48,12 @@ const updateTab = panelTab( ), ); +const maintenanceTab = panelTab( + "maintenance", + "built_sites.maintenance_title", + (site) => Promise.resolve(), +); + export const builtSitePage: EditEntityPage = defineEditEntityPage({ basePath, deleteLabelKey: "built_sites.delete_this_site", @@ -53,7 +61,7 @@ export const builtSitePage: EditEntityPage = defineEditEntityPage({ Promise.resolve( , ), - extraTabs: [renewalTab, secretsTab, updateTab], + extraTabs: [renewalTab, maintenanceTab, secretsTab, updateTab], guard: requireOwnerOr, guideFooter: () => Promise.resolve(), load: (id) => builtSitesCrudTable.findById(id), diff --git a/src/features/admin/built-sites.ts b/src/features/admin/built-sites.ts index 1a915e2e2..768f732bc 100644 --- a/src/features/admin/built-sites.ts +++ b/src/features/admin/built-sites.ts @@ -1,4 +1,5 @@ import { mapValues } from "@std/collections"; +import { t } from "#i18n"; import { handlersFor } from "#routes/admin/handlers.ts"; /** * Admin built site management routes - owner only @@ -8,26 +9,20 @@ import { handlersFor } from "#routes/admin/handlers.ts"; import { isBuilderEnabled } from "#routes/admin/builder.ts"; import { createOwnerCrudHandlers } from "#routes/admin/owner-crud.ts"; import { requireOwnerOr } from "#routes/auth.ts"; -import { applyFlash, requireCsrfForm } from "#routes/csrf.ts"; -import { createIdEntityHandler, type IdRouteHandler } from "#routes/entity.ts"; -import { - errorRedirect, - htmlResponse, - notFoundResponse, - redirect, -} from "#routes/response.ts"; +import { applyFlash } from "#routes/csrf.ts"; +import { htmlResponse, notFoundResponse } from "#routes/response.ts"; import type { RouteHandlerFn } from "#routes/router.ts"; /* jscpd:ignore-end */ import { siteHostingAccess } from "#shared/builder.ts"; import { logActivity } from "#shared/db/activityLog.ts"; import { dbName, hasRecentBackup } from "#shared/db/backup-storage.ts"; -import type { BuiltSite, BuiltSiteFormInput } from "#shared/db/built-sites.ts"; import { - builtSites, - builtSitesCrudTable, + type BuiltSite, + type BuiltSiteFormInput, isUpdateTier, providerOrBunny, -} from "#shared/db/built-sites.ts"; +} from "#shared/db/built-sites/types.ts"; +import { builtSites, builtSitesCrudTable } from "#shared/db/built-sites.ts"; import { getFlash } from "#shared/flash-context.ts"; import type { FormValues } from "#shared/forms/definition.ts"; import { isProvisioned } from "#shared/renewal-helpers.ts"; @@ -41,6 +36,7 @@ import { rotateRenewalToken, syncReadOnlyFrom, } from "#shared/site-assignment.ts"; +import { provisionSiteScheduler } from "#shared/site-scheduler.ts"; import { addMissingSiteSecrets } from "#shared/site-secrets.ts"; import { deployAndReport } from "#shared/site-update.ts"; import { @@ -54,6 +50,12 @@ import { adminBuiltSitesPage, } from "#templates/admin/built-sites.tsx"; import { getBuiltSiteForm } from "#templates/fields/admin.ts"; +import { + builtSiteAction, + builtSiteTabError, + builtSiteTabResult, + builtSiteTabSuccess, +} from "./built-site-action.ts"; import { builtSitePage } from "./built-site-page.tsx"; /** Extract built site input from validated form values. @@ -106,24 +108,17 @@ const crud = createOwnerCrudHandlers({ singular: "Built site", }); -type BuiltSiteTab = "renewal" | "secrets" | "update"; -const tabSuccess = (id: number, tab: BuiltSiteTab, message: string): Response => - redirect(builtSitePage.path(id, tab), message, true); -const tabError = (id: number, tab: BuiltSiteTab, message: string): Response => - errorRedirect(builtSitePage.path(id, tab), message); +const renewalPushResult = builtSiteTabResult( + "renewal", + (error) => `Deadline could not be pushed to the site: ${error}`, +); -const editPushResult = ( - id: number, - result: { ok: true } | { ok: false; error: string }, - success: string, -): Response => - result.ok - ? tabSuccess(id, "renewal", success) - : tabError( - id, - "renewal", - `Deadline could not be pushed to the site: ${result.error}`, - ); +const handleProvisionSiteScheduler = builtSiteAction(async (_site, _form, id) => + builtSiteTabResult("maintenance")(t("built_sites.maintenance_provisioned"))( + id, + await provisionSiteScheduler(id), + ), +); const editPushOk = ( id: number, @@ -132,8 +127,8 @@ const editPushOk = ( failure: string, ): Response => pushOk - ? tabSuccess(id, "renewal", success) - : tabError(id, "renewal", failure); + ? builtSiteTabSuccess(id, "renewal", success) + : builtSiteTabError(id, "renewal", failure); /** Max months any single bump/provision can request — guards against form tampering. */ const MAX_RENEWAL_MONTHS = 120; @@ -149,27 +144,7 @@ const readClampedMonths = (form: { const parseDeadlineDate = (dateStr: string): string | null => isIsoDate(dateStr) ? `${dateStr}T23:59:59Z` : null; -const builtSiteHandler = createIdEntityHandler( - builtSitesCrudTable.findById, -); -type BuiltSitePost = ( - site: BuiltSite, - form: { getString: (key: string) => string }, - id: number, -) => Promise; -const builtSiteHandlers = { - post: (action: BuiltSitePost): IdRouteHandler => - builtSiteHandler(requireOwnerOr)( - async (site, _session, request, { id }) => { - const csrf = await requireCsrfForm(request, () => - htmlResponse("CSRF token invalid", 403), - ); - return csrf.ok ? action(site, csrf.form, id) : csrf.response; - }, - ), -}; - -type EditResult = Awaited>; +type EditResult = Awaited>; const runSiteUpdate = async ( site: BuiltSite, @@ -177,7 +152,7 @@ const runSiteUpdate = async ( deploy: () => Promise<{ tagName: string; name: string }>, ): Promise => { if (!(await hasRecentBackup(undefined, dbName(site.dbUrl)))) { - return tabError( + return builtSiteTabError( id, "update", "No backup of this site in the last hour — back it up before updating.", @@ -186,8 +161,8 @@ const runSiteUpdate = async ( return deployAndReport({ deploy, logPrefix: `Updated built site '${site.name}'`, - onError: (message) => tabError(id, "update", message), - onSuccess: (message) => tabSuccess(id, "update", message), + onError: (message) => builtSiteTabError(id, "update", message), + onSuccess: (message) => builtSiteTabSuccess(id, "update", message), successPrefix: `Updated '${site.name}'`, }); }; @@ -197,9 +172,9 @@ const runSiteUpdate = async ( * The site migrates on its next request after deploy, so a recent backup of * *this site's* database (taken to our storage by the upgrade workflow) is * required before pushing a new version. */ -const handleUpdateSite = builtSiteHandlers.post(async (site, _form, id) => { +const handleUpdateSite = builtSiteAction(async (site, _form, id) => { const access = siteHostingAccess(site, "it can't be updated"); - if (!access.ok) return tabError(id, "update", access.error); + if (!access.ok) return builtSiteTabError(id, "update", access.error); return runSiteUpdate(site, id, () => site.hostingProvider === "deno" ? deployLatestReleaseToDeno(site.hostingId) @@ -208,9 +183,13 @@ const handleUpdateSite = builtSiteHandlers.post(async (site, _form, id) => { }); /** POST /admin/built-sites/:id/rotate-renewal-token */ -const handleRotateToken = builtSiteHandlers.post(async (site, _form, id) => { +const handleRotateToken = builtSiteAction(async (site, _form, id) => { if (!isProvisioned(site)) { - return tabError(id, "renewal", "Renewal is not provisioned for this site"); + return builtSiteTabError( + id, + "renewal", + "Renewal is not provisioned for this site", + ); } const result = await rotateRenewalToken( site, @@ -233,23 +212,31 @@ const handleRotateToken = builtSiteHandlers.post(async (site, _form, id) => { * Re-verifies the live secrets first, then sets only the ones still missing — * an existing secret is never overwritten (it may have been changed for a * reason). */ -const handleAddSecrets = builtSiteHandlers.post(async (site, _form, id) => { +const handleAddSecrets = builtSiteAction(async (site, _form, id) => { const result = await addMissingSiteSecrets(site); if (!result.ok) { - return tabError(id, "secrets", `Secrets could not be set: ${result.error}`); + return builtSiteTabError( + id, + "secrets", + `Secrets could not be set: ${result.error}`, + ); } if (result.added.length === 0) { - return tabSuccess(id, "secrets", "No missing secrets — nothing to set"); + return builtSiteTabSuccess( + id, + "secrets", + "No missing secrets — nothing to set", + ); } const summary = `${result.added.length} missing secret(s): ${result.added.join( ", ", )}`; await logActivity(`Set ${summary} on '${site.name}'`); - return tabSuccess(id, "secrets", `Set ${summary}`); + return builtSiteTabSuccess(id, "secrets", `Set ${summary}`); }); /** POST /admin/built-sites/:id/bump-deadline */ -const handleBumpDeadline = builtSiteHandlers.post(async (site, form, id) => { +const handleBumpDeadline = builtSiteAction(async (site, form, id) => { const months = readClampedMonths(form); const newIso = addMonthsToRenewalDeadline(site, months); const result = await syncReadOnlyFrom(site, newIso); @@ -258,32 +245,30 @@ const handleBumpDeadline = builtSiteHandlers.post(async (site, form, id) => { `Admin bumped '${site.name}' deadline by ${months} month(s)`, ); } - return editPushResult(id, result, "Deadline bumped"); + return renewalPushResult("Deadline bumped")(id, result); }); /** POST /admin/built-sites/:id/override-deadline */ -const handleOverrideDeadline = builtSiteHandlers.post( - async (site, form, id) => { - const dateStr = form.getString("date"); - if (!dateStr) return tabError(id, "renewal", "Choose a deadline date"); - const cutoffIso = parseDeadlineDate(dateStr); - if (!cutoffIso) { - return tabError(id, "renewal", "Choose a valid deadline date"); - } - const result = await syncReadOnlyFrom(site, cutoffIso); - if (result.ok) { - await logActivity( - `Admin overrode '${site.name}' deadline to ${cutoffIso}`, - ); - } - return editPushResult(id, result, "Deadline updated"); - }, -); +const handleOverrideDeadline = builtSiteAction(async (site, form, id) => { + const dateStr = form.getString("date"); + if (!dateStr) { + return builtSiteTabError(id, "renewal", "Choose a deadline date"); + } + const cutoffIso = parseDeadlineDate(dateStr); + if (!cutoffIso) { + return builtSiteTabError(id, "renewal", "Choose a valid deadline date"); + } + const result = await syncReadOnlyFrom(site, cutoffIso); + if (result.ok) { + await logActivity(`Admin overrode '${site.name}' deadline to ${cutoffIso}`); + } + return renewalPushResult("Deadline updated")(id, result); +}); /** POST /admin/built-sites/:id/re-sync-deadline */ -const handleReSyncDeadline = builtSiteHandlers.post(async (site, _form, id) => { +const handleReSyncDeadline = builtSiteAction(async (site, _form, id) => { if (!site.readOnlyFrom) { - return tabError(id, "renewal", "No deadline to re-sync"); + return builtSiteTabError(id, "renewal", "No deadline to re-sync"); } const renewalUrl = isProvisioned(site) && site.renewalToken @@ -293,7 +278,7 @@ const handleReSyncDeadline = builtSiteHandlers.post(async (site, _form, id) => { if (result.ok) { await logActivity(`Admin re-synced deadline for '${site.name}'`); } - return editPushResult(id, result, "Deadline re-synced"); + return renewalPushResult("Deadline re-synced")(id, result); }); /** POST /admin/built-sites/:id/provision-renewal @@ -301,42 +286,40 @@ const handleReSyncDeadline = builtSiteHandlers.post(async (site, _form, id) => { * Gates on the existence of at least one qualifying renewal tier listing so an * admin doesn't generate a token that would dead-end at an empty /renew picker. * (The customer picks the actual tier at renew time.) */ -const handleProvisionRenewal = builtSiteHandlers.post( - async (site, form, id) => { - if (isProvisioned(site)) { - return tabError( - id, - "renewal", - "Renewal is already provisioned for this site", - ); - } - const tier = await pickTierListing(); - if (!tier) { - return tabError( - id, - "renewal", - "Create a qualifying renewal tier listing before provisioning", - ); - } - const months = readClampedMonths(form); - const result = await provisionSiteRenewal( - site, - months, - `Provision push failed for site ${id}`, +const handleProvisionRenewal = builtSiteAction(async (site, form, id) => { + if (isProvisioned(site)) { + return builtSiteTabError( + id, + "renewal", + "Renewal is already provisioned for this site", ); - if (result.pushOk) { - await logActivity( - `Admin provisioned renewals for '${site.name}' (${months}mo)`, - ); - } - return editPushOk( + } + const tier = await pickTierListing(); + if (!tier) { + return builtSiteTabError( id, - result.pushOk, - "Renewal provisioned", - "Renewal could not be pushed to the site", + "renewal", + "Create a qualifying renewal tier listing before provisioning", ); - }, -); + } + const months = readClampedMonths(form); + const result = await provisionSiteRenewal( + site, + months, + `Provision push failed for site ${id}`, + ); + if (result.pushOk) { + await logActivity( + `Admin provisioned renewals for '${site.name}' (${months}mo)`, + ); + } + return editPushOk( + id, + result.pushOk, + "Renewal provisioned", + "Renewal could not be pushed to the site", + ); +}); /** GET /admin/built-sites — overrides the CRUD list so we can render the * renewal-tier summary alongside the sites table. */ @@ -383,6 +366,7 @@ export const adminHandlers = gateOnBuilder( postBuiltSitesByIdEdit: crud.editPost, postBuiltSitesByIdOverrideDeadline: handleOverrideDeadline, postBuiltSitesByIdProvisionRenewal: handleProvisionRenewal, + postBuiltSitesByIdProvisionScheduler: handleProvisionSiteScheduler, postBuiltSitesByIdReSyncDeadline: handleReSyncDeadline, postBuiltSitesByIdRotateRenewalToken: handleRotateToken, postBuiltSitesByIdUpdate: handleUpdateSite, diff --git a/src/features/admin/debug.ts b/src/features/admin/debug.ts index 0406ec6aa..28f18f306 100644 --- a/src/features/admin/debug.ts +++ b/src/features/admin/debug.ts @@ -272,13 +272,6 @@ const getDebugPageState = async (): Promise => { provider: showOrEmpty(paymentProvider), webhookConfigured: webhookConfiguredFor(paymentProvider), }, - prune: { - addresses: formatLastPruned(settings.lastPrunedAddresses), - logins: formatLastPruned(settings.lastPrunedLogins), - payments: formatLastPruned(settings.lastPrunedPayments), - sessions: formatLastPruned(settings.lastPrunedSessions), - strings: formatLastPruned(settings.lastPrunedStrings), - }, runtime: getRuntimeInfo(), site: { bookingFee: settings.bookingFee, @@ -294,13 +287,6 @@ const getDebugPageState = async (): Promise => { }; }; -/** Format a stored last-pruned ms-epoch string as ISO. - * `raw` is always a positive ms-epoch string by the time we render: every - * incoming request triggers maybeRunPrunes() as pending work, which writes a - * fresh timestamp before the /admin/debug handler reads the snapshot. */ -const formatLastPruned = (raw: string): string => - new Date(Number(raw)).toISOString(); - /** * Handle GET /admin/debug - owner only */ diff --git a/src/features/admin/settings-page.ts b/src/features/admin/settings-page.ts index fc1de00dd..6be35a34e 100644 --- a/src/features/admin/settings-page.ts +++ b/src/features/admin/settings-page.ts @@ -17,8 +17,10 @@ import { import { getAdminFeatureUsage } from "#shared/db/admin-features.ts"; import { settings } from "#shared/db/settings.ts"; import { EMAIL_PROVIDER_LABELS, getHostEmailConfig } from "#shared/email.ts"; +import { getEnv } from "#shared/env.ts"; import { getFlash } from "#shared/flash-context.ts"; import { getPaymentWebhookUrl } from "#shared/payment-webhook-url.ts"; +import { SCHEDULED_TASK_KEY_ENV } from "#shared/scheduled-keys.ts"; import { isStorageEnabled } from "#shared/storage.ts"; import { getSuperuserState } from "#shared/superuser.ts"; import { adminSettingsPage } from "#templates/admin/settings.tsx"; @@ -43,7 +45,7 @@ const getSettingsPageState = async () => { embedHosts: settings.embedHosts, enabledFeatures: enabledFeaturesWithUsage(settings.features, featureUsage), headerImageUrl: settings.headerImageUrl, - paymentProvider: settings.paymentProvider ?? "", + paymentProvider: settings.paymentProvider, squareSandbox: settings.square.sandbox, squareTokenConfigured: settings.square.hasToken, squareWebhookConfigured: settings.square.webhookSignatureKey !== "", @@ -68,8 +70,8 @@ const renderSettingsPage = async (session: AuthSession) => { /** Gather state for the advanced settings page */ const getAdvancedSettingsPageState = async ( - subdomainPreview = "", - subdomainPreviewFullDomain = "", + subdomainPreview: string, + subdomainPreviewFullDomain: string, ) => { const bunnyCdnConfigured = isBunnyCdnEnabled(); const bunnyDnsEnabled = isBunnyDnsEnabled(); @@ -86,17 +88,14 @@ const getAdvancedSettingsPageState = async ( attendeeColumnOrder: settings.attendeeColumnOrder, bunnyCdnEnabled: bunnyCdnConfigured, bunnyDnsEnabled, - bunnyDnsSubdomainSuffix: bunnyDnsEnabled - ? getBunnyDnsSubdomainSuffix() - : "", + bunnyDnsSubdomainSuffix: getBunnyDnsSubdomainSuffix(), bunnySubdomain: settings.bunnySubdomain, businessEmail: settings.businessEmail, cdnHostname: cdnResult?.ok ? cdnResult.hostname : "", confirmationTemplates, customCss: settings.customCss, - customDomain: (bunnyCdnConfigured ? settings.customDomain : null) ?? "", - customDomainLastValidated: - (bunnyCdnConfigured ? settings.customDomainLastValidated : null) ?? "", + customDomain: settings.customDomain, + customDomainLastValidated: settings.customDomainLastValidated, emailApiKeyConfigured: settings.email.hasApiKey, emailFromAddress: settings.email.fromAddress, emailProvider: settings.email.provider, @@ -121,7 +120,8 @@ const getAdvancedSettingsPageState = async ( return `Host env (${hostConfig.issuerId})`; })(), listingColumnOrder: settings.listingColumnOrder, - paymentProvider: settings.paymentProvider ?? "", + paymentProvider: settings.paymentProvider, + scheduledTaskKey: getEnv(SCHEDULED_TASK_KEY_ENV), showPublicApi: settings.showPublicApi, smsGatewayBaseUrl: settings.smsGatewayBaseUrl, smsGatewayPassphraseConfigured: settings.smsGateway.hasPassphrase, @@ -137,8 +137,8 @@ const getAdvancedSettingsPageState = async ( /** Render the advanced settings page with current state */ const renderAdvancedSettingsPage = async ( session: AuthSession, - subdomainPreview = "", - subdomainPreviewFullDomain = "", + subdomainPreview: string, + subdomainPreviewFullDomain: string, ) => { const state = await getAdvancedSettingsPageState( subdomainPreview, @@ -160,7 +160,7 @@ export const handleAdminSettingsAdvancedGet: TypedRouteHandler<"GET /admin/setti ownerPage(async (session) => { const flash = getFlash(); const [subdomainPreview = "", subdomainPreviewFullDomain = ""] = - flash.result?.split("\n") ?? []; + flash.result ? flash.result.split("\n") : []; return await renderAdvancedSettingsPage( session, subdomainPreview, diff --git a/src/features/app/read-only.ts b/src/features/app/read-only.ts index c242ae10d..b470be74e 100644 --- a/src/features/app/read-only.ts +++ b/src/features/app/read-only.ts @@ -24,7 +24,6 @@ const READ_ONLY_SAFE_PATHS = [ /^\/unsubscribe$/, /^\/contact$/, /^\/instance\/site-credentials$/, - /^\/scheduled$/, /^\/checkin\/[^/]+$/, ]; @@ -35,7 +34,7 @@ const isMutatingMethod = (method: string): boolean => method === "PUT"; const isAdminMutation = (path: string, method: string): boolean => - isMutatingMethod(method) && (path === "/admin" || path.startsWith("/admin/")); + isMutatingMethod(method) && path.startsWith("/admin/"); const isAllowedAdminOperation = (path: string, method: string): boolean => READ_ONLY_ADMIN_OPERATIONS.some( diff --git a/src/features/app/request.ts b/src/features/app/request.ts index a4175df37..f469ebdc4 100644 --- a/src/features/app/request.ts +++ b/src/features/app/request.ts @@ -30,14 +30,12 @@ import { clearSessionCookie, parseFlashValue, } from "#shared/cookies.ts"; -import { maybeBackfillActivityLog } from "#shared/db/activity-log-backfill.ts"; import { DatabaseBusyError } from "#shared/db/client.ts"; import { initDb, MigrationInProgressError, MissingSettingsTableError, } from "#shared/db/migrations.ts"; -import { maybeRunPrunes } from "#shared/db/prune.ts"; import { enableQueryLog } from "#shared/db/query-log.ts"; import { settings } from "#shared/db/settings.ts"; import { assertSettingsReadsDeclared } from "#shared/db/settings-audit.ts"; @@ -56,17 +54,18 @@ import { logError, logRequest, } from "#shared/logger.ts"; -import { addPendingWork, flushPendingWork } from "#shared/pending-work.ts"; +import { reportMaintenanceFailure } from "#shared/maintenance/report.ts"; import { getRethrowErrors } from "#shared/test-overrides.ts"; +import { requestScopedHandler } from "../request-scopes.ts"; import { defineAppRoute, routeMainApp } from "./routes.ts"; import { bufferRequestIfNeeded, ensureCustomCssResponse, isSetupPath, + runOrganicMaintenanceWhenDue, shouldLogQueries, shouldPrefetchSettings, shouldRetryBusyRequest, - shouldRunPrunes, trackingRedirectLocation, } from "./rules.ts"; @@ -157,13 +156,26 @@ const prepareRequestEnvironment = async ( await settings.loadKeys(settingsForPath(path)); - if (shouldRunPrunes(method, path)) { - addPendingWork(maybeRunPrunes()); - } - addPendingWork(maybeBackfillActivityLog()); loadEffectiveDomain(url); }; +const runOrganicMaintenanceAfterResponse = ( + method: string, + path: string, + response: Response, +): Promise => + runOrganicMaintenanceWhenDue(method, path, response.status, async () => { + try { + const [{ MAINTENANCE_TASKS }, { maintenance }] = await Promise.all([ + import("#shared/maintenance/registry.ts"), + import("#shared/maintenance/runner.ts"), + ]); + await maintenance.runOrganic(MAINTENANCE_TASKS); + } catch (error) { + reportMaintenanceFailure("organic maintenance failed", error); + } + }); + const routeAndFinalize = async ( request: Request, url: URL, @@ -213,8 +225,8 @@ const handleRoutingError = ( return temporaryErrorResponse(); }; -/** Run the request pipeline inside the request-scoped contexts from index.ts. */ -export const processRequest = async ( +/** Run the application request pipeline. */ +const processRequest = async ( request: Request, server: ServerContext | undefined, ): Promise => { @@ -271,8 +283,10 @@ export const processRequest = async ( assertSettingsReadsDeclared(`${method} ${path}`); } catch (error) { response = finish(handleRoutingError(error, method, path)); - } finally { - await flushPendingWork(); } + await runOrganicMaintenanceAfterResponse(method, path, response); return response; }; + +/** Handle one request inside every request-scoped store. */ +export const handleRequest = requestScopedHandler(processRequest); diff --git a/src/features/app/routes.ts b/src/features/app/routes.ts index a858e5922..81875e2d9 100644 --- a/src/features/app/routes.ts +++ b/src/features/app/routes.ts @@ -172,10 +172,6 @@ const loadAdminApiRoutes = once(async () => createRouter((await import("#routes/admin/api.ts")).adminApiRoutes), ); -const loadScheduledRoutes = once(async () => - createRouter((await import("#routes/scheduled.ts")).scheduledRoutes), -); - const loadInstanceRoutes = once(async () => createRouter((await import("#routes/instance.ts")).instanceRoutes), ); @@ -387,7 +383,6 @@ const prefixHandlers: Record = { ), lazyRoute(exactRouteLoaders.renewal), ), - scheduled: prefixRoute([], lazyRoute(loadScheduledRoutes)), sms: prefixRoute([], lazyRoute(routeLoaders.smsWebhook)), t: prefixRoute( publicMessageGroups("listing-qr", "payment", "tickets"), diff --git a/src/features/app/rules.ts b/src/features/app/rules.ts index 4943bd7e0..855bf84aa 100644 --- a/src/features/app/rules.ts +++ b/src/features/app/rules.ts @@ -1,3 +1,4 @@ +import { lazyRef } from "#fp"; import { getCleanUrl, lowerContentType } from "#routes/middleware.ts"; import { emptyCustomCssResponse, @@ -35,9 +36,62 @@ export const bufferRequestIfNeeded: RequestTransform = (request) => export const shouldLogQueries = (method: string, prefix: string): boolean => method === "GET" && prefix === "admin"; -/** Whether routine background pruning is safe during this request. */ -export const shouldRunPrunes = (method: string, path: string): boolean => - method !== "POST" || path !== "/admin/privacy/orphans"; +const ORGANIC_MAINTENANCE_THROTTLE_MS = 60_000; +const ORGANIC_UNSAFE_PREFIXES = [ + "/address-lookup", + "/api", + "/calculate", + "/gwallet", + "/join", + "/order", + "/pay", + "/payment", + "/renew", + "/scheduled", + "/setup", + "/sms", + "/ticket", + "/v1", + "/wallet", +] as const; + +const pathIs = (path: string, prefix: string): boolean => + path === prefix || path.startsWith(`${prefix}/`); + +const shouldRunOrganicMaintenance = ( + method: string, + path: string, + status: number, +): boolean => + (method === "GET" || method === "HEAD") && + status >= 200 && + status < 300 && + !ORGANIC_UNSAFE_PREFIXES.some((prefix) => pathIs(path, prefix)); + +const [getNextOrganicWake, setNextOrganicWake] = lazyRef(() => 0); + +const claimOrganicMaintenanceWake = (time = Date.now()): boolean => { + if (time < getNextOrganicWake()) return false; + setNextOrganicWake(time + ORGANIC_MAINTENANCE_THROTTLE_MS); + return true; +}; + +/** Run maintenance after a safe response when this isolate's wake is due. */ +export const runOrganicMaintenanceWhenDue = async ( + method: string, + path: string, + status: number, + run: () => void | Promise, + time = Date.now(), +): Promise => { + if ( + !shouldRunOrganicMaintenance(method, path, status) || + !claimOrganicMaintenanceWake(time) + ) { + return; + } + await run(); +}; /** Whether a busy-database page may retry the request automatically. */ export const shouldRetryBusyRequest = (method: string): boolean => diff --git a/src/features/instance.ts b/src/features/instance.ts index 552fa32c5..a7e926e6a 100644 --- a/src/features/instance.ts +++ b/src/features/instance.ts @@ -35,11 +35,11 @@ import { defineRoutes } from "#routes/router.ts"; import { getMainInstanceKey, isInstanceApiEnabled } from "#shared/config.ts"; import { constantTimeEqual } from "#shared/crypto/utils.ts"; import { - builtSites, DEFAULT_UPDATE_TIER, isUpdateTier, siteAcceptsDeployTier, -} from "#shared/db/built-sites.ts"; +} from "#shared/db/built-sites/types.ts"; +import { builtSites } from "#shared/db/built-sites.ts"; /** Extract the bearer token from the Authorization header (empty if absent). */ const bearerToken = (request: Request): string => { diff --git a/src/features/middleware.ts b/src/features/middleware.ts index bbd8b6e59..1735d9e0b 100644 --- a/src/features/middleware.ts +++ b/src/features/middleware.ts @@ -158,11 +158,9 @@ export const isValidContentType = (request: Request, path: string): boolean => { if (request.method !== "POST") { return true; } - // The scheduled maintenance ping and the inter-instance credentials endpoint - // carry no body and use no cookie/session auth (the latter authenticates via a - // bearer key), so there's nothing to CSRF-protect — accept a bare - // `curl -X POST` with no content-type. - if (path === "/scheduled" || path === "/instance/site-credentials") { + // The inter-instance credentials endpoint carries no body and authenticates + // via a bearer key, so there is nothing to CSRF-protect. + if (path === "/instance/site-credentials") { return true; } const contentType = lowerContentType(request); diff --git a/src/features/index.ts b/src/features/request-scopes.ts similarity index 64% rename from src/features/index.ts rename to src/features/request-scopes.ts index 916bdeb56..707f1d47b 100644 --- a/src/features/index.ts +++ b/src/features/request-scopes.ts @@ -1,5 +1,4 @@ import { parseAcceptLanguage, runWithLocale } from "#i18n"; -import { processRequest } from "#routes/app/request.ts"; import type { ServerContext } from "#routes/types.ts"; import { getClientIp } from "#routes/url.ts"; import { runWithClientIp } from "#shared/client-context.ts"; @@ -12,16 +11,19 @@ import { runWithIframeContext } from "#shared/iframe.ts"; import { runWithRequestId } from "#shared/logger.ts"; import { runWithRequestCache } from "#shared/request-cache.ts"; import { runWithSessionContext } from "#shared/session-context.ts"; +import { runWithSubrequestBudget } from "#shared/subrequest-budget.ts"; -/** Handle one request inside every request-scoped store. */ -export const handleRequest = async ( +/** Run one response builder inside every request-scoped store. */ +export const runWithRequestScopes = ( request: Request, - server?: ServerContext, + server: ServerContext | undefined, + fn: () => Promise, ): Promise => { const locale = parseAcceptLanguage(request.headers.get("accept-language")); - const scopes: ((fn: () => Promise) => Promise)[] = [ - (fn) => runWithLocale(locale, fn), - (fn) => runWithClientIp(getClientIp(request, server), fn), + const scopes: ((next: () => Promise) => Promise)[] = [ + (next) => runWithLocale(locale, next), + (next) => runWithClientIp(getClientIp(request, server), next), + runWithSubrequestBudget, runWithRequestId, runWithRequestCache, runWithQueryLogContext, @@ -35,6 +37,16 @@ export const handleRequest = async ( return scopes.reduceRight<() => Promise>( (next, scope) => () => scope(next), - () => processRequest(request, server), + fn, )(); }; + +type RequestHandler = ( + request: Request, + server?: ServerContext, +) => Promise; + +export const requestScopedHandler = + (handler: RequestHandler): RequestHandler => + (request, server) => + runWithRequestScopes(request, server, () => handler(request, server)); diff --git a/src/features/scheduled.ts b/src/features/scheduled.ts index 77be840eb..d6a0997f7 100644 --- a/src/features/scheduled.ts +++ b/src/features/scheduled.ts @@ -1,84 +1,34 @@ -/** - * Public maintenance-ping endpoint. - * - * Database pruning runs as interval-gated pending work on *every* request (see - * `prepareRequestEnvironment`), so any traffic keeps a site pruned. This - * endpoint exists so a cron can guarantee pruning still happens on a site with - * no organic traffic: hitting `/scheduled` is just a cheap request that — like - * any dynamic request — triggers this site's prune, then returns a tiny JSON - * body. (Static asset routes short-circuit before pruning, so a cron must hit a - * dynamic path like this one, not `/favicon.ico`.) - * - * On a builder (`CAN_BUILD_SITES`), `POST /scheduled` additionally pokes the - * least-recently-poked built site with a plain GET, which triggers *that* - * site's own per-request prune. So one cron on the master walks every client at - * the cron's pace, and quiet client sites get pruned with no shared secret — - * the poke is an ordinary unauthenticated request, and pruning only ever - * deletes already-expired rows. Only POST walks, so a crawler's GET can't make - * the master fan out. - */ +/* jscpd:ignore-start */ +import { requestScopedHandler } from "#routes/request-scopes.ts"; +import { loadEffectiveDomain } from "#shared/config.ts"; +import { initDb } from "#shared/db/migrations.ts"; +import { settings } from "#shared/db/settings.ts"; +import { reportMaintenanceFailure } from "#shared/maintenance/report.ts"; +import { maintenance } from "#shared/maintenance/runner.ts"; +import { scheduledResponse } from "#shared/scheduled-access.ts"; +import { CONFIG_KEYS } from "#shared/settings/keys.ts"; +/* jscpd:ignore-end */ -import { isBuilderEnabled } from "#routes/admin/builder.ts"; -import { jsonResponse } from "#routes/response.ts"; -import { defineRoutes } from "#routes/router.ts"; -import { - claimNextBuiltSiteForPrune, - siteBaseUrl, -} from "#shared/db/built-sites.ts"; -import { fetchTextFollowingSafeRedirects } from "#shared/safe-fetch.ts"; - -const SCHEDULED_PATH = "/scheduled"; - -/** How long the master waits for a poked built site before giving up. Kept - * short to bound how long an unauthenticated POST can hold the master's - * outbound connection; the client's rotation stamp is already bumped, so a - * timeout just means it gets walked again next cycle, not a stalled rotation. */ -const POKE_TIMEOUT_MS = 15_000; - -/** Outcome of poking a built site. Deliberately free of any client-identifying - * detail (hostname, error text): this endpoint is public on a builder, so the - * response must not let a caller enumerate which sites the builder operates. */ -type PokeResult = { ok: boolean; status: number } | { failed: true }; - -/** - * Poke the least-recently-poked built site with a plain GET so its own - * per-request pruning runs. The site's rotation stamp is bumped (inside - * `claimNextBuiltSiteForPrune`) before the request goes out, so a slow or - * failing site never stalls the round-robin. The poke goes through the - * safe-redirect fetch, which validates the origin and every redirect hop - * against the SSRF policy, so a built site whose stored URL redirects to an - * internal address can't make the master follow it. Returns null when the - * builder has no built sites yet. - */ -const pokeNextBuiltSite = async (): Promise => { - const next = await claimNextBuiltSiteForPrune(); - if (!next) return null; +export const handleScheduledRequest = requestScopedHandler(async (request) => { try { - const url = `${siteBaseUrl(next.siteUrl)}${SCHEDULED_PATH}`; - const result = await fetchTextFollowingSafeRedirects(url, { - method: "GET", - signal: AbortSignal.timeout(POKE_TIMEOUT_MS), - }); - return { ok: result.ok, status: result.status }; - } catch { - return { failed: true }; + await initDb(); + await settings.loadKeys([ + CONFIG_KEYS.BUNNY_SUBDOMAIN, + CONFIG_KEYS.CUSTOM_DOMAIN, + CONFIG_KEYS.CUSTOM_DOMAIN_LAST_VALIDATED, + CONFIG_KEYS.SETUP_COMPLETE, + ]); + loadEffectiveDomain(new URL(request.url)); + if (!(await settings.setup.isComplete())) { + throw new Error("Scheduled maintenance requires completed setup"); + } + const { MAINTENANCE_TASKS } = await import( + "#shared/maintenance/registry.ts" + ); + await maintenance.run(MAINTENANCE_TASKS); + return scheduledResponse(204); + } catch (error) { + reportMaintenanceFailure("scheduled maintenance failed", error); + return scheduledResponse(503); } -}; - -/** - * Handle a scheduled-tasks ping. This site's own prune is already scheduled as - * pending work for the request (`prepareRequestEnvironment`), so the handler - * just steps the built-site rotation when a builder is POSTed to. - */ -const handleScheduled = async (request: Request): Promise => { - const walk = request.method === "POST" && isBuilderEnabled(); - const poked = walk ? await pokeNextBuiltSite() : null; - return jsonResponse({ ok: true, poked }); -}; - -/** Scheduled-tasks routes — any hit self-prunes (via the request); POST on a - * builder also pokes the next built site. */ -export const scheduledRoutes = defineRoutes({ - "GET /scheduled": handleScheduled, - "POST /scheduled": handleScheduled, }); diff --git a/src/features/settings-bundles.ts b/src/features/settings-bundles.ts index 68fc7e50d..9bec39014 100644 --- a/src/features/settings-bundles.ts +++ b/src/features/settings-bundles.ts @@ -11,7 +11,6 @@ import { CONFIG_KEYS, EMAIL_BODY_KEYS, - PRUNE_KEYS, SNAPSHOT_KEYS, } from "#shared/db/settings.ts"; @@ -29,8 +28,6 @@ export const getPrefix = (path: string): string => { * and every HTML error page) reads theme + underline_links + header_image_url * - `applySecurityHeaders` rebuilds the CSP on every routed response, reading * the payment provider (and square_sandbox when the provider is Square) - * - pruning self-guards on last_pruned_* - * - the activity-log backfill self-guards on its done flag + last-run stamp * - session auth + PII decryption read the key material * - listing reads resolve listing defaults at the cache layer * (`resolveListingDefaults`), which can run on any route that loads a listing; @@ -50,19 +47,8 @@ const INFRA_SETTINGS: readonly string[] = [ CONFIG_KEYS.HEADER_IMAGE_URL, CONFIG_KEYS.PAYMENT_PROVIDER, CONFIG_KEYS.SQUARE_SANDBOX, - CONFIG_KEYS.LAST_PRUNED_PAYMENTS, - CONFIG_KEYS.LAST_PRUNED_SESSIONS, - CONFIG_KEYS.LAST_PRUNED_SUMUP, - // The orphaned-attendee auto-purge runs from the same fire-and-forget - // scheduler, so its enable flag, retention age, and last-run stamp must be - // readable on every request. - ...PRUNE_KEYS, CONFIG_KEYS.AUTO_PURGE_ORPHANS, CONFIG_KEYS.ORPHAN_PURGE_RETENTION, - // The activity-log backfill runs from the same fire-and-forget scheduler and - // self-guards on these every request until it has converted every legacy row. - CONFIG_KEYS.ACTIVITY_LOG_BACKFILL_DONE, - CONFIG_KEYS.LAST_ACTIVITY_LOG_BACKFILL, CONFIG_KEYS.PUBLIC_KEY, CONFIG_KEYS.WRAPPED_PRIVATE_KEY, ]; @@ -228,9 +214,6 @@ const PREFIX_SETTINGS: Record = { // Renewal renders the same booking form, whose contact-field builder checks // the address-lookup provider. renew: [...BOOKING_FLOW_SETTINGS, CONFIG_KEYS.ADDRESS_LOOKUP_PROVIDER], - // Cron prune trigger: maybeRunPrunes only reads the last_pruned_*/orphan - // settings, which are all in INFRA, so infra alone is enough. - scheduled: [], setup: [], // --- Inbound SMS webhook (JSON only) --- sms: [ diff --git a/src/features/url.ts b/src/features/url.ts index 0f009bddf..0874e959f 100644 --- a/src/features/url.ts +++ b/src/features/url.ts @@ -39,13 +39,6 @@ export const getBaseUrl = (request: Request): string => { return `${url.protocol}//${url.host}`; }; -/** - * Normalize path by stripping trailing slashes (except root "/") - * This allows consistent path comparisons like "/admin" instead of checking both "/admin" and "/admin/" - */ -export const normalizePath = (path: string): string => - path !== "/" && path.endsWith("/") ? path.slice(0, -1) : path; - /** * Parse request URL and extract path/method * Paths are normalized to strip trailing slashes @@ -64,3 +57,5 @@ export const getSearchParam = (request: Request, key: string): string => { const url = new URL(request.url); return url.searchParams.get(key) ?? ""; }; + +import { normalizePath } from "#shared/path.ts"; diff --git a/src/locales/en/built-sites.json b/src/locales/en/built-sites.json index ff3c89597..642c9b97c 100644 --- a/src/locales/en/built-sites.json +++ b/src/locales/en/built-sites.json @@ -20,6 +20,12 @@ "built_sites.create_built_site_button": "Create Built Site", "built_sites.edit_site_title": "Edit Built Site", "built_sites.renewal_title": "Renewal", + "built_sites.maintenance_title": "Scheduled maintenance", + "built_sites.maintenance_intro": "Use this key in the monitor for this site. Each site has its own key.", + "built_sites.maintenance_site_key": "Site key", + "built_sites.maintenance_provision": "Set up scheduled maintenance", + "built_sites.maintenance_resend": "Send key to site again", + "built_sites.maintenance_provisioned": "Scheduled maintenance key sent to the site.", "built_sites.current_deadline": "Current deadline:", "built_sites.raw_iso": "Raw ISO", "built_sites.renewal_url": "Renewal URL:", diff --git a/src/locales/en/debug.json b/src/locales/en/debug.json index b8f066535..8dfb76764 100644 --- a/src/locales/en/debug.json +++ b/src/locales/en/debug.json @@ -79,12 +79,9 @@ "debug.field.effective_domain": "Effective domain", "debug.field.schema_status": "Schema status", "debug.field.schema_hash": "Schema hash", - "debug.field.table": "Table", - "debug.field.last_pruned_utc": "Last pruned (UTC)", "debug.section.runtime": "Runtime", "debug.section.site": "Site", "debug.section.availability": "Availability", "debug.section.bunny": "Bunny", - "debug.section.database_pruning": "Database pruning", "debug.section.database_domain": "Database & Domain" } diff --git a/src/locales/en/settings.json b/src/locales/en/settings.json index 3f0811f72..62d4ed44b 100644 --- a/src/locales/en/settings.json +++ b/src/locales/en/settings.json @@ -1,4 +1,7 @@ { + "settings.advanced.scheduled_title": "Scheduled maintenance", + "settings.advanced.scheduled_intro": "Use this key in the monitor for this site.", + "settings.advanced.scheduled_unset": "No scheduled maintenance key is set on this site.", "settings.title": "Settings", "settings.guide_link": "Settings guide", "settings.header_image": "Header Image", diff --git a/src/serve-app.ts b/src/serve-app.ts index 7f26adbae..e2e2be359 100644 --- a/src/serve-app.ts +++ b/src/serve-app.ts @@ -14,6 +14,7 @@ import { once } from "#fp"; import { handleRequest } from "#routes"; import { temporaryErrorResponse } from "#routes/response.ts"; import { validateBootChecks } from "#shared/boot-checks.ts"; +import { seedEffectiveDomainHost } from "#shared/config.ts"; import { setN1GuardNotifyOnly } from "#shared/db/query-log.ts"; import { getEnv } from "#shared/env.ts"; import { @@ -22,6 +23,10 @@ import { logDebug, logError, } from "#shared/logger.ts"; +import { + scheduledAccessFromEnv, + scheduledResponse, +} from "#shared/scheduled-access.ts"; import { initSentry } from "#shared/sentry.ts"; const runtimeLoadFinishedAt = performance.now(); @@ -75,11 +80,20 @@ const initialize = once((): Promise => { * turned into a generic 503 so a single bad request never crashes the isolate. */ export const serveHandler = async (request: Request): Promise => { + const scheduledAccess = scheduledAccessFromEnv(request); + if (scheduledAccess.kind === "rejected") { + return scheduledResponse(scheduledAccess.status); + } + const url = new URL(request.url); + if (scheduledAccess.kind === "authorized") seedEffectiveDomainHost(url); try { await initialize(); + if (scheduledAccess.kind === "authorized") { + const { handleScheduledRequest } = await import("#routes/scheduled.ts"); + return await handleScheduledRequest(request); + } return await handleRequest(request); } catch (error) { - const url = new URL(request.url); logError({ code: ErrorCode.CDN_REQUEST, detail: `unhandled ${formatRequestError( @@ -89,6 +103,8 @@ export const serveHandler = async (request: Request): Promise => { )}`, error, }); - return temporaryErrorResponse(); + return scheduledAccess.kind === "authorized" + ? scheduledResponse(503) + : temporaryErrorResponse(); } }; diff --git a/src/shared/admin-surface/routes/built-sites.ts b/src/shared/admin-surface/routes/built-sites.ts index 5889c7fd3..49f62c530 100644 --- a/src/shared/admin-surface/routes/built-sites.ts +++ b/src/shared/admin-surface/routes/built-sites.ts @@ -53,6 +53,12 @@ export const routes = [ "POST", "/admin/built-sites/:id/provision-renewal", ), + route( + "postBuiltSitesByIdProvisionScheduler", + "builtSites", + "POST", + "/admin/built-sites/:id/provision-scheduler", + ), route( "postBuiltSitesByIdReSyncDeadline", "builtSites", diff --git a/src/shared/boot-checks.ts b/src/shared/boot-checks.ts index 30b001fe9..085f237c6 100644 --- a/src/shared/boot-checks.ts +++ b/src/shared/boot-checks.ts @@ -7,6 +7,7 @@ import { utf8ByteLength } from "#shared/bytes.ts"; import { validateEncryptionKey } from "#shared/crypto/encryption.ts"; import { getEnv } from "#shared/env.ts"; +import { validateScheduledTaskKey } from "#shared/scheduled-keys.ts"; export type BootCheck = { name: string; @@ -34,6 +35,7 @@ export const validateOptionalMainInstanceKey = (): void => { export const BOOT_CHECKS: readonly BootCheck[] = [ { name: "DB_ENCRYPTION_KEY", run: validateEncryptionKey }, { name: "MAIN_INSTANCE_KEY", run: validateOptionalMainInstanceKey }, + { name: "SCHEDULED_TASK_KEY", run: validateScheduledTaskKey }, ]; export const validateBootChecks = (): void => { diff --git a/src/shared/builder.ts b/src/shared/builder.ts index 27a7ee126..eb0b6be5c 100644 --- a/src/shared/builder.ts +++ b/src/shared/builder.ts @@ -22,12 +22,17 @@ import { bunnyHostingProvider } from "#shared/bunny-cdn.ts"; import { bunnyDbProvider } from "#shared/bunny-db.ts"; import { getDefaultDbProvider } from "#shared/config.ts"; import { toBase64 } from "#shared/crypto/utils.ts"; -import type { DbProvider, HostingProvider } from "#shared/db/built-sites.ts"; +import type { + DbProvider, + HostingProvider, +} from "#shared/db/built-sites/types.ts"; import { denoHostingProvider } from "#shared/deno-deploy-api.ts"; import { getEnv } from "#shared/env.ts"; +import { errorMessage } from "#shared/error-message.ts"; import { fetchText } from "#shared/fetch.ts"; import type { HostingProviderApi } from "#shared/provider-types.ts"; import { errorResult } from "#shared/result.ts"; +import { generateScheduledTaskKey } from "#shared/scheduled-keys.ts"; import { withSiteDb } from "#shared/site-db.ts"; import { tryStep } from "#shared/try-step.ts"; import { tursoDbProvider } from "#shared/turso-api.ts"; @@ -103,6 +108,12 @@ export type BuildSiteResult = } | { ok: false; error: string }; +export type PreparedBuildSite = Extract & { + scheduledTaskKey: string; +}; + +export type RetainPreparedSite = (site: PreparedBuildSite) => Promise; + type BuildSiteCredentials = { dbUrl: string; dbToken: string }; /** Generate a random 32-byte base64 encryption key */ @@ -216,20 +227,20 @@ const buildSiteOnProvider = async ( dbCredentials: BuildSiteCredentials, dbProvider: DbProvider, hostingProvider: HostingProvider, + retain: RetainPreparedSite, ): Promise => { const fullName = `Tickets - ${input.siteName}`; const encryptionKey = builderApi.generateEncryptionKey(); + const scheduledTaskKey = generateScheduledTaskKey(); const secrets: [string, string][] = [ ...buildBaseSecrets(dbCredentials, encryptionKey), + ["SCHEDULED_TASK_KEY", scheduledTaskKey], ...collectHostSecrets(hostingProvider), ]; - const result = await resolveHostingProvider(hostingProvider).createSite( - fullName, - code, - secrets, - ); + const provider = resolveHostingProvider(hostingProvider); + const result = await provider.prepareSite(fullName, code, secrets); if (!result.ok) return result; - return { + const prepared: PreparedBuildSite = { dbProvider, dbToken: dbCredentials.dbToken, dbUrl: dbCredentials.dbUrl, @@ -237,7 +248,20 @@ const buildSiteOnProvider = async ( hostingId: result.value.hostingId, hostingProvider, ok: true, + scheduledTaskKey, }; + try { + await retain(prepared); + } catch (error) { + return { + error: `Failed to retain site: ${errorMessage(error)}`, + ok: false, + }; + } + const published = await provider.publishSite(result.value.hostingId, code); + if (!published.ok) return published; + const { scheduledTaskKey: _scheduledTaskKey, ...built } = prepared; + return built; }; /** @@ -246,6 +270,7 @@ const buildSiteOnProvider = async ( */ export const buildSite = async ( input: BuildSiteInput, + retain: RetainPreparedSite, ): Promise => { // 1. Source the bundle code: caller-supplied or latest GitHub release const codeResult = await getBuildCode(input); @@ -263,6 +288,7 @@ export const buildSite = async ( dbCredentials, dbProvider, input.hostingProvider ?? "bunny", + retain, ); }; diff --git a/src/shared/bunny-cdn.ts b/src/shared/bunny-cdn.ts index 85cb8692c..6b5c5cc94 100644 --- a/src/shared/bunny-cdn.ts +++ b/src/shared/bunny-cdn.ts @@ -12,7 +12,7 @@ import { getBunnyScriptId, } from "#shared/config.ts"; import { type FetchResult, fetchText, parseApiError } from "#shared/fetch.ts"; -import { ErrorCode, logDebug, logError } from "#shared/logger.ts"; +import { ErrorCode, logError } from "#shared/logger.ts"; import { delay } from "#shared/now.ts"; import type { HostingProviderApi } from "#shared/provider-types.ts"; import { errorResult, okResult, type Result } from "#shared/result.ts"; @@ -344,10 +344,6 @@ const registerBunnySubdomainImpl = async ( Value: target, }; const dnsUrl = `${BUNNY_API_BASE}/dnszone/${zoneId}/records`; - logDebug( - "Domain", - `Adding DNS CNAME: url=${dnsUrl} name=${recordName} value=${target} fullDomain=${fullDomain}`, - ); const addResponse = await bunnyJsonRequest( dnsUrl, JSON.stringify(dnsRecordBody), @@ -382,12 +378,6 @@ const registerBunnySubdomainImpl = async ( attempt < CERT_RETRY_COUNT && !cdnResult.ok; attempt++ ) { - logDebug( - "Domain", - `Certificate not ready, retrying in ${certRetryDelay( - attempt, - )}ms (attempt ${attempt + 1}/${CERT_RETRY_COUNT})`, - ); await bunnyCdnApi.delay(certRetryDelay(attempt)); cdnResult = await bunnyCdnApi.validateCustomDomain(fullDomain); } @@ -412,7 +402,6 @@ const deleteDnsRecordImpl = async ( recordId: number, ): Promise => { const url = `${BUNNY_API_BASE}/dnszone/${zoneId}/records/${recordId}`; - logDebug("Domain", `Deleting DNS record: ${url}`); const response = await bunnyKeyRequest(url, "DELETE"); return okOrError(response, "Delete DNS record"); }; @@ -587,9 +576,16 @@ const deployScriptCodeImpl = async ( // Hosting provider interface implementation (site builder + secrets backfill) // --------------------------------------------------------------------------- +const bunnyVoidResult = (result: BunnyApiResult): Result => + result.ok ? okResult(undefined) : errorResult(result.error); + export const bunnyHostingProvider: HostingProviderApi = { configEnvVar: "BUNNY_API_KEY", - async createSite(name, code, secrets) { + async getSecretNames(hostingId) { + const result = await bunnyCdnApi.listEdgeScriptSecrets(Number(hostingId)); + return result.ok ? okResult(result.secrets.map((s) => s.Name)) : result; + }, + async prepareSite(name, code, secrets) { const createResult = await bunnyCdnApi.createEdgeScript(name, code); if (!createResult.ok) return createResult; const { scriptId, pullZoneId, defaultHostname } = createResult; @@ -613,13 +609,12 @@ export const bunnyHostingProvider: HostingProviderApi = { ok: false as const, }; } - const publishResult = await bunnyCdnApi.publishEdgeScript(scriptId); - if (!publishResult.ok) return publishResult; return okResult({ defaultHostname, hostingId: String(scriptId) }); }, - async getSecretNames(hostingId) { - const result = await bunnyCdnApi.listEdgeScriptSecrets(Number(hostingId)); - return result.ok ? okResult(result.secrets.map((s) => s.Name)) : result; + async publishSite(hostingId) { + return bunnyVoidResult( + await bunnyCdnApi.publishEdgeScript(Number(hostingId)), + ); }, async setSecrets(hostingId, secrets) { const scriptId = Number(hostingId); @@ -673,7 +668,5 @@ export const getCdnHostname = (): Promise => export const deployScriptCode = async ( code: string, scriptId?: number | string, -): Promise> => { - const result = await bunnyCdnApi.deployScriptCode(code, scriptId); - return result.ok ? okResult(undefined) : errorResult(result.error); -}; +): Promise> => + bunnyVoidResult(await bunnyCdnApi.deployScriptCode(code, scriptId)); diff --git a/src/shared/db/activity-log-backfill.ts b/src/shared/db/activity-log-backfill.ts index ba0e1ce0f..a126cd3d4 100644 --- a/src/shared/db/activity-log-backfill.ts +++ b/src/shared/db/activity-log-backfill.ts @@ -9,25 +9,21 @@ * * It is resumable without a cursor: a converted row no longer matches the * `enc:` prefix, so each batch shrinks the remaining set. A batch that finds - * nothing flips a done flag so the scan stops permanently. Scheduling mirrors - * the prune tasks — fire-and-forget from the request handler, interval-gated by - * a `last_run` timestamp — to stay within the edge subrequest budget while - * converging over successive requests. + * nothing leaves the task disabled until more legacy data exists. */ /* jscpd:ignore-start */ import { decrypt, ENCRYPTION_PREFIX } from "#shared/crypto/encryption.ts"; import { encryptWithOwnerKey } from "#shared/crypto/keys.ts"; import type { EnvKeyEncrypted } from "#shared/crypto/sealed.ts"; -import { executeBatch, queryAll, update } from "#shared/db/client.ts"; -import { settings } from "#shared/db/settings.ts"; -import { taskIsDue } from "#shared/interval-gate.ts"; import { - ACTIVITY_LOG_BACKFILL_BATCH, - ACTIVITY_LOG_BACKFILL_INTERVAL_MS, -} from "#shared/limits.ts"; + executeBatch, + queryAll, + rowExists, + update, +} from "#shared/db/client.ts"; +import { ACTIVITY_LOG_BACKFILL_BATCH } from "#shared/limits.ts"; import { logDebug } from "#shared/logger.ts"; -import { nowMs } from "#shared/now.ts"; /* jscpd:ignore-end */ @@ -46,7 +42,7 @@ export const backfillActivityLogBatch = async ( publicKey: string, ): Promise => { const rows = await queryAll( - "SELECT id, message FROM activity_log WHERE message LIKE ? LIMIT ?", + "SELECT id, message FROM activity_log WHERE message LIKE ? ORDER BY id LIMIT ?", [`${ENCRYPTION_PREFIX}%`, ACTIVITY_LOG_BACKFILL_BATCH], ); if (rows.length === 0) return 0; @@ -68,37 +64,16 @@ export const backfillActivityLogBatch = async ( return rows.length; }; -/** Backfill is finished, or cannot run yet because no key pair is configured. */ -const backfillIdle = (): boolean => - settings.activityLogBackfillDone === "true" || !settings.publicKey; +export const hasLegacyActivityLog = (): Promise => + rowExists("SELECT 1 FROM activity_log WHERE message LIKE ? LIMIT 1", [ + `${ENCRYPTION_PREFIX}%`, + ]); -/** - * Run one backfill batch if due. Safe to call fire-and-forget - * (`addPendingWork`); never throws. Writes the run timestamp before working - * (claiming the interval so concurrent requests don't double-run) and marks the - * job done once a batch finds no remaining legacy rows. - */ -export const maybeBackfillActivityLog = async (): Promise => { - if (backfillIdle()) return; - const now = nowMs(); - if ( - !taskIsDue( - settings.lastActivityLogBackfill, - ACTIVITY_LOG_BACKFILL_INTERVAL_MS, - now, - ) - ) { - return; - } - try { - await settings.update.lastActivityLogBackfill(String(now)); - const converted = await backfillActivityLogBatch(settings.publicKey); - if (converted === 0) { - await settings.update.activityLogBackfillDone("true"); - } else { - logDebug("Backfill", `activity_log: re-encrypted ${converted} rows`); - } - } catch (e) { - logDebug("Backfill", `activity_log failed: ${String(e)}`); +export const runActivityLogBackfill = async ( + publicKey: string, +): Promise => { + const converted = await backfillActivityLogBatch(publicKey); + if (converted > 0) { + logDebug("Backfill", `activity_log: re-encrypted ${converted} rows`); } }; diff --git a/src/shared/db/address-cache.ts b/src/shared/db/address-cache.ts index 782f01a2e..fb34ebefc 100644 --- a/src/shared/db/address-cache.ts +++ b/src/shared/db/address-cache.ts @@ -18,8 +18,8 @@ import { import { decrypt, encrypt } from "#shared/crypto/encryption.ts"; import { hmacHash } from "#shared/crypto/hashing.ts"; import type { BlindIndex, EnvKeyEncrypted } from "#shared/crypto/sealed.ts"; -import { execute, queryOne } from "#shared/db/client.ts"; -import { ADDRESS_CACHE_MS } from "#shared/limits.ts"; +import { execute, queryOne, type SqlStatement } from "#shared/db/client.ts"; +import { ADDRESS_CACHE_MS, MAINTENANCE_PRUNE_BATCH } from "#shared/limits.ts"; import { nowMs } from "#shared/now.ts"; import { defineStoredJson } from "#shared/validation/stored-json.ts"; /* jscpd:ignore-end */ @@ -37,6 +37,15 @@ export const computeAddressSearchIndex = ( const freshCutoffIso = (): string => new Date(nowMs() - ADDRESS_CACHE_MS).toISOString(); +export const addressCachePruneStatement = (): SqlStatement => ({ + args: [freshCutoffIso(), MAINTENANCE_PRUNE_BATCH], + sql: `DELETE FROM address_cache + WHERE rowid IN ( + SELECT rowid FROM address_cache WHERE created < ? + ORDER BY rowid LIMIT ? + )`, +}); + const cachedAddressesJson = defineStoredJson( v.union([v.array(v.string()), v.array(AddressMatchSchema)]), ); diff --git a/src/shared/db/api-key-attempts.ts b/src/shared/db/api-key-attempts.ts index d78b5860a..471bea90d 100644 --- a/src/shared/db/api-key-attempts.ts +++ b/src/shared/db/api-key-attempts.ts @@ -4,7 +4,7 @@ * Only *failed* Bearer lookups are counted, so legitimate clients with a valid * key are never throttled. This caps brute-force guessing and the database load * of a token-guessing flood. Counters share the login_attempts table under a - * dedicated namespace and are cleaned by pruneLoginAttempts. + * dedicated namespace and are cleaned by database pruning. */ import { makeIpRateLimiter } from "#shared/db/login-attempts.ts"; diff --git a/src/shared/db/built-site-scheduler.ts b/src/shared/db/built-site-scheduler.ts new file mode 100644 index 000000000..3fa0a52b2 --- /dev/null +++ b/src/shared/db/built-site-scheduler.ts @@ -0,0 +1,16 @@ +import { updateBuiltSite } from "#shared/db/built-sites.ts"; +import { generateScheduledTaskKey } from "#shared/scheduled-keys.ts"; + +/** Create one active key for a built site, or return its existing key. */ +export const ensureBuiltSiteSchedulerKey = async ( + siteId: number, +): Promise => { + const candidate = generateScheduledTaskKey(); + let key = candidate; + const updated = await updateBuiltSite(siteId, (existing) => { + key = existing.scheduledTaskKey ?? candidate; + return existing.scheduledTaskKey ? null : { scheduledTaskKey: candidate }; + }); + if (!updated) throw new Error(`Built site not found: ${siteId}`); + return key; +}; diff --git a/src/shared/db/built-sites.ts b/src/shared/db/built-sites.ts index 92cef6fa9..2eebaeb37 100644 --- a/src/shared/db/built-sites.ts +++ b/src/shared/db/built-sites.ts @@ -4,232 +4,45 @@ */ import type { InValue } from "@libsql/client"; -import * as v from "valibot"; /* jscpd:ignore-start */ import { decrypt, encrypt } from "#shared/crypto/encryption.ts"; -import type { EnvKeyEncrypted } from "#shared/crypto/sealed.ts"; import { queryAll, queryOne, rowExistsForIdList } from "#shared/db/client.ts"; -import type { ColumnDef, Table, TableSchema } from "#shared/db/table.ts"; +import { retryWrite } from "#shared/db/retry-write.ts"; +import type { Table, TableSchema } from "#shared/db/table.ts"; import { cachedTable, col, defineTable } from "#shared/db/table.ts"; -import { nowIso } from "#shared/now.ts"; -import { guardFor } from "#shared/validation/guard.ts"; -import { defineStoredJson } from "#shared/validation/stored-json.ts"; -import { OptionalStringSchema } from "#shared/validation/string.ts"; -/* jscpd:ignore-end */ - -/** - * The release channels a built site can opt into, ordered most- to - * least-eager. The array order IS the rank (its index): an alpha site takes - * every deploy, a beta site takes beta + release, a release site only stable - * releases. So a site on tier S accepts a deploy published at tier T exactly - * when `indexOf(S) <= indexOf(T)` — see {@link siteAcceptsDeployTier}. - */ -export const UPDATE_TIERS = ["alpha", "beta", "release"] as const; -export const UpdateTierSchema = v.picklist(UPDATE_TIERS); - -/** One of the {@link UPDATE_TIERS} release channels. */ -export type UpdateTier = v.InferOutput; - -/** Default channel for a new built site — the most conservative (stable only). */ -export const DEFAULT_UPDATE_TIER: UpdateTier = "release"; - -/** Narrow an arbitrary string to an {@link UpdateTier}. */ -export const isUpdateTier = guardFor(UpdateTierSchema); - -/** - * True when a site on `siteTier` should receive a deploy published at - * `deployTier`. A release deploy reaches every site, a beta deploy reaches beta - * + alpha sites, an alpha deploy only alpha sites — i.e. the site's channel must - * be at the deploy's tier or more eager. - */ -export const siteAcceptsDeployTier = ( - siteTier: UpdateTier, - deployTier: UpdateTier, -): boolean => - UPDATE_TIERS.indexOf(siteTier) <= UPDATE_TIERS.indexOf(deployTier); - -/** Encrypted site-data blob version */ -const SITE_DATA_BLOB_VERSION = 1; - -/** Hosting provider for a built site. */ -export type HostingProvider = "bunny" | "deno"; - -/** Database provider for a built site. */ -export type DbProvider = "bunny" | "turso"; - -/** Use the named provider when selected, otherwise use Bunny. */ -export const providerOrBunny = < - TProvider extends Exclude, ->( - value: string | null | undefined, - provider: TProvider, -): "bunny" | TProvider => (value === provider ? provider : "bunny"); - -/** Encrypted site data blob shape, including fields absent from older blobs. */ -const SiteDataBlobSchema = v.strictObject({ - d: OptionalStringSchema, - dp: v.optional(v.picklist(["bunny", "turso"])), - hp: v.optional(v.picklist(["bunny", "deno"])), - n: v.string(), - rt: OptionalStringSchema, - s: OptionalStringSchema, - t: OptionalStringSchema, - u: v.string(), - v: v.optional(v.literal(SITE_DATA_BLOB_VERSION), SITE_DATA_BLOB_VERSION), -}); -export type SiteDataBlob = v.InferOutput; -const siteDataJson = defineStoredJson(SiteDataBlobSchema); - -/** Built site row as stored in the database */ -export interface BuiltSiteRow { - assignable: number; - assigned_attendee_id: number | null; - assigned_listing_id: number | null; - created: string; - id: number; - read_only_from: string; - renewal_token_index: string | null; - site_data: string; - /** Release channel — a CHECK constraint keeps this a valid UpdateTier. */ - updates: UpdateTier; -} - -type BuiltSitePlainInput = { - assignable?: number; - assignedAttendeeId?: number | null; - assignedListingId?: number | null; - renewalTokenIndex?: string | null; - readOnlyFrom?: string; - updates?: UpdateTier; -}; - -/** Built site input for creating a new row */ -export type BuiltSiteInput = BuiltSitePlainInput & { - siteData: string; -}; - -/** Decrypted built site for display */ -export interface BuiltSite { - assignable: boolean; - assignedAttendeeId: number | null; - assignedListingId: number | null; - created: string; - dbProvider: DbProvider; - dbToken: string; - dbUrl: string; - hostingId: string; - hostingProvider: HostingProvider; - id: number; - name: string; - readOnlyFrom: string; - /** Plain renewal token from the site-data blob when renewal access exists. Null when not provisioned. */ - renewalToken: string | null; - renewalTokenIndex: string | null; - /** Site URL (default hostname for this site). */ - siteUrl: string; - /** Release channel this site opts into (see {@link UPDATE_TIERS}). */ - updates: UpdateTier; -} - -/** Form input for CRUD operations. `updates` is optional — programmatic - * inserts (e.g. auto-assignment) omit it and fall back to DEFAULT_UPDATE_TIER. */ -export type BuiltSiteFormInput = Pick< - BuiltSite, - "name" | "siteUrl" | "dbUrl" | "dbToken" | "hostingId" | "assignable" -> & { - updates?: UpdateTier; - hostingProvider?: HostingProvider; - dbProvider?: DbProvider; -}; +import { + blobToSiteFields, + buildSiteDataBlobFromInput, + parseSiteDataBlob, +} from "./built-sites/blob.ts"; +import { + builtSiteCrudBlobSchema, + builtSiteCrudPlainSchema, + builtSiteFormMappings, + builtSiteInputKeyMap, + builtSitePlainColumns, + builtSitePlainSchema, + createdCol, + emptyBuiltSiteFormInput, + idCol, + mapPlainFields, + plainSiteInput, +} from "./built-sites/fields.ts"; +import { + type BuiltSite, + type BuiltSiteBlobInput, + type BuiltSiteFormInput, + type BuiltSiteInput, + type BuiltSitePlainFields, + type BuiltSiteRow, + type BuiltSiteUpdate, + type DbProvider, + DEFAULT_UPDATE_TIER, + type HostingProvider, + type UpdateTier, +} from "./built-sites/types.ts"; -const idCol = col.generated(); -const createdCol = col.withDefault(() => nowIso()); - -const assignableCol = {} as ColumnDef; -const nullCol = col.withDefault(() => null); -const nullStrCol = col.withDefault(() => null); - -type BuiltSitePlainFields = Pick< - BuiltSite, - | "assignable" - | "assignedAttendeeId" - | "assignedListingId" - | "readOnlyFrom" - | "renewalTokenIndex" - | "updates" ->; - -const passthrough = (value: T): T => value; -const nullable = (value: T | null): T | null => value ?? null; - -const builtSitePlainColumns = [ - { - dbKey: "assignable", - formDefault: false, - fromRow: (value: number): boolean => Boolean(value), - inputKey: "assignable", - schema: assignableCol, - siteKey: "assignable", - toInput: (value: boolean): number => (value ? 1 : 0), - }, - { - dbKey: "assigned_attendee_id", - fromRow: nullable, - inputKey: "assignedAttendeeId", - schema: nullCol, - siteKey: "assignedAttendeeId", - toInput: nullable, - }, - { - dbKey: "assigned_listing_id", - fromRow: nullable, - inputKey: "assignedListingId", - schema: nullCol, - siteKey: "assignedListingId", - toInput: nullable, - }, - { - dbKey: "read_only_from", - fromRow: passthrough, - inputKey: "readOnlyFrom", - schema: col.withDefault(() => ""), - siteKey: "readOnlyFrom", - toInput: passthrough, - }, - { - dbKey: "renewal_token_index", - fromRow: nullable, - inputKey: "renewalTokenIndex", - schema: nullStrCol, - siteKey: "renewalTokenIndex", - toInput: nullable, - }, - { - dbKey: "updates", - formDefault: DEFAULT_UPDATE_TIER, - fromRow: passthrough, - inputKey: "updates", - schema: col.withDefault(() => DEFAULT_UPDATE_TIER), - siteKey: "updates", - toInput: passthrough, - }, -] as const; - -type BuiltSitePlainColumn = (typeof builtSitePlainColumns)[number]; - -const crudSchemaFor = ( - columns: readonly Column[], -): Pick, Column["siteKey"]> => - Object.fromEntries(columns.map(({ siteKey }) => [siteKey, {}])) as Pick< - TableSchema, - Column["siteKey"] - >; - -const builtSitePlainSchema = Object.fromEntries( - builtSitePlainColumns.map(({ dbKey, schema }) => [dbKey, schema]), -) as Pick, BuiltSitePlainColumn["dbKey"]>; - -const builtSiteCrudPlainSchema = crudSchemaFor(builtSitePlainColumns); +/* jscpd:ignore-end */ const rawBuiltSiteSchema = { ...builtSitePlainSchema, @@ -246,169 +59,11 @@ const rawBuiltSitesTable = defineTable({ schema: rawBuiltSiteSchema, }); -type BuiltSiteBlobFields = Pick< - BuiltSite, - | "hostingId" - | "siteUrl" - | "dbToken" - | "dbUrl" - | "name" - | "renewalToken" - | "hostingProvider" - | "dbProvider" ->; - -type BuiltSiteBlobInput = Omit & { - renewalToken?: string | null | undefined; -}; - -const builtSiteBlobColumns = [ - { - blobKey: "n", - defaultValue: "", - formDbKey: "name", - required: true, - siteKey: "name", - }, - { - blobKey: "u", - defaultValue: "", - formDbKey: "site_url", - required: true, - siteKey: "siteUrl", - }, - { - blobKey: "d", - defaultValue: "", - formDbKey: "db_url", - required: false, - siteKey: "dbUrl", - }, - { - blobKey: "t", - defaultValue: "", - formDbKey: "db_token", - required: false, - siteKey: "dbToken", - }, - { - blobKey: "s", - defaultValue: "", - formDbKey: "hosting_id", - required: false, - siteKey: "hostingId", - }, - { - blobKey: "hp", - defaultValue: "bunny" as HostingProvider, - formDbKey: "hosting_provider", - required: false, - siteKey: "hostingProvider", - }, - { - blobKey: "dp", - defaultValue: "bunny" as DbProvider, - formDbKey: "db_provider", - required: false, - siteKey: "dbProvider", - }, - { - blobKey: "rt", - defaultValue: null, - required: false, - siteKey: "renewalToken", - }, -] as const; - -type BuiltSiteBlobColumn = (typeof builtSiteBlobColumns)[number]; - -const builtSiteCrudBlobSchema = - crudSchemaFor(builtSiteBlobColumns); - -type BuiltSiteFormMapping = { - dbKey: string; - defaultValue: boolean | string; - siteKey: keyof BuiltSiteFormInput; -}; - -const builtSiteFormMappings: BuiltSiteFormMapping[] = [ - ...builtSitePlainColumns.flatMap((column) => - "formDefault" in column - ? [ - { - dbKey: column.dbKey, - defaultValue: column.formDefault, - siteKey: column.siteKey, - }, - ] - : [], - ), - ...builtSiteBlobColumns.flatMap((column) => - "formDbKey" in column - ? [ - { - dbKey: column.formDbKey, - defaultValue: column.defaultValue, - siteKey: column.siteKey, - }, - ] - : [], - ), -]; - -const builtSiteInputKeyMap = Object.fromEntries( - builtSiteFormMappings.map(({ dbKey, siteKey }) => [dbKey, siteKey]), -) as Record; - -const emptyBuiltSiteFormInput = (): BuiltSiteFormInput => - Object.fromEntries( - builtSiteFormMappings.map(({ defaultValue, siteKey }) => [ - siteKey, - defaultValue, - ]), - ) as BuiltSiteFormInput; - -const buildSiteDataBlobFromInput = ( - input: Partial, -): string => { - const blob = Object.fromEntries([ - ["v", SITE_DATA_BLOB_VERSION], - ...builtSiteBlobColumns.flatMap((column) => { - const value = (input[column.siteKey as keyof BuiltSiteBlobInput] ?? - column.defaultValue) as string | null; - return column.required || value ? [[column.blobKey, value]] : []; - }), - ]); - return siteDataJson.write(blob, "built_sites.site_data"); -}; - -const blobToSiteFields = (blob: SiteDataBlob): BuiltSiteBlobFields => - Object.fromEntries( - builtSiteBlobColumns.map((column) => [ - column.siteKey, - column.required - ? blob[column.blobKey as keyof SiteDataBlob] - : (blob[column.blobKey as keyof SiteDataBlob] ?? column.defaultValue), - ]), - ) as BuiltSiteBlobFields; - -const mapPlainFields = ( - input: Partial, - key: Key, -): Partial> => - Object.fromEntries( - builtSitePlainColumns.flatMap((column) => { - if (!Object.hasOwn(input, column.siteKey)) return []; - const value = input[column.siteKey] as never; - return [[column[key], column.toInput(value)]]; - }), - ) as Partial>; - /** Build raw table input from site-shaped fields */ const toRawInput = ( input: Partial & Partial, ): BuiltSiteInput => ({ - ...(mapPlainFields(input, "inputKey") as Partial), + ...plainSiteInput(input), siteData: buildSiteDataBlobFromInput(input), }); @@ -419,10 +74,6 @@ const toDbColumnValues = ( site_data: buildSiteDataBlobFromInput(input), }); -/** Parse a decrypted site data blob */ -export const parseSiteDataBlob = (json: string): SiteDataBlob => - siteDataJson.read(json, "built_sites.site_data"); - /** Convert a raw DB row (after decryption) to a BuiltSite */ const rowToBuiltSite = (row: BuiltSiteRow): BuiltSite => { const blob = parseSiteDataBlob(row.site_data); @@ -459,6 +110,47 @@ const queryAndDecrypt = async (sql: string): Promise => { return sites; }; +const findBuiltSiteById = async (id: InValue): Promise => { + const row = await rawBuiltSitesTable.findById(id); + return row ? rowToBuiltSite(row) : null; +}; + +export const findBuiltSiteByIdPrimary = async ( + id: InValue, +): Promise => { + const row = await rawBuiltSitesTable.findByIdPrimary!(id); + return row ? rowToBuiltSite(row) : null; +}; + +/** Update a whole built-site record without overwriting a concurrent blob write. */ +export const updateBuiltSite = ( + id: InValue, + changesFor: (existing: BuiltSite) => BuiltSiteUpdate | null, +): Promise => { + const updateStatement = rawBuiltSitesTable.updateStatement!; + return retryWrite(`Could not update built site ${String(id)}`, async () => { + const existing = await findBuiltSiteByIdPrimary(id); + if (!existing) return { value: null }; + const changes = changesFor(existing); + if (!changes) return { value: existing }; + const nextRevision = existing.siteDataRevision + 1; + const statement = await updateStatement( + id, + toRawInput({ + ...existing, + ...changes, + siteDataRevision: nextRevision, + }), + { args: [existing.siteDataRevision], sql: "site_data_revision = ?" }, + ); + const stored = await queryOne(statement.sql, statement.args); + if (stored) { + return { value: rowToBuiltSite(await rawBuiltSitesTable.fromDb(stored)) }; + } + return null; + }); +}; + /** * CRUD-compatible table adapter that presents BuiltSite (with individual fields) * while storing data as an encrypted blob underneath. @@ -469,11 +161,7 @@ export const builtSitesCrudTable: Table = { findAll: (): Promise => builtSites.getAll(), - findById: async (id: InValue): Promise => { - const row = await rawBuiltSitesTable.findById(id); - if (!row) return null; - return rowToBuiltSite(row); - }, + findById: findBuiltSiteById, findByIds: async (ids: InValue[]): Promise<(BuiltSite | null)[]> => (await rawBuiltSitesTable.findByIds(ids)).map((row) => @@ -527,15 +215,7 @@ export const builtSitesCrudTable: Table = { update: async ( id: InValue, input: Partial, - ): Promise => { - const existing = await builtSitesCrudTable.findById(id); - if (!existing) return null; - const row = (await builtSites.table.update( - id, - toRawInput({ ...existing, ...input }), - )) as BuiltSiteRow; - return rowToBuiltSite(row); - }, + ): Promise => updateBuiltSite(id, () => input), }; /** Normalize a site's bunny URL to its absolute origin — scheme + host only, @@ -551,41 +231,8 @@ export const siteBaseUrl = (siteUrl: string): string => { return new URL(withScheme).origin; }; -/** - * Atomically claim the least-recently-poked built site and return its id and - * bunny URL — the scheduler pokes it to trigger its prune. A single UPDATE picks - * the row via its WHERE subquery and stamps `last_pruned` in the same statement, - * so two overlapping cron pokes can't both grab the same site: SQLite serialises - * the writes and the second claim sees the first's fresh stamp, stepping on to - * the next site. `last_pruned` empty ('') sorts first, so never-poked sites go - * before any dated one and the master walks every site in round-robin order. - * The stamp lands before the caller pokes the site, so a slow or failing site - * doesn't stall the rotation. Returns null when there are no built sites. - */ -export const claimNextBuiltSiteForPrune = async (): Promise<{ - id: number; - siteUrl: string; -} | null> => { - const row = await queryOne<{ - id: number; - site_data: EnvKeyEncrypted; - }>( - `UPDATE built_sites AS builtSite SET last_pruned = ? - WHERE builtSite.id = ( - SELECT candidate.id FROM built_sites AS candidate - ORDER BY candidate.last_pruned ASC, candidate.id ASC - LIMIT 1 - ) - RETURNING id, site_data`, - [nowIso()], - ); - if (!row) return null; - const siteUrl = parseSiteDataBlob(await decrypt(row.site_data)).u; - return { id: row.id, siteUrl }; -}; - /** Insert a new built site record */ -export const insertBuiltSite = ( +export const insertBuiltSite = async ( name: string, siteUrl: string, dbUrl = "", @@ -595,8 +242,9 @@ export const insertBuiltSite = ( updates: UpdateTier = DEFAULT_UPDATE_TIER, hostingProvider: HostingProvider = "bunny", dbProvider: DbProvider = "bunny", + scheduledTaskKey: string | null = null, ): Promise => - builtSites.table.insert( + await builtSites.table.insert( toRawInput({ assignable, dbProvider, @@ -605,6 +253,7 @@ export const insertBuiltSite = ( hostingId, hostingProvider, name, + scheduledTaskKey, siteUrl, updates, }), @@ -630,28 +279,17 @@ export const hasAssignedBuiltSite = rowExistsForIdList( AND assigned_listing_id IN (${listingIdPlaceholders}) LIMIT 1`, ); -const withBuiltSiteForUpdate = async ( - siteId: number, - update: (existing: BuiltSite) => Promise, -): Promise => { - const existing = await builtSitesCrudTable.findById(siteId); - return existing ? update(existing) : null; -}; - /** Assign a built site to an attendee/listing — sets assignable=0 and stores IDs */ export const assignBuiltSite = ( siteId: number, attendeeId: number, listingId: number, ): Promise => - withBuiltSiteForUpdate(siteId, async () => { - const row = (await builtSites.table.update(siteId, { - assignable: 0, - assignedAttendeeId: attendeeId, - assignedListingId: listingId, - })) as BuiltSiteRow; - return rowToBuiltSite(row); - }); + updateBuiltSite(siteId, () => ({ + assignable: false, + assignedAttendeeId: attendeeId, + assignedListingId: listingId, + })); /** Look up a built site by renewal token index (HMAC blind index) */ export const getBuiltSiteByRenewalTokenIndex = async ( @@ -675,19 +313,12 @@ export const updateBuiltSiteRenewalState = ( renewalToken?: string; }, ): Promise => - withBuiltSiteForUpdate(siteId, async (existing) => { - const token = updates.renewalToken ?? existing.renewalToken ?? undefined; - const row = (await builtSites.table.update(siteId, { - siteData: buildSiteDataBlobFromInput({ - ...existing, - renewalToken: token, - }), - ...(updates.renewalTokenIndex !== undefined - ? { renewalTokenIndex: updates.renewalTokenIndex } - : {}), - ...(updates.readOnlyFrom !== undefined - ? { readOnlyFrom: updates.readOnlyFrom } - : {}), - })) as BuiltSiteRow; - return rowToBuiltSite(row); - }); + updateBuiltSite(siteId, (existing) => ({ + renewalToken: updates.renewalToken ?? existing.renewalToken, + ...(updates.renewalTokenIndex !== undefined + ? { renewalTokenIndex: updates.renewalTokenIndex } + : {}), + ...(updates.readOnlyFrom !== undefined + ? { readOnlyFrom: updates.readOnlyFrom } + : {}), + })); diff --git a/src/shared/db/built-sites/blob.ts b/src/shared/db/built-sites/blob.ts new file mode 100644 index 000000000..1bfb561e4 --- /dev/null +++ b/src/shared/db/built-sites/blob.ts @@ -0,0 +1,75 @@ +import * as v from "valibot"; +import { isScheduledTaskKey } from "#shared/scheduled-keys.ts"; +import { defineStoredJson } from "#shared/validation/stored-json.ts"; +import { builtSiteBlobColumns } from "./fields.ts"; +import type { + BuiltSiteBlobFields, + BuiltSiteBlobInput, + DbProvider, + HostingProvider, +} from "./types.ts"; + +const SITE_DATA_BLOB_VERSION = 2; + +export interface SiteDataBlob { + d?: string; + dp?: DbProvider; + hp?: HostingProvider; + n: string; + rt?: string; + s?: string; + sk?: string; + t?: string; + u: string; + v: 1 | typeof SITE_DATA_BLOB_VERSION; +} + +const siteDataFields = { + d: v.optional(v.string()), + dp: v.optional(v.picklist(["bunny", "turso"])), + hp: v.optional(v.picklist(["bunny", "deno"])), + n: v.string(), + rt: v.optional(v.string()), + s: v.optional(v.string()), + t: v.optional(v.string()), + u: v.string(), +}; + +const SiteDataBlobSchema = v.variant("v", [ + v.strictObject({ ...siteDataFields, v: v.literal(1) }), + v.strictObject({ + ...siteDataFields, + sk: v.optional(v.pipe(v.string(), v.check(isScheduledTaskKey))), + v: v.literal(SITE_DATA_BLOB_VERSION), + }), +]); +const siteDataJson = defineStoredJson(SiteDataBlobSchema); + +export const buildSiteDataBlobFromInput = ( + input: Partial, +): string => { + const blob = Object.fromEntries([ + ["v", SITE_DATA_BLOB_VERSION], + ...builtSiteBlobColumns.flatMap((column) => { + const key = column.siteKey as keyof BuiltSiteBlobInput; + const value = ( + Object.hasOwn(input, key) ? input[key] : column.defaultValue + ) as string | null; + return column.required || value ? [[column.blobKey, value]] : []; + }), + ]); + return siteDataJson.write(blob, "built_sites.site_data"); +}; + +export const blobToSiteFields = (blob: SiteDataBlob): BuiltSiteBlobFields => + Object.fromEntries( + builtSiteBlobColumns.map((column) => [ + column.siteKey, + column.required + ? blob[column.blobKey as keyof SiteDataBlob] + : (blob[column.blobKey as keyof SiteDataBlob] ?? column.defaultValue), + ]), + ) as BuiltSiteBlobFields; + +export const parseSiteDataBlob = (json: string): SiteDataBlob => + siteDataJson.read(json, "built_sites.site_data") as SiteDataBlob; diff --git a/src/shared/db/built-sites/fields.ts b/src/shared/db/built-sites/fields.ts new file mode 100644 index 000000000..dc2adbea2 --- /dev/null +++ b/src/shared/db/built-sites/fields.ts @@ -0,0 +1,222 @@ +import type { InValue } from "@libsql/client"; +import type { ColumnDef, TableSchema } from "#shared/db/table.ts"; +import { col } from "#shared/db/table.ts"; +import { nowIso } from "#shared/now.ts"; +import { + type BuiltSite, + type BuiltSiteFormInput, + type BuiltSitePlainFields, + type BuiltSitePlainInput, + type BuiltSiteRow, + type DbProvider, + DEFAULT_UPDATE_TIER, + type HostingProvider, + type UpdateTier, +} from "./types.ts"; + +export const idCol = col.generated(); +export const createdCol = col.withDefault(() => nowIso()); + +const assignableCol = {} as ColumnDef; +const nullCol = col.withDefault(() => null); +const nullStrCol = col.withDefault(() => null); +const passthrough = (value: T): T => value; +const nullable = (value: T | null): T | null => value ?? null; + +export const builtSitePlainColumns = [ + { + dbKey: "assignable", + formDefault: false, + fromRow: (value: number): boolean => Boolean(value), + schema: assignableCol, + siteKey: "assignable", + toInput: (value: boolean): number => (value ? 1 : 0), + }, + { + dbKey: "assigned_attendee_id", + fromRow: nullable, + schema: nullCol, + siteKey: "assignedAttendeeId", + toInput: nullable, + }, + { + dbKey: "assigned_listing_id", + fromRow: nullable, + schema: nullCol, + siteKey: "assignedListingId", + toInput: nullable, + }, + { + dbKey: "read_only_from", + fromRow: passthrough, + schema: col.withDefault(() => ""), + siteKey: "readOnlyFrom", + toInput: passthrough, + }, + { + dbKey: "renewal_token_index", + fromRow: nullable, + schema: nullStrCol, + siteKey: "renewalTokenIndex", + toInput: nullable, + }, + { + dbKey: "site_data_revision", + fromRow: passthrough, + schema: col.withDefault(() => 0), + siteKey: "siteDataRevision", + toInput: passthrough, + }, + { + dbKey: "updates", + formDefault: DEFAULT_UPDATE_TIER, + fromRow: passthrough, + schema: col.withDefault(() => DEFAULT_UPDATE_TIER), + siteKey: "updates", + toInput: passthrough, + }, +] as const; + +export type BuiltSitePlainColumn = (typeof builtSitePlainColumns)[number]; + +const crudSchemaFor = ( + columns: readonly Column[], +): Pick, Column["siteKey"]> => + Object.fromEntries(columns.map(({ siteKey }) => [siteKey, {}])) as Pick< + TableSchema, + Column["siteKey"] + >; + +export const builtSitePlainSchema = Object.fromEntries( + builtSitePlainColumns.map(({ dbKey, schema }) => [dbKey, schema]), +) as Pick, BuiltSitePlainColumn["dbKey"]>; + +export const builtSiteCrudPlainSchema = crudSchemaFor(builtSitePlainColumns); + +export const builtSiteBlobColumns = [ + { + blobKey: "n", + defaultValue: "", + formDbKey: "name", + required: true, + siteKey: "name", + }, + { + blobKey: "u", + defaultValue: "", + formDbKey: "site_url", + required: true, + siteKey: "siteUrl", + }, + { + blobKey: "d", + defaultValue: "", + formDbKey: "db_url", + required: false, + siteKey: "dbUrl", + }, + { + blobKey: "t", + defaultValue: "", + formDbKey: "db_token", + required: false, + siteKey: "dbToken", + }, + { + blobKey: "s", + defaultValue: "", + formDbKey: "hosting_id", + required: false, + siteKey: "hostingId", + }, + { + blobKey: "hp", + defaultValue: "bunny" as HostingProvider, + formDbKey: "hosting_provider", + required: false, + siteKey: "hostingProvider", + }, + { + blobKey: "dp", + defaultValue: "bunny" as DbProvider, + formDbKey: "db_provider", + required: false, + siteKey: "dbProvider", + }, + { + blobKey: "rt", + defaultValue: null, + required: false, + siteKey: "renewalToken", + }, + { + blobKey: "sk", + defaultValue: null, + required: false, + siteKey: "scheduledTaskKey", + }, +] as const; + +type BuiltSiteBlobColumn = (typeof builtSiteBlobColumns)[number]; +export const builtSiteCrudBlobSchema = + crudSchemaFor(builtSiteBlobColumns); + +type BuiltSiteFormMapping = { + dbKey: string; + defaultValue: boolean | string; + siteKey: keyof BuiltSiteFormInput; +}; + +export const builtSiteFormMappings: BuiltSiteFormMapping[] = [ + ...builtSitePlainColumns.flatMap((column) => + "formDefault" in column + ? [ + { + dbKey: column.dbKey, + defaultValue: column.formDefault, + siteKey: column.siteKey, + }, + ] + : [], + ), + ...builtSiteBlobColumns.flatMap((column) => + "formDbKey" in column + ? [ + { + dbKey: column.formDbKey, + defaultValue: column.defaultValue, + siteKey: column.siteKey, + }, + ] + : [], + ), +]; + +export const builtSiteInputKeyMap = Object.fromEntries( + builtSiteFormMappings.map(({ dbKey, siteKey }) => [dbKey, siteKey]), +) as Record; + +export const emptyBuiltSiteFormInput = (): BuiltSiteFormInput => + Object.fromEntries( + builtSiteFormMappings.map(({ defaultValue, siteKey }) => [ + siteKey, + defaultValue, + ]), + ) as BuiltSiteFormInput; + +export const mapPlainFields = ( + input: Partial, + key: Key, +): Partial> => + Object.fromEntries( + builtSitePlainColumns.flatMap((column) => { + if (!Object.hasOwn(input, column.siteKey)) return []; + const value = input[column.siteKey] as never; + return [[column[key], column.toInput(value)]]; + }), + ) as Partial>; + +export const plainSiteInput = ( + input: Partial, +): Partial => + mapPlainFields(input, "siteKey") as Partial; diff --git a/src/shared/db/built-sites/types.ts b/src/shared/db/built-sites/types.ts new file mode 100644 index 000000000..39cafac0b --- /dev/null +++ b/src/shared/db/built-sites/types.ts @@ -0,0 +1,110 @@ +import * as v from "valibot"; +import { guardFor } from "#shared/validation/guard.ts"; + +/** Release channels ordered from most to least eager. */ +export const UPDATE_TIERS = ["alpha", "beta", "release"] as const; +export const UpdateTierSchema = v.picklist(UPDATE_TIERS); +export type UpdateTier = v.InferOutput; +export const DEFAULT_UPDATE_TIER: UpdateTier = "release"; +export const isUpdateTier = guardFor(UpdateTierSchema); + +export const siteAcceptsDeployTier = ( + siteTier: UpdateTier, + deployTier: UpdateTier, +): boolean => + UPDATE_TIERS.indexOf(siteTier) <= UPDATE_TIERS.indexOf(deployTier); + +export type HostingProvider = "bunny" | "deno"; +export type DbProvider = "bunny" | "turso"; + +export const providerOrBunny = < + TProvider extends Exclude, +>( + value: string | null | undefined, + provider: TProvider, +): "bunny" | TProvider => (value === provider ? provider : "bunny"); + +export interface BuiltSiteRow { + assignable: number; + assigned_attendee_id: number | null; + assigned_listing_id: number | null; + created: string; + id: number; + read_only_from: string; + renewal_token_index: string | null; + site_data: string; + site_data_revision: number; + updates: UpdateTier; +} + +export type BuiltSitePlainInput = { + assignable?: number; + assignedAttendeeId?: number | null; + assignedListingId?: number | null; + readOnlyFrom?: string; + renewalTokenIndex?: string | null; + updates?: UpdateTier; +}; + +export type BuiltSiteInput = BuiltSitePlainInput & { siteData: string }; + +export interface BuiltSite { + assignable: boolean; + assignedAttendeeId: number | null; + assignedListingId: number | null; + created: string; + dbProvider: DbProvider; + dbToken: string; + dbUrl: string; + hostingId: string; + hostingProvider: HostingProvider; + id: number; + name: string; + readOnlyFrom: string; + renewalToken: string | null; + renewalTokenIndex: string | null; + scheduledTaskKey: string | null; + siteDataRevision: number; + siteUrl: string; + updates: UpdateTier; +} + +export type BuiltSiteFormInput = Pick< + BuiltSite, + "name" | "siteUrl" | "dbUrl" | "dbToken" | "hostingId" | "assignable" +> & { + updates?: UpdateTier; + hostingProvider?: HostingProvider; + dbProvider?: DbProvider; +}; + +export type BuiltSitePlainFields = Pick< + BuiltSite, + | "assignable" + | "assignedAttendeeId" + | "assignedListingId" + | "readOnlyFrom" + | "renewalTokenIndex" + | "siteDataRevision" + | "updates" +>; + +export type BuiltSiteBlobFields = Pick< + BuiltSite, + | "hostingId" + | "siteUrl" + | "dbToken" + | "dbUrl" + | "name" + | "renewalToken" + | "hostingProvider" + | "dbProvider" + | "scheduledTaskKey" +>; + +export type BuiltSiteBlobInput = Omit & { + renewalToken?: string | null | undefined; +}; + +export type BuiltSiteUpdate = Partial & + Partial; diff --git a/src/shared/db/login-attempts.ts b/src/shared/db/login-attempts.ts index 3483db973..1e5035de5 100644 --- a/src/shared/db/login-attempts.ts +++ b/src/shared/db/login-attempts.ts @@ -5,7 +5,7 @@ * abuse-prone entry points (e.g. public booking). Each call site namespaces its * counters with a `prefix` so they never collide — a booking flood can't lock * anyone out of logging in, and vice versa. Rows whose lockout has expired are - * removed on the next check and by pruneLoginAttempts; counter-only rows (no + * removed on the next check and by database pruning; counter-only rows (no * lockout) are left to be overwritten by the next attempt from that IP. */ diff --git a/src/shared/db/migrations/2026-06-19_built_sites_last_pruned.ts b/src/shared/db/migrations/2026-06-19_built_sites_last_pruned.ts index c16b6f9e0..3ff2ba59f 100644 --- a/src/shared/db/migrations/2026-06-19_built_sites_last_pruned.ts +++ b/src/shared/db/migrations/2026-06-19_built_sites_last_pruned.ts @@ -2,8 +2,6 @@ import { schemaMigration } from "./define.ts"; export default schemaMigration( "2026-06-19_built_sites_last_pruned", - "Add a last_pruned column to built_sites so the scheduled-tasks endpoint can forward a prune to the least-recently-pruned built site and walk every site at a steady pace", - { - columns: { built_sites: ["last_pruned"] }, - }, + "Historical built-site prune marker.", + {}, ); diff --git a/src/shared/db/migrations/2026-07-18_drop_built_sites_last_pruned.ts b/src/shared/db/migrations/2026-07-18_drop_built_sites_last_pruned.ts new file mode 100644 index 000000000..329bad760 --- /dev/null +++ b/src/shared/db/migrations/2026-07-18_drop_built_sites_last_pruned.ts @@ -0,0 +1,7 @@ +import { columnDropMigration } from "./define.ts"; + +export default columnDropMigration( + "2026-07-18_drop_built_sites_last_pruned", + "built_sites", + "Remove the unused built-site prune marker.", +); diff --git a/src/shared/db/migrations/2026-07-18_maintenance_tasks.ts b/src/shared/db/migrations/2026-07-18_maintenance_tasks.ts new file mode 100644 index 000000000..f3b51ac4b --- /dev/null +++ b/src/shared/db/migrations/2026-07-18_maintenance_tasks.ts @@ -0,0 +1,90 @@ +import { executeBatch, queryAll } from "#shared/db/client.ts"; +import { + ACTIVITY_LOG_BACKFILL_INTERVAL_MS, + PRUNE_INTERVAL_MS, +} from "#shared/limits.ts"; +import { schemaMigration } from "./define.ts"; + +const PRUNE_MARKERS = [ + "last_pruned_addresses", + "last_pruned_contacts", + "last_pruned_invites", + "last_pruned_logins", + "last_pruned_orphans", + "last_pruned_payments", + "last_pruned_sessions", + "last_pruned_strings", + "last_pruned_sumup", + "last_pruned_tokens", +] as const; +const ACTIVITY_MARKERS = [ + "activity_log_backfill_done", + "last_activity_log_backfill", +] as const; +const OLD_MARKERS = [...PRUNE_MARKERS, ...ACTIVITY_MARKERS]; + +type MarkerRow = { key: string; value: string }; + +const markerTime = ( + rows: MarkerRow[], + keys: readonly string[], + intervalMs: number, + fallback: number, +): number => { + const times = rows + .filter((row) => keys.includes(row.key)) + .map((row) => Number(row.value)) + .filter((value) => Number.isFinite(value) && value > 0); + return times.length === 0 ? fallback : Math.min(...times) + intervalMs; +}; + +export default schemaMigration( + "2026-07-18_maintenance_tasks", + "Add durable scheduled maintenance task claims.", + { + columns: { built_sites: ["site_data_revision"] }, + indexes: ["idx_maintenance_tasks_due"], + newTables: ["maintenance_tasks"], + }, + async () => { + const rows = await queryAll( + `SELECT key, value FROM settings + WHERE key IN (${OLD_MARKERS.map(() => "?").join(", ")})`, + [...OLD_MARKERS], + ); + const now = Date.now(); + const done = rows.some( + (row) => row.key === "activity_log_backfill_done" && row.value === "true", + ); + await executeBatch([ + { + args: [ + "database_pruning", + markerTime(rows, PRUNE_MARKERS, PRUNE_INTERVAL_MS, now), + ], + sql: "INSERT OR IGNORE INTO maintenance_tasks (name, next_run_at) VALUES (?, ?)", + }, + ...(done + ? [] + : [ + { + args: [ + "activity_log_backfill", + markerTime( + rows, + ["last_activity_log_backfill"], + ACTIVITY_LOG_BACKFILL_INTERVAL_MS, + now, + ), + ], + sql: "INSERT OR IGNORE INTO maintenance_tasks (name, next_run_at) VALUES (?, ?)", + }, + ]), + { + args: [...OLD_MARKERS], + sql: `DELETE FROM settings + WHERE key IN (${OLD_MARKERS.map(() => "?").join(", ")})`, + }, + ]); + }, +); diff --git a/src/shared/db/migrations/2026-07-19_maintenance_checkpoint.ts b/src/shared/db/migrations/2026-07-19_maintenance_checkpoint.ts new file mode 100644 index 000000000..bb743a164 --- /dev/null +++ b/src/shared/db/migrations/2026-07-19_maintenance_checkpoint.ts @@ -0,0 +1,7 @@ +import { schemaMigration } from "./define.ts"; + +export default schemaMigration( + "2026-07-19_maintenance_checkpoint", + "Remember where bounded maintenance scans should continue.", + { columns: { maintenance_tasks: ["checkpoint"] } }, +); diff --git a/src/shared/db/migrations/registry.ts b/src/shared/db/migrations/registry.ts index 64840b785..2765da409 100644 --- a/src/shared/db/migrations/registry.ts +++ b/src/shared/db/migrations/registry.ts @@ -357,6 +357,21 @@ export const MIGRATION_REGISTRY: MigrationRegistryEntry[] = [ "2026-07-16_drop_checkout_stage_revisions", () => import("./2026-07-16_drop_checkout_stage_revisions.ts"), ), + // Move local background work onto durable, token-fenced task claims. + entry( + "2026-07-18_maintenance_tasks", + () => import("./2026-07-18_maintenance_tasks.ts"), + ), + // Fleet fan-out is gone, so built sites no longer need a rotation marker. + entry( + "2026-07-18_drop_built_sites_last_pruned", + () => import("./2026-07-18_drop_built_sites_last_pruned.ts"), + ), + // Let bounded maintenance scans continue without rescanning earlier rows. + entry( + "2026-07-19_maintenance_checkpoint", + () => import("./2026-07-19_maintenance_checkpoint.ts"), + ), ]; /* jscpd:ignore-end */ diff --git a/src/shared/db/migrations/schema/tables-core.ts b/src/shared/db/migrations/schema/tables-core.ts index 6f45bcd06..36f7d8037 100644 --- a/src/shared/db/migrations/schema/tables-core.ts +++ b/src/shared/db/migrations/schema/tables-core.ts @@ -26,6 +26,30 @@ export const coreTables: [name: string, table: Table][] = [ }, ], + [ + "maintenance_tasks", + { + columns: [ + ["name", "TEXT PRIMARY KEY"], + ["checkpoint", "TEXT"], + ["next_run_at", "INTEGER NOT NULL"], + ["lease_token", "TEXT"], + ["lease_expires_at", "INTEGER"], + ["last_started_at", "INTEGER"], + [ + "last_finished_at", + "INTEGER CHECK ((lease_token IS NULL) = (lease_expires_at IS NULL))", + ], + ], + indexes: [ + { + columns: ["next_run_at", "lease_expires_at", "name"], + name: "idx_maintenance_tasks_due", + }, + ], + }, + ], + [ "listings", { diff --git a/src/shared/db/migrations/schema/tables-questions.ts b/src/shared/db/migrations/schema/tables-questions.ts index 9ae90216a..92929bdb3 100644 --- a/src/shared/db/migrations/schema/tables-questions.ts +++ b/src/shared/db/migrations/schema/tables-questions.ts @@ -156,11 +156,7 @@ export const questionTables: [name: string, table: Table][] = [ createdColumn, ["renewal_token_index", "TEXT DEFAULT NULL"], ["read_only_from", "TEXT NOT NULL DEFAULT ''"], - // ISO timestamp of the last time the master poked this site to trigger - // its prune; '' (never) sorts first so the master walks every site in - // round-robin order. Operational metadata, not PII, so it lives outside - // the encrypted site_data blob. - ["last_pruned", "TEXT NOT NULL DEFAULT ''"], + ["site_data_revision", "INTEGER NOT NULL DEFAULT 0"], // Release channel this site opts into: 'alpha' takes every deploy, // 'beta' takes beta + release, 'release' only stable releases. The // upgrade workflow passes the tier it is publishing and the master diff --git a/src/shared/db/migrations/schema/version.ts b/src/shared/db/migrations/schema/version.ts index 723fd0a9e..e9999fd09 100644 --- a/src/shared/db/migrations/schema/version.ts +++ b/src/shared/db/migrations/schema/version.ts @@ -1,7 +1,7 @@ /** Schema version label and the migrations bookkeeping table name. */ export const LATEST_UPDATE = - "Unify admin feature visibility, enforce attendee status integrity, and add checkout stage storage."; + "Add secure local scheduled maintenance and durable task claims."; export const SCHEMA_MIGRATIONS_TABLE = "schema_migrations"; export const LATEST_DB_UPDATE_KEY = "latest_db_update"; diff --git a/src/shared/db/prune.ts b/src/shared/db/prune.ts index 2b1b1b967..56f0bbec1 100644 --- a/src/shared/db/prune.ts +++ b/src/shared/db/prune.ts @@ -1,50 +1,17 @@ -/** - * Database pruning — delete rows from tables that grow unboundedly - * but whose contents are only useful for a short window. - * - * Tables pruned: - * - processed_payments: payment-session replay records. Failed rows and rows - * that no longer hold a useful refund reference are pruned after the retry - * window; unrefunded attendees keep their stored charge references. - * - sessions: once expires < now, the row is dead. Small grace window so - * expired-but-present sessions have a recognisable identity briefly. - * - login_attempts: rows with an expired lockout are dead. (Rows with NULL - * locked_until are left alone: they represent in-progress attempt counts - * and have no timestamp we can key off.) - * - contact_preferences: opaque per-contact recognition/contact-history rows. - * `last_activity` is bumped on booking and outreach; pruning subscribed rows - * bounds table growth and makes returning-customer recognition - * recency-bounded. Unsubscribed rows are suppression records and are kept. - * - strings: owner-key-encrypted free-text answer values. The attendee_answers - * triggers maintain each row's reference count but never delete (a pending - * paid checkout can hold a `string_id` in its metadata before finalizing), so - * this age-based prune is the sole cleanup for unused rows. - * - address_cache: encrypted address-lookup results. Reads already ignore rows - * older than ADDRESS_CACHE_DAYS, so this prune just reclaims the storage. - * - attendees (orphaned only): rows with no surviving listing booking, older - * than the age chosen on the Privacy page. Opt-in — only scheduled while - * `auto_purge_orphans` is on (see PRUNE_TASKS). - * - users (expired invites only): un-activated invited users whose invite has - * expired. Removing the row drops its invite_wrapped_data_key handoff — a - * DATA_KEY wrapped under the emailed invite code — so an intercepted but - * expired invite link can no longer unwrap it from a database dump. - * - * The scheduler is fire-and-forget via `addPendingWork` from the request - * handler. Each table has its own `last_pruned_*` timestamp; a table is - * pruned only when PRUNE_INTERVAL_MS has elapsed since its last run. - */ +/** Fixed-size database pruning used by the maintenance task and owner actions. */ -import { execute } from "#shared/db/client.ts"; -import { purgeOrphanedAttendees } from "#shared/db/orphan-attendees.ts"; -import { writeRawBatch } from "#shared/db/settings/raw-writes.ts"; -import { setSnapshotField } from "#shared/db/settings/snapshot.ts"; +import { decrypt } from "#shared/crypto/encryption.ts"; +import { addressCachePruneStatement } from "#shared/db/address-cache.ts"; +import { attendeeDependentDeleteStatements } from "#shared/db/attendees/delete.ts"; +import { + executeBatchWithResults, + queryAll, + type SqlStatement, +} from "#shared/db/client.ts"; import { settings } from "#shared/db/settings.ts"; -import { pruneExpiredInvites } from "#shared/db/users.ts"; -import { taskIsDue } from "#shared/interval-gate.ts"; import { - ADDRESS_CACHE_MS, + MAINTENANCE_PRUNE_BATCH, PRUNE_CONTACTS_RETENTION_MS, - PRUNE_INTERVAL_MS, PRUNE_LOGINS_RETENTION_MS, PRUNE_PAYMENTS_RETENTION_MS, PRUNE_SESSIONS_RETENTION_MS, @@ -53,257 +20,211 @@ import { PRUNE_UNUSED_STRINGS_RETENTION_MS, } from "#shared/limits.ts"; import { logDebug } from "#shared/logger.ts"; -import { nowMs } from "#shared/now.ts"; +import { now, nowMs } from "#shared/now.ts"; import { orphanRetentionCutoffIso } from "#shared/orphan-retention.ts"; -import { CONFIG_KEYS } from "#shared/settings/keys.ts"; +import type { User } from "#shared/types.ts"; -/** - * Build a pruner that deletes rows older than `retentionMs`, binding an - * ISO-timestamp cutoff to the single `?` placeholder in `sql`. - */ -const isoAgePruner = - (sql: string, retentionMs: number) => async (): Promise => { - const cutoffIso = new Date(nowMs() - retentionMs).toISOString(); - const result = await execute(sql, [cutoffIso]); - return result.rowsAffected; - }; +type PruneStatement = SqlStatement; -/** - * Delete old processed_payments rows only when they cannot help a future admin - * refund: terminal failures, finalized rows with no stored charge reference, - * rows whose attendee is gone, and rows whose attendee already has refund_cash in - * the ledger. Unresolved reservations stay for deleteAllStaleReservations. - */ -export const prunePayments = isoAgePruner( - `DELETE FROM processed_payments AS payment - WHERE payment.processed_at < ? - AND ( - payment.failure_data != '' - OR ( - payment.attendee_id IS NOT NULL - AND ( - payment.payment_reference = '' - OR NOT EXISTS ( - SELECT 1 FROM attendees AS attendee - WHERE attendee.id = payment.attendee_id - ) - OR EXISTS ( - SELECT 1 FROM transfers AS transfer - WHERE transfer.kind = 'refund_cash' - AND transfer.source_type = 'attendee' - AND transfer.source_id = CAST(payment.attendee_id AS TEXT) - ) - ) - ) - )`, - PRUNE_PAYMENTS_RETENTION_MS, -); +export interface DatabasePruningResult { + checkpoint: string | null; + fullBatch: boolean; +} -/** - * Delete SumUp checkout staging rows older than their (short) retention. - * The row carries encrypted PII and is only needed between checkout creation - * and payment completion — SumUp checkouts expire after 30 minutes and - * webhook retries stop after 2 hours, so 24h retention is already generous. - */ -export const pruneSumupCheckouts = isoAgePruner( - "DELETE FROM sumup_checkouts WHERE created_at < ?", - PRUNE_SUMUP_RETENTION_MS, -); +const boundedDelete = ( + table: string, + where: string, + args: (number | string)[], +): PruneStatement => ({ + args: [...args, MAINTENANCE_PRUNE_BATCH], + sql: `DELETE FROM ${table} + WHERE rowid IN ( + SELECT rowid FROM ${table} WHERE ${where} + ORDER BY rowid LIMIT ? + )`, +}); -/** Delete unreferenced encrypted free-text strings older than retention. */ -export const pruneUnusedStrings = isoAgePruner( - "DELETE FROM strings WHERE used_count = 0 AND created < ?", - PRUNE_UNUSED_STRINGS_RETENTION_MS, -); +const isoCutoff = (retentionMs: number): string => + new Date(nowMs() - retentionMs).toISOString(); -/** Delete cached address-lookup results older than ADDRESS_CACHE_DAYS. - * Reads in address-cache.ts filter on the same cutoff, so this prune is pure - * housekeeping — an expired row is already unservable before it is deleted. */ -export const pruneAddressCache = isoAgePruner( - "DELETE FROM address_cache WHERE created < ?", - ADDRESS_CACHE_MS, -); +const paymentStatement = (): PruneStatement => ({ + args: [isoCutoff(PRUNE_PAYMENTS_RETENTION_MS), MAINTENANCE_PRUNE_BATCH], + sql: `DELETE FROM processed_payments + WHERE rowid IN ( + SELECT payment.rowid + FROM processed_payments AS payment + WHERE payment.processed_at < ? + AND ( + payment.failure_data != '' + OR ( + payment.attendee_id IS NOT NULL + AND ( + payment.payment_reference = '' + OR NOT EXISTS ( + SELECT 1 FROM attendees AS attendee + WHERE attendee.id = payment.attendee_id + ) + OR EXISTS ( + SELECT 1 FROM transfers AS transfer + WHERE transfer.kind = 'refund_cash' + AND transfer.source_type = 'attendee' + AND transfer.source_id = CAST(payment.attendee_id AS TEXT) + ) + ) + ) + ) + ORDER BY payment.rowid LIMIT ? + )`, +}); -/** - * Delete sessions whose `expires` is older than (now - retention window). - * Uses millisecond-epoch numeric comparison (same format as the column). - */ -export const pruneSessions = async (): Promise => { - const cutoffMs = nowMs() - PRUNE_SESSIONS_RETENTION_MS; - const result = await execute("DELETE FROM sessions WHERE expires < ?", [ - cutoffMs, - ]); - return result.rowsAffected; -}; +const pruneStatements = (): PruneStatement[] => [ + paymentStatement(), + boundedDelete("sumup_checkouts", "created_at < ?", [ + isoCutoff(PRUNE_SUMUP_RETENTION_MS), + ]), + boundedDelete("strings", "used_count = 0 AND created < ?", [ + isoCutoff(PRUNE_UNUSED_STRINGS_RETENTION_MS), + ]), + addressCachePruneStatement(), + boundedDelete("sessions", "expires < ?", [ + nowMs() - PRUNE_SESSIONS_RETENTION_MS, + ]), + boundedDelete( + "login_attempts", + "locked_until IS NOT NULL AND locked_until < ?", + [nowMs() - PRUNE_LOGINS_RETENTION_MS], + ), + boundedDelete("token_attempts", "last_attempt < ?", [ + nowMs() - PRUNE_TOKENS_RETENTION_MS, + ]), + boundedDelete( + "contact_preferences", + "unsubscribed = 0 AND last_activity < ?", + [nowMs() - PRUNE_CONTACTS_RETENTION_MS], + ), +]; -/** - * Delete login_attempts rows whose lockout expired more than the retention - * window ago. Rows with NULL `locked_until` have no timestamp and are left - * alone (they will be overwritten on the next attempt from that IP). - */ -export const pruneLoginAttempts = async (): Promise => { - const cutoffMs = nowMs() - PRUNE_LOGINS_RETENTION_MS; - const result = await execute( - "DELETE FROM login_attempts WHERE locked_until IS NOT NULL AND locked_until < ?", - [cutoffMs], - ); - return result.rowsAffected; +const ORPHAN_IDS = `SELECT attendee.id + FROM attendees AS attendee + WHERE attendee.created < ? + AND NOT EXISTS ( + SELECT 1 FROM listing_attendees AS booking + WHERE booking.attendee_id = attendee.id + ) + ORDER BY attendee.id LIMIT ?`; + +const orphanStatements = (): PruneStatement[] => { + if (!settings.autoPurgeOrphans) return []; + const args = [ + orphanRetentionCutoffIso(settings.orphanPurgeRetention, nowMs()), + MAINTENANCE_PRUNE_BATCH, + ]; + return [ + ...attendeeDependentDeleteStatements({ args, sql: ORPHAN_IDS }), + { + args, + sql: `DELETE FROM attendees WHERE id IN (${ORPHAN_IDS})`, + }, + ]; }; -/** - * Delete token_attempts rows untouched for longer than the retention window. - * `last_attempt` is set on every failure record, so this covers both - * expired-lockout rows and stale counter-only rows. - */ -export const pruneTokenAttempts = async (): Promise => { - const cutoffMs = nowMs() - PRUNE_TOKENS_RETENTION_MS; - const result = await execute( - "DELETE FROM token_attempts WHERE last_attempt < ?", - [cutoffMs], - ); - return result.rowsAffected; +type InviteCandidate = Pick; + +type InvitePage = { + checkpoint: string | null; + expiredIds: number[]; + hasMore: boolean; }; -/** Delete subscribed contact-preference rows untouched beyond retention. */ -export const pruneContacts = async (): Promise => { - const cutoffMs = nowMs() - PRUNE_CONTACTS_RETENTION_MS; - const result = await execute( - "DELETE FROM contact_preferences WHERE unsubscribed = 0 AND last_activity < ?", - [cutoffMs], - ); - return result.rowsAffected; +const checkpointId = (checkpoint: string | null): number | null => { + if (checkpoint === null) return null; + const id = Number(checkpoint); + if (!Number.isSafeInteger(id) || id < 1) { + throw new Error(`Invalid invite pruning checkpoint: ${checkpoint}`); + } + return id; }; -/** - * Delete orphaned attendees (no surviving listing booking) older than the - * owner-configured age. Unlike the other tasks this is opt-in: it is only added - * to the schedule while `auto_purge_orphans` is on, and the age comes from the - * Privacy page rather than a fixed constant. - */ -export const pruneOrphanAttendees = (): Promise => - purgeOrphanedAttendees( - orphanRetentionCutoffIso(settings.orphanPurgeRetention, nowMs()), +const expiredInvitePage = async ( + checkpoint: string | null, +): Promise => { + const rows = await queryAll( + `SELECT id, invite_expiry FROM users + WHERE wrapped_data_key IS NULL + AND password_hash = '' + AND invite_expiry IS NOT NULL + AND (? IS NULL OR id > ?) + ORDER BY id + LIMIT ?`, + [ + checkpointId(checkpoint), + checkpointId(checkpoint), + MAINTENANCE_PRUNE_BATCH + 1, + ], ); - -type PruneTask = { - field: Parameters[0]; - key: string; - name: string; - lastRaw: string; - run: () => Promise; + const candidates = rows.slice(0, MAINTENANCE_PRUNE_BATCH); + const cutoff = now().getTime(); + const inviteStates = await Promise.all( + candidates.map(async (row) => ({ + expired: new Date(await decrypt(row.invite_expiry!)).getTime() < cutoff, + id: row.id, + })), + ); + const hasMore = rows.length > MAINTENANCE_PRUNE_BATCH; + return { + checkpoint: hasMore ? String(candidates[candidates.length - 1]!.id) : null, + expiredIds: inviteStates + .filter(({ expired }) => expired) + .map(({ id }) => id), + hasMore, + }; }; -const PRUNE_TASKS = (): PruneTask[] => [ - { - field: "last_pruned_payments", - key: CONFIG_KEYS.LAST_PRUNED_PAYMENTS, - lastRaw: settings.lastPrunedPayments, - name: "processed_payments", - run: prunePayments, - }, - { - field: "last_pruned_sumup", - key: CONFIG_KEYS.LAST_PRUNED_SUMUP, - lastRaw: settings.lastPrunedSumup, - name: "sumup_checkouts", - run: pruneSumupCheckouts, - }, - { - field: "last_pruned_strings", - key: CONFIG_KEYS.LAST_PRUNED_STRINGS, - lastRaw: settings.lastPrunedStrings, - name: "strings", - run: pruneUnusedStrings, - }, - { - field: "last_pruned_sessions", - key: CONFIG_KEYS.LAST_PRUNED_SESSIONS, - lastRaw: settings.lastPrunedSessions, - name: "sessions", - run: pruneSessions, - }, - { - field: "last_pruned_logins", - key: CONFIG_KEYS.LAST_PRUNED_LOGINS, - lastRaw: settings.lastPrunedLogins, - name: "login_attempts", - run: pruneLoginAttempts, - }, - { - field: "last_pruned_tokens", - key: CONFIG_KEYS.LAST_PRUNED_TOKENS, - lastRaw: settings.lastPrunedTokens, - name: "token_attempts", - run: pruneTokenAttempts, - }, - { - field: "last_pruned_contacts", - key: CONFIG_KEYS.LAST_PRUNED_CONTACTS, - lastRaw: settings.lastPrunedContacts, - name: "contact_preferences", - run: pruneContacts, - }, - { - field: "last_pruned_addresses", - key: CONFIG_KEYS.LAST_PRUNED_ADDRESSES, - lastRaw: settings.lastPrunedAddresses, - name: "address_cache", - run: pruneAddressCache, - }, - { - field: "last_pruned_invites", - key: CONFIG_KEYS.LAST_PRUNED_INVITES, - lastRaw: settings.lastPrunedInvites, - name: "expired_invites", - run: pruneExpiredInvites, - }, - // Opt-in: scheduled only while the owner leaves automatic orphan purging on. - ...(settings.autoPurgeOrphans - ? [ - { - field: "last_pruned_orphans" as const, - key: CONFIG_KEYS.LAST_PRUNED_ORPHANS, - lastRaw: settings.lastPrunedOrphans, - name: "orphan_attendees", - run: pruneOrphanAttendees, - }, - ] - : []), -]; - -/** - * Run one prune task: write the timestamp first (claims the slot so concurrent - * requests don't double-run), then delete. Errors are caught and logged so - * one failing task can't block the others or surface to the user. - */ -const runTask = async (task: PruneTask): Promise => { - try { - const deleted = await task.run(); - if (deleted > 0) { - logDebug("Prune", `${task.name}: deleted ${deleted} rows`); - } - } catch (e) { - logDebug("Prune", `${task.name} failed: ${String(e)}`); - } +const inviteStatements = (ids: number[]): PruneStatement[] => { + if (ids.length === 0) return []; + const inIds = ids.map(() => "?").join(", "); + const unactivatedIds = `SELECT id FROM users + WHERE id IN (${inIds}) + AND wrapped_data_key IS NULL + AND password_hash = '' + AND invite_expiry IS NOT NULL`; + return [ + { field: "user_id", table: "api_keys" }, + { field: "user_id", table: "sessions" }, + { field: "user_id", table: "user_logistics_agents" }, + { field: "id", table: "users" }, + ].map(({ field, table }) => ({ + args: ids, + sql: `DELETE FROM ${table} WHERE ${field} IN (${unactivatedIds})`, + })); }; -/** - * Run all prune tasks that are due. Safe to call from a fire-and-forget - * context (addPendingWork). Never throws. - */ -export const maybeRunPrunes = async (): Promise => { - const now = nowMs(); - const due = PRUNE_TASKS().filter((t) => - taskIsDue(t.lastRaw, PRUNE_INTERVAL_MS, now), +const lastResultIndexes = (batches: PruneStatement[][]): number[] => + batches.reduce((indexes, batch) => { + const previous = indexes[indexes.length - 1] ?? -1; + indexes.push(previous + batch.length); + return indexes; + }, [] as number[]); + +export const runDatabasePruning = async ( + checkpoint: string | null = null, +): Promise => { + const invitePage = await expiredInvitePage(checkpoint); + const batches = [ + ...pruneStatements().map((statement) => [statement]), + orphanStatements(), + inviteStatements(invitePage.expiredIds), + ].filter((batch) => batch.length > 0); + const results = await executeBatchWithResults(batches.flat()); + const fullBatch = + invitePage.hasMore || + lastResultIndexes(batches).some( + (index) => results[index]!.rowsAffected === MAINTENANCE_PRUNE_BATCH, + ); + const deleted = results.reduce( + (total, result) => total + result.rowsAffected, + 0, ); - if (due.length === 0) return; - const value = String(now); - try { - await writeRawBatch(due.map((task) => [task.key, value] as const)); - } catch (error) { - logDebug("Prune", `marker batch failed: ${String(error)}`); - return; - } - for (const task of due) setSnapshotField(task.field, value); - await Promise.all(due.map(runTask)); + if (deleted > 0) logDebug("Prune", `deleted ${deleted} expired rows`); + return { checkpoint: invitePage.checkpoint, fullBatch }; }; diff --git a/src/shared/db/query-log.ts b/src/shared/db/query-log.ts index f4735ec39..a7e186a99 100644 --- a/src/shared/db/query-log.ts +++ b/src/shared/db/query-log.ts @@ -18,7 +18,10 @@ import { lazyRef, map, pipe, reduce, sort } from "#fp"; import { withLazyLogger } from "#shared/lazy-logger.ts"; import { shouldSuppressDebugLogs } from "#shared/log-settings.ts"; import { createScope } from "#shared/request-scoped.ts"; -import { BUNNY_SUBREQUEST_LIMIT } from "#shared/subrequest-budget.ts"; +import { + BUNNY_SUBREQUEST_LIMIT, + countSubrequest, +} from "#shared/subrequest-budget.ts"; /** A single logged query */ export type QueryLogEntry = { @@ -171,6 +174,7 @@ export const TRANSACTION_ROUNDTRIP_THRESHOLD = 30; * network. Unlike advisory N+1 reporting, this stays hard in production because * Bunny would reject the same request immediately afterwards anyway. */ export const countDatabaseRoundTrip = (operation: string): void => { + countSubrequest("database", operation); const state = queryLogScope.current(); if (!state) return; state.databaseRoundTrips += 1; diff --git a/src/shared/db/retry-write.ts b/src/shared/db/retry-write.ts new file mode 100644 index 000000000..94525ca42 --- /dev/null +++ b/src/shared/db/retry-write.ts @@ -0,0 +1,13 @@ +type WriteResult = { value: T }; + +/** Retry a revision-fenced write when another writer wins the race. */ +export const retryWrite = async ( + failure: string, + write: () => Promise | null>, +): Promise => { + for (let attempt = 0; attempt < 4; attempt++) { + const result = await write(); + if (result !== null) return result.value; + } + throw new Error(failure); +}; diff --git a/src/shared/db/settings.ts b/src/shared/db/settings.ts index 685808bc0..26c18d378 100644 --- a/src/shared/db/settings.ts +++ b/src/shared/db/settings.ts @@ -90,7 +90,7 @@ import { serializeListingDefaults, } from "#shared/listing-defaults.ts"; import { CONFIG_KEYS } from "#shared/settings/keys.ts"; -import { EMAIL_BODY_KEYS, PRUNE_KEYS } from "#shared/settings/registry.ts"; +import { EMAIL_BODY_KEYS } from "#shared/settings/registry.ts"; import type { EmailTemplateFormat, EmailTemplateType, @@ -111,7 +111,6 @@ export { CONFIG_KEYS, EMAIL_BODY_KEYS, getCurrentSettingsVersion, - PRUNE_KEYS, SNAPSHOT_KEYS, }; diff --git a/src/shared/db/users.ts b/src/shared/db/users.ts index b23ebb017..251856d1b 100644 --- a/src/shared/db/users.ts +++ b/src/shared/db/users.ts @@ -378,7 +378,7 @@ export const acceptInvite = async ( * * Single-use: the UPDATE is guarded on `password_hash = ''` (the unactivated * marker — buildUserInsert stores a literal empty string until a password is - * set, and pruneExpiredInvites uses the same marker). So a replay or a race only + * set, and database pruning uses the same marker). So a replay or a race only * affects the row on the first submit; later submits no-op and return false * rather than overwriting the password the first submit set. */ @@ -495,20 +495,3 @@ export const isInviteExpired = async (user: User): Promise => * cannot delete an active account. invite_expiry is encrypted per row, but only * a handful of invites are ever outstanding, so decrypting each is cheap. */ -export const pruneExpiredInvites = async (): Promise => { - const rows = await queryAll>( - "SELECT id, invite_expiry FROM users WHERE wrapped_data_key IS NULL AND password_hash = '' AND invite_expiry IS NOT NULL", - ); - const cutoff = now().getTime(); - let pruned = 0; - for (const row of rows) { - // An unparseable expiry yields NaN, which compares false — such a row is - // left alone rather than deleted on a bad value. - const expiryMs = new Date(await decrypt(row.invite_expiry!)).getTime(); - if (expiryMs < cutoff) { - await deleteUser(row.id); - pruned += 1; - } - } - return pruned; -}; diff --git a/src/shared/deno-deploy-api.ts b/src/shared/deno-deploy-api.ts index b86006073..59d192361 100644 --- a/src/shared/deno-deploy-api.ts +++ b/src/shared/deno-deploy-api.ts @@ -14,8 +14,8 @@ import { } from "#shared/config.ts"; import { fetchText, parseApiError } from "#shared/fetch.ts"; import type { - CreateSiteFn, HostingProviderApi, + PrepareSiteFn, } from "#shared/provider-types.ts"; import { errorResult, okResult, type Result } from "#shared/result.ts"; @@ -34,12 +34,6 @@ interface GetAppResponse { slug: string; } -interface DeploymentResponse { - domains?: string[]; - hostnames?: string[]; - id: string; -} - /** Headers for all Deno Deploy API requests. */ const denoApiHeaders = (): Record => ({ Authorization: `Bearer ${getDenoDeployToken()}`, @@ -96,18 +90,18 @@ const fetchAppEnvVars = async ( * (Re-sending existing secrets risks clearing them: the GET response masks * secret values, so a round-trip GET→merge→PATCH would PATCH with empty values.) */ -const setEnvVarsImpl = async (appId: string, secrets: [string, string][]) => { - const envVarsArray = secrets.map(([key, value]) => ({ - contexts: ["production"], - key, - secret: true, - value, - })); +const envVar = ([key, value]: [string, string]) => ({ + contexts: ["production"], + key, + secret: true, + value, +}); +const setEnvVarsImpl = async (appId: string, secrets: [string, string][]) => { const patchRes = await fetchText( `${DENO_API_BASE}/apps/${encodeURIComponent(appId)}`, { - body: JSON.stringify({ env_vars: envVarsArray }), + body: JSON.stringify({ env_vars: secrets.map(envVar) }), headers: denoApiHeaders(), method: "PATCH", }, @@ -121,12 +115,12 @@ const setEnvVarsImpl = async (appId: string, secrets: [string, string][]) => { * Deploy code to a Deno Deploy app (production deployment). * Returns the primary hostname for the deployment. */ -const deployCodeImpl = async ( - appId: string, - code: string, -): Promise> => { +const deployCodeImpl: HostingProviderApi["publishSite"] = async ( + appId, + code, +) => { const res = await fetchText( - `${DENO_API_BASE}/apps/${encodeURIComponent(appId)}/deployments`, + `${DENO_API_BASE}/apps/${encodeURIComponent(appId)}/deploy`, { body: JSON.stringify({ assets: { @@ -141,13 +135,7 @@ const deployCodeImpl = async ( ); if (!res.ok) return parseApiError(res, "Deploy code"); - - const data: DeploymentResponse = JSON.parse(res.text); - const hostname = data.domains?.[0] ?? data.hostnames?.[0]; - if (!hostname) { - return errorResult("Deploy code failed: no hostname in response"); - } - return okResult(`https://${hostname}`); + return okResult(undefined); }; /** @@ -168,7 +156,7 @@ export const denoDeployApi = { setEnvVars: setEnvVarsImpl, }; -const createDenoSiteImpl: CreateSiteFn = async (name, code, secrets) => { +const prepareDenoSiteImpl: PrepareSiteFn = async (name, _code, secrets) => { const createResult = await denoDeployApi.createApp(slugifyForDeno(name)); if (!createResult.ok) return createResult; const setResult = await denoDeployApi.setEnvVars( @@ -178,21 +166,17 @@ const createDenoSiteImpl: CreateSiteFn = async (name, code, secrets) => { if (!setResult.ok) { return errorResult(`Failed to set secrets: ${setResult.error}`); } - const deployResult = await denoDeployApi.deployCode( - createResult.value.appId, - code, - ); - if (!deployResult.ok) return deployResult; return okResult({ - defaultHostname: deployResult.value, + defaultHostname: `https://${createResult.value.slug}.deno.dev`, hostingId: createResult.value.appId, }); }; export const denoHostingProvider: HostingProviderApi = { configEnvVar: "DENO_DEPLOY_TOKEN", - createSite: createDenoSiteImpl, getSecretNames: (hostingId) => denoDeployApi.getEnvVarNames(hostingId), + prepareSite: prepareDenoSiteImpl, + publishSite: (hostingId, code) => denoDeployApi.deployCode(hostingId, code), setSecrets: (hostingId, secrets) => denoDeployApi.setEnvVars(hostingId, secrets), }; diff --git a/src/shared/fetch.ts b/src/shared/fetch.ts index 929a17c25..651b40cfa 100644 --- a/src/shared/fetch.ts +++ b/src/shared/fetch.ts @@ -9,6 +9,7 @@ import { extendedBy } from "#fp"; import { errorResult, type Result } from "#shared/result.ts"; +import { countExternalSubrequest } from "#shared/subrequest-budget.ts"; /** A fetch result whose body has already been read to a string. */ export type FetchResult = { @@ -58,6 +59,7 @@ export const fetchText = async ( url: string, init?: RequestInit, ): Promise => { + countExternalSubrequest(`fetch ${new URL(url).origin}`); const response = await fetch(url, init); const text = await response.text(); return { diff --git a/src/shared/interval-gate.ts b/src/shared/interval-gate.ts deleted file mode 100644 index 37236aa4f..000000000 --- a/src/shared/interval-gate.ts +++ /dev/null @@ -1,25 +0,0 @@ -/** - * Interval gate for fire-and-forget housekeeping tasks. - * - * The prune tasks and the activity-log backfill both run little and often from - * the request handler, each gated by its own "when did I last run" timestamp so - * it stays inside the edge subrequest budget. They store that timestamp as a - * ms-epoch string setting; this decides whether enough time has passed to run - * the task again. - */ - -import { parsePositiveInt } from "#shared/limits.ts"; - -/** - * True when at least `intervalMs` has passed since the task last ran. - * - * `lastRun` is the stored ms-epoch timestamp of the previous run. An empty or - * unreadable value counts as 0 — "never run", so the task is due right away. - * Pass the current time in so a caller can gate several tasks against one clock - * reading. - */ -export const taskIsDue = ( - lastRun: string, - intervalMs: number, - now: number, -): boolean => now - parsePositiveInt(lastRun, 0) >= intervalMs; diff --git a/src/shared/limits.ts b/src/shared/limits.ts index 50770885b..46a234145 100644 --- a/src/shared/limits.ts +++ b/src/shared/limits.ts @@ -305,7 +305,7 @@ const DAY_MS = 24 * 60 * 60 * 1000; * (Stripe and Square both retry for up to ~3 days). Processed payment rows that * no longer carry useful refund references MUST outlive this window: pruning one * while a retry can still arrive could re-process the paid session and re-issue - * a refund. Rows still needed for admin refunds are kept longer by prunePayments. + * a refund. Database pruning keeps rows that are still needed for admin refunds. */ export const WEBHOOK_RETRY_WINDOW_DAYS = 3; @@ -428,6 +428,13 @@ export const PRUNE_INTERVAL_HOURS = limit( "hours", ); +export const MAINTENANCE_PRUNE_BATCH = limit( + "MAINTENANCE_PRUNE_BATCH", + 500, + "Maintenance prune batch size", + "rows", +); + /** * Rows re-encrypted per activity-log backfill batch (default: 200). The whole * batch is written in one `executeBatch` (a single subrequest) after one SELECT, @@ -587,9 +594,6 @@ export const formatLimitValue = (value: number, unit: string): string => { if (unit === "bytes") return formatBytes(value); if (unit === "ms") return formatMs(value); if (unit === "seconds") return formatSeconds(value); - if (unit === "chars") return `${value} chars`; - if (unit === "days") return `${value} days`; - if (unit === "hours") return `${value} hours`; return `${value} ${unit}`; }; diff --git a/src/shared/maintenance/claims.ts b/src/shared/maintenance/claims.ts new file mode 100644 index 000000000..9ee8670d2 --- /dev/null +++ b/src/shared/maintenance/claims.ts @@ -0,0 +1,84 @@ +import { generateSecureToken } from "#shared/crypto/utils.ts"; +import { executeBatch, inPlaceholders, queryOne } from "#shared/db/client.ts"; + +const DATABASE_NOW_MS = + "CAST((julianday('now') - 2440587.5) * 86400000 AS INTEGER)"; + +export type MaintenanceClaim = { + checkpoint: string | null; + leaseToken: string; + name: string; +}; + +export const syncMaintenanceTaskRows = async ( + enabledNames: readonly string[], + disabledNames: readonly string[], +): Promise => { + const inserts = enabledNames.map((name) => ({ + args: [name], + sql: `INSERT OR IGNORE INTO maintenance_tasks (name, next_run_at) + VALUES (?, ${DATABASE_NOW_MS})`, + })); + const removals = + disabledNames.length === 0 + ? [] + : [ + { + args: [...disabledNames], + sql: `DELETE FROM maintenance_tasks + WHERE name IN (${inPlaceholders(disabledNames)}) + AND lease_token IS NULL`, + }, + ]; + const statements = [...inserts, ...removals]; + if (statements.length > 0) await executeBatch(statements); +}; + +export const claimNextMaintenanceTask = async ( + allowedNames: readonly string[], + leaseMs: number, +): Promise => { + if (allowedNames.length === 0) return null; + const leaseToken = generateSecureToken(); + const row = await queryOne<{ checkpoint: string | null; name: string }>( + `UPDATE maintenance_tasks AS task + SET lease_token = ?, + lease_expires_at = ${DATABASE_NOW_MS} + ?, + last_started_at = ${DATABASE_NOW_MS} + WHERE task.name = ( + SELECT candidate.name + FROM maintenance_tasks AS candidate + WHERE candidate.name IN (${inPlaceholders(allowedNames)}) + AND candidate.next_run_at <= ${DATABASE_NOW_MS} + AND ( + candidate.lease_token IS NULL + OR candidate.lease_expires_at <= ${DATABASE_NOW_MS} + ) + ORDER BY candidate.next_run_at ASC, candidate.name ASC + LIMIT 1 + ) + RETURNING name, checkpoint`, + [leaseToken, leaseMs, ...allowedNames], + ); + return row + ? { checkpoint: row.checkpoint, leaseToken, name: row.name } + : null; +}; + +export const finishMaintenanceTask = async ( + claim: MaintenanceClaim, + result: { checkpoint: string | null; intervalMs: number }, +): Promise => { + const row = await queryOne<{ name: string }>( + `UPDATE maintenance_tasks + SET next_run_at = ${DATABASE_NOW_MS} + ?, + checkpoint = ?, + lease_token = NULL, + lease_expires_at = NULL, + last_finished_at = ${DATABASE_NOW_MS} + WHERE name = ? AND lease_token = ? + RETURNING name`, + [result.intervalMs, result.checkpoint, claim.name, claim.leaseToken], + ); + if (!row) throw new Error(`Lost maintenance lease for ${claim.name}`); +}; diff --git a/src/shared/maintenance/definition.ts b/src/shared/maintenance/definition.ts new file mode 100644 index 000000000..4d4b7c569 --- /dev/null +++ b/src/shared/maintenance/definition.ts @@ -0,0 +1,106 @@ +export const MAINTENANCE_MIN_INTERVAL_MS = 60_000; +export const MAINTENANCE_REQUEST_CALL_LIMIT = 40; +const MAINTENANCE_RESERVED_CALLS = 7; +export const MAINTENANCE_TASK_CALL_LIMIT = + MAINTENANCE_REQUEST_CALL_LIMIT - MAINTENANCE_RESERVED_CALLS; +export const MAINTENANCE_REQUEST_DEADLINE_MS = 25_000; +export const MAINTENANCE_RELEASE_HEADROOM_MS = 1_000; + +export type MaintenanceWakePolicy = "organic_safe" | "scheduled_only"; + +export type MaintenanceTaskBudget = { + remaining: () => { database: number; external: number; total: number }; +}; + +export type MaintenanceTaskContext = { + budget: MaintenanceTaskBudget; + checkpoint: string | null; + deadline: number; + requestFollowUp: () => void; + setCheckpoint: (checkpoint: string | null) => void; +}; + +export interface MaintenanceTaskDeclaration { + deadlineMs: number; + enabled: () => boolean | Promise; + failureRetryIntervalMs: number; + intervalMs: number; + maxDatabaseCalls: number; + maxExternalCalls: number; + name: string; + run: (context: MaintenanceTaskContext) => void | Promise; + settingsKeys: readonly string[]; + wakePolicy: MaintenanceWakePolicy; +} + +const assertNonNegativeInteger = ( + task: MaintenanceTaskDeclaration, + value: number, + label: string, +): void => { + if (!Number.isInteger(value) || value < 0) { + throw new Error( + `${task.name} ${label} calls must be a non-negative integer`, + ); + } +}; + +const validateTask = (task: MaintenanceTaskDeclaration): void => { + if (!/^[a-z][a-z0-9_]*$/.test(task.name)) { + throw new Error(`Invalid maintenance task name: ${task.name}`); + } + if (task.intervalMs < MAINTENANCE_MIN_INTERVAL_MS) { + throw new Error( + `${task.name} interval must be at least ${MAINTENANCE_MIN_INTERVAL_MS}ms`, + ); + } + if ( + task.failureRetryIntervalMs < MAINTENANCE_MIN_INTERVAL_MS || + task.failureRetryIntervalMs > task.intervalMs + ) { + throw new Error( + `${task.name} failure retry must be between ${MAINTENANCE_MIN_INTERVAL_MS}ms and ${task.intervalMs}ms`, + ); + } + if ( + task.deadlineMs <= 0 || + task.deadlineMs > MAINTENANCE_REQUEST_DEADLINE_MS + ) { + throw new Error( + `${task.name} deadline must be greater than 0ms and at most ${MAINTENANCE_REQUEST_DEADLINE_MS}ms`, + ); + } + assertNonNegativeInteger(task, task.maxDatabaseCalls, "database"); + assertNonNegativeInteger(task, task.maxExternalCalls, "external"); + const calls = task.maxDatabaseCalls + task.maxExternalCalls; + if (calls > MAINTENANCE_TASK_CALL_LIMIT) { + throw new Error( + `${task.name} declares ${calls} calls; maximum is ${MAINTENANCE_TASK_CALL_LIMIT}`, + ); + } +}; + +export const defineMaintenanceTasks = < + const Tasks extends readonly MaintenanceTaskDeclaration[], +>( + tasks: Tasks, +): Tasks => { + const names = new Set(); + for (const task of tasks) { + if (names.has(task.name)) { + throw new Error(`Duplicate maintenance task: ${task.name}`); + } + names.add(task.name); + validateTask(task); + } + return tasks; +}; + +export const maintenanceTaskByName = ( + tasks: readonly MaintenanceTaskDeclaration[], + name: string, +): MaintenanceTaskDeclaration => { + const task = tasks.find((candidate) => candidate.name === name); + if (!task) throw new Error(`Claimed undeclared maintenance task: ${name}`); + return task; +}; diff --git a/src/shared/maintenance/registry.ts b/src/shared/maintenance/registry.ts new file mode 100644 index 000000000..83cfd72a1 --- /dev/null +++ b/src/shared/maintenance/registry.ts @@ -0,0 +1,50 @@ +import { hasLegacyActivityLog } from "#shared/db/activity-log-backfill.ts"; +import { settings } from "#shared/db/settings.ts"; +import { + ACTIVITY_LOG_BACKFILL_INTERVAL_MS, + PRUNE_INTERVAL_MS, +} from "#shared/limits.ts"; +import { defineMaintenanceTasks } from "#shared/maintenance/definition.ts"; +import { CONFIG_KEYS } from "#shared/settings/keys.ts"; + +const FAILURE_RETRY_MS = 5 * 60 * 1000; + +export const MAINTENANCE_TASKS = defineMaintenanceTasks([ + { + deadlineMs: 15_000, + enabled: () => true, + failureRetryIntervalMs: FAILURE_RETRY_MS, + intervalMs: PRUNE_INTERVAL_MS, + maxDatabaseCalls: 2, + maxExternalCalls: 0, + name: "database_pruning", + run: async ({ checkpoint, requestFollowUp, setCheckpoint }) => { + const { runDatabasePruning } = await import("#shared/db/prune.ts"); + const result = await runDatabasePruning(checkpoint); + setCheckpoint(result.checkpoint); + if (result.fullBatch) requestFollowUp(); + }, + settingsKeys: [ + CONFIG_KEYS.AUTO_PURGE_ORPHANS, + CONFIG_KEYS.ORPHAN_PURGE_RETENTION, + ], + wakePolicy: "organic_safe", + }, + { + deadlineMs: 10_000, + enabled: async () => Boolean(settings.publicKey) && hasLegacyActivityLog(), + failureRetryIntervalMs: ACTIVITY_LOG_BACKFILL_INTERVAL_MS, + intervalMs: ACTIVITY_LOG_BACKFILL_INTERVAL_MS, + maxDatabaseCalls: 2, + maxExternalCalls: 0, + name: "activity_log_backfill", + run: async () => { + const { runActivityLogBackfill } = await import( + "#shared/db/activity-log-backfill.ts" + ); + await runActivityLogBackfill(settings.publicKey); + }, + settingsKeys: [CONFIG_KEYS.PUBLIC_KEY], + wakePolicy: "organic_safe", + }, +]); diff --git a/src/shared/maintenance/report.ts b/src/shared/maintenance/report.ts new file mode 100644 index 000000000..91db298e4 --- /dev/null +++ b/src/shared/maintenance/report.ts @@ -0,0 +1,8 @@ +import { ErrorCode, logError } from "#shared/logger.ts"; + +export const reportMaintenanceFailure = ( + detail: string, + error: unknown, +): void => { + logError({ code: ErrorCode.CDN_REQUEST, detail, error }); +}; diff --git a/src/shared/maintenance/runner.ts b/src/shared/maintenance/runner.ts new file mode 100644 index 000000000..63a01c89e --- /dev/null +++ b/src/shared/maintenance/runner.ts @@ -0,0 +1,194 @@ +import { unique } from "#fp"; +import { settings } from "#shared/db/settings.ts"; +import { + getSubrequestRemaining, + getSubrequestUsage, + withSubrequestAllowance, +} from "#shared/subrequest-budget.ts"; +import { + claimNextMaintenanceTask, + finishMaintenanceTask, + type MaintenanceClaim, + syncMaintenanceTaskRows, +} from "./claims.ts"; +import { + MAINTENANCE_MIN_INTERVAL_MS, + MAINTENANCE_RELEASE_HEADROOM_MS, + MAINTENANCE_REQUEST_CALL_LIMIT, + MAINTENANCE_REQUEST_DEADLINE_MS, + type MaintenanceTaskDeclaration, + type MaintenanceWakePolicy, + maintenanceTaskByName, +} from "./definition.ts"; + +export type RunMaintenanceOptions = { + combinedAllowance?: number; + externalAllowance?: number; + requestDeadline?: number; + wakePolicy?: MaintenanceWakePolicy; +}; + +const taskCalls = (task: MaintenanceTaskDeclaration): number => + task.maxDatabaseCalls + task.maxExternalCalls; + +// Claim, finish, and the final no-work claim remain outside the task allowance. +const TASK_RUNNER_CALL_RESERVE = 3; + +const taskFits = ( + task: MaintenanceTaskDeclaration, + remaining: { database: number; external: number; total: number }, +): boolean => + task.maxExternalCalls <= remaining.external && + taskCalls(task) <= remaining.total - TASK_RUNNER_CALL_RESERVE; + +const enabledTasks = async ( + tasks: readonly MaintenanceTaskDeclaration[], +): Promise<{ + disabledNames: string[]; + enabled: MaintenanceTaskDeclaration[]; +}> => { + const states = await Promise.all( + tasks.map(async (task) => ({ enabled: await task.enabled(), task })), + ); + return { + disabledNames: states + .filter((state) => !state.enabled) + .map(({ task }) => task.name), + enabled: states.filter((state) => state.enabled).map(({ task }) => task), + }; +}; + +const runClaimedTask = async ( + task: MaintenanceTaskDeclaration, + claim: MaintenanceClaim, + requestDeadline: number, +): Promise => { + const deadline = Math.min( + Date.now() + task.deadlineMs, + requestDeadline - MAINTENANCE_RELEASE_HEADROOM_MS, + ); + let failure: unknown | null = null; + let needsFollowUp = false; + let checkpoint = claim.checkpoint; + try { + await withSubrequestAllowance( + { + database: task.maxDatabaseCalls, + external: task.maxExternalCalls, + total: taskCalls(task), + }, + () => + task.run({ + budget: { remaining: getSubrequestRemaining }, + checkpoint: claim.checkpoint, + deadline, + requestFollowUp: () => { + needsFollowUp = true; + }, + setCheckpoint: (nextCheckpoint) => { + checkpoint = nextCheckpoint; + }, + }), + ); + } catch (error) { + failure = error; + } + await finishMaintenanceTask(claim, { + checkpoint: failure === null ? checkpoint : claim.checkpoint, + intervalMs: + failure === null + ? needsFollowUp + ? MAINTENANCE_MIN_INTERVAL_MS + : task.intervalMs + : task.failureRetryIntervalMs, + }); + return failure; +}; + +const runWithinAllowance = async ( + declarations: readonly MaintenanceTaskDeclaration[], + options: RunMaintenanceOptions, + requestDeadline: number, +): Promise => { + const wakePolicy = options.wakePolicy ?? "scheduled_only"; + const candidates = declarations.filter( + (task) => + wakePolicy === "scheduled_only" || task.wakePolicy === "organic_safe", + ); + await settings.loadKeys( + unique(candidates.flatMap((task) => task.settingsKeys)), + ); + const { enabled, disabledNames } = await enabledTasks(candidates); + await syncMaintenanceTaskRows( + enabled.map((task) => task.name), + disabledNames, + ); + const failures: { error: unknown; name: string }[] = []; + + while (Date.now() < requestDeadline - MAINTENANCE_RELEASE_HEADROOM_MS) { + const remaining = getSubrequestRemaining(); + const allowed = enabled.filter((task) => taskFits(task, remaining)); + if (allowed.length === 0) break; + const claim = await claimNextMaintenanceTask( + allowed.map((task) => task.name), + requestDeadline - Date.now() + MAINTENANCE_RELEASE_HEADROOM_MS, + ); + if (!claim) break; + const task = maintenanceTaskByName(enabled, claim.name); + try { + const failure = await runClaimedTask(task, claim, requestDeadline); + if (failure !== null) failures.push({ error: failure, name: task.name }); + } catch (error) { + failures.push({ error, name: task.name }); + } + } + + if (failures.length > 0) { + throw new AggregateError( + failures.map((failure) => failure.error), + `Maintenance failed: ${failures.map((failure) => failure.name).join(", ")}`, + ); + } +}; + +const runMaintenance = ( + declarations: readonly MaintenanceTaskDeclaration[], + options: RunMaintenanceOptions = {}, +): Promise => { + const requestDeadline = + options.requestDeadline ?? Date.now() + MAINTENANCE_REQUEST_DEADLINE_MS; + const used = getSubrequestUsage().total; + const combinedAllowance = Math.max( + 0, + Math.min( + options.combinedAllowance ?? MAINTENANCE_REQUEST_CALL_LIMIT, + MAINTENANCE_REQUEST_CALL_LIMIT - used, + ), + ); + if (combinedAllowance === 0) return Promise.resolve(); + return withSubrequestAllowance( + { + database: combinedAllowance, + external: Math.min( + options.externalAllowance ?? combinedAllowance, + combinedAllowance, + ), + total: combinedAllowance, + }, + () => runWithinAllowance(declarations, options, requestDeadline), + ); +}; + +/** Run only database-safe work from an ordinary foreground request. */ +const runOrganicMaintenance = ( + declarations: readonly MaintenanceTaskDeclaration[], +): Promise => + runMaintenance(declarations, { + externalAllowance: 0, + wakePolicy: "organic_safe", + }); + +export const maintenance = { + run: runMaintenance, + runOrganic: runOrganicMaintenance, +}; diff --git a/src/shared/path.ts b/src/shared/path.ts new file mode 100644 index 000000000..474701ddd --- /dev/null +++ b/src/shared/path.ts @@ -0,0 +1,6 @@ +/** Remove trailing slashes from every non-root URL path. */ +export const normalizePath = (path: string): string => { + let end = path.length; + while (end > 1 && path[end - 1] === "/") end--; + return path.slice(0, end); +}; diff --git a/src/shared/provider-types.ts b/src/shared/provider-types.ts index f8179ab3b..e044739dc 100644 --- a/src/shared/provider-types.ts +++ b/src/shared/provider-types.ts @@ -19,10 +19,7 @@ export const databaseCredentialsFromResponse = ( return okResult({ dbId, dbToken: token, dbUrl }); }; -/** Create a site on a hosting provider: deploy `code` under `name` with the - * given secrets, returning the new site's id and default hostname. Declared - * once so every provider implementation shares the exact same signature. */ -export type CreateSiteFn = ( +export type PrepareSiteFn = ( name: string, code: string, secrets: [string, string][], @@ -30,8 +27,9 @@ export type CreateSiteFn = ( export interface HostingProviderApi { configEnvVar: string; - createSite: CreateSiteFn; getSecretNames(hostingId: string): Promise>; + prepareSite: PrepareSiteFn; + publishSite(hostingId: string, code: string): Promise>; setSecrets( hostingId: string, secrets: [string, string][], diff --git a/src/shared/renewal-helpers.ts b/src/shared/renewal-helpers.ts index 288186636..29c5e43a6 100644 --- a/src/shared/renewal-helpers.ts +++ b/src/shared/renewal-helpers.ts @@ -2,12 +2,12 @@ * Shared helpers for the renewal feature used by admin templates and routes. */ -import type { BuiltSite } from "#shared/db/built-sites.ts"; +import type { BuiltSite } from "#shared/db/built-sites/types.ts"; import { nowMs } from "#shared/now.ts"; /** Is a built site provisioned for renewals? (has a renewal token index) */ export const isProvisioned = (site: BuiltSite): boolean => - site.renewalTokenIndex !== null; + site.renewalTokenIndex !== null && site.renewalTokenIndex !== ""; /** Format a read_only_from ISO string for display in the admin UI */ export const formatDeadlineLabel = (iso: string, now = nowMs()): string => { diff --git a/src/shared/scheduled-access.ts b/src/shared/scheduled-access.ts new file mode 100644 index 000000000..5d610ee6a --- /dev/null +++ b/src/shared/scheduled-access.ts @@ -0,0 +1,46 @@ +import { constantTimeEqual } from "#shared/crypto/utils.ts"; +import { getEnv } from "#shared/env.ts"; +import { normalizePath } from "#shared/path.ts"; +import { SCHEDULED_TASK_KEY_ENV } from "#shared/scheduled-keys.ts"; + +export const SCHEDULED_PATH = "/scheduled"; + +export type ScheduledAccess = + | { kind: "not_scheduled" } + | { kind: "authorized" } + | { kind: "rejected"; status: 401 | 404 }; + +const bearerValue = (authorization: string | null): string | null => { + if (!authorization) return null; + const match = /^Bearer ([^\s]+)$/i.exec(authorization); + return match?.[1] ?? null; +}; + +export const checkScheduledAccess = ( + request: Pick, + key: string | undefined, +): ScheduledAccess => { + if (normalizePath(new URL(request.url).pathname) !== SCHEDULED_PATH) { + return { kind: "not_scheduled" }; + } + if (request.method !== "POST" || key === undefined) { + return { kind: "rejected", status: 404 }; + } + const supplied = bearerValue(request.headers.get("authorization")); + if (supplied === null) return { kind: "rejected", status: 401 }; + return constantTimeEqual(supplied, key) + ? { kind: "authorized" } + : { kind: "rejected", status: 401 }; +}; + +export const scheduledAccessFromEnv = (request: Request): ScheduledAccess => + checkScheduledAccess(request, getEnv(SCHEDULED_TASK_KEY_ENV)); + +export const scheduledResponse = (status: 204 | 401 | 404 | 503): Response => + new Response(null, { + headers: { + "cache-control": "no-store", + ...(status === 401 ? { "www-authenticate": "Bearer" } : {}), + }, + status, + }); diff --git a/src/shared/scheduled-keys.ts b/src/shared/scheduled-keys.ts new file mode 100644 index 000000000..24ed2972a --- /dev/null +++ b/src/shared/scheduled-keys.ts @@ -0,0 +1,29 @@ +import { + fromBase64Url, + generateSecureToken, + toBase64Url, +} from "#shared/crypto/utils.ts"; +import { getEnv } from "#shared/env.ts"; + +export const SCHEDULED_TASK_KEY_ENV = "SCHEDULED_TASK_KEY"; +export const SCHEDULED_KEY_BYTES = 32; + +export const generateScheduledTaskKey = (): string => generateSecureToken(); + +export const isScheduledTaskKey = (value: string): boolean => { + if (!/^[A-Za-z0-9_-]{43}$/.test(value)) return false; + const bytes = fromBase64Url(value); + return bytes.length === SCHEDULED_KEY_BYTES && toBase64Url(bytes) === value; +}; + +const validateKey = (name: string, value: string | undefined): void => { + if (value === undefined) return; + if (!isScheduledTaskKey(value)) { + throw new Error( + `${name} must be canonical unpadded base64url for exactly ${SCHEDULED_KEY_BYTES} bytes`, + ); + } +}; + +export const validateScheduledTaskKey = (): void => + validateKey(SCHEDULED_TASK_KEY_ENV, getEnv(SCHEDULED_TASK_KEY_ENV)); diff --git a/src/shared/sentry-sdk.ts b/src/shared/sentry-sdk.ts index 771146d06..a6ce6845b 100644 --- a/src/shared/sentry-sdk.ts +++ b/src/shared/sentry-sdk.ts @@ -20,6 +20,7 @@ import { getIsolationScope, isInitialized, } from "@sentry/deno"; +import { countExternalSubrequest } from "#shared/subrequest-budget.ts"; type InitOptions = { dsn: string; @@ -33,6 +34,7 @@ const makeFetchTransport: ConstructorParameters< typeof DenoClient >[0]["transport"] = (options) => createTransport(options, async (request) => { + countExternalSubrequest("Sentry transport"); const response = await fetch(options.url, { body: fetchBody(request.body), headers: new Headers(options.headers), diff --git a/src/shared/settings/keys.ts b/src/shared/settings/keys.ts index bbb35c01d..d7a1d52ed 100644 --- a/src/shared/settings/keys.ts +++ b/src/shared/settings/keys.ts @@ -1,5 +1,4 @@ export const CONFIG_KEY_NAMES = [ - "ACTIVITY_LOG_BACKFILL_DONE", "ADDRESS_LOOKUP_API_KEY", "ADDRESS_LOOKUP_PROVIDER", "APPLE_WALLET_PASS_TYPE_ID", @@ -39,17 +38,6 @@ export const CONFIG_KEY_NAMES = [ "GOOGLE_WALLET_SERVICE_ACCOUNT_KEY", "HEADER_IMAGE_URL", "HOMEPAGE_TEXT", - "LAST_ACTIVITY_LOG_BACKFILL", - "LAST_PRUNED_ADDRESSES", - "LAST_PRUNED_CONTACTS", - "LAST_PRUNED_INVITES", - "LAST_PRUNED_LOGINS", - "LAST_PRUNED_ORPHANS", - "LAST_PRUNED_PAYMENTS", - "LAST_PRUNED_SESSIONS", - "LAST_PRUNED_STRINGS", - "LAST_PRUNED_SUMUP", - "LAST_PRUNED_TOKENS", "LATEST_SCRIPT_VERSION", "LATEST_SCRIPT_VERSION_NAME", "LISTING_COLUMN_ORDER", diff --git a/src/shared/settings/registry.ts b/src/shared/settings/registry.ts index b5a126a77..2bbc1687f 100644 --- a/src/shared/settings/registry.ts +++ b/src/shared/settings/registry.ts @@ -4,7 +4,7 @@ export type { ConfigKey }; export { CONFIG_KEYS }; type StringStorage = "plaintext" | "encrypted"; -type StringTag = "emailBody" | "prune"; +type StringTag = "emailBody"; type AccessorConfig = { name: string; @@ -22,20 +22,6 @@ const setting = ( definition: Definition, ): Definition => definition; -/** A "last pruned" timestamp setting: plaintext, tagged for the pruner, and - * reached by an accessor of the given name. Collapses the repeated shape of the - * housekeeping timestamps below into one call. */ -const pruneSetting = ( - name: Name, - key: Key, -) => - setting({ - accessor: { name }, - key, - storage: "plaintext", - tags: ["prune"], - }); - /** An email template/credential setting: encrypted and tagged so it is rebuilt * whenever the email body changes. Collapses the repeated shape of the email * settings below into one call. */ @@ -130,43 +116,11 @@ export const STRING_SETTING_DEFINITIONS = [ key: CONFIG_KEYS.ATTENDEE_COLUMN_ORDER, storage: "plaintext", }), - setting({ - accessor: { name: "lastPrunedPayments" }, - key: CONFIG_KEYS.LAST_PRUNED_PAYMENTS, - storage: "plaintext", - }), - setting({ - accessor: { name: "lastPrunedSessions" }, - key: CONFIG_KEYS.LAST_PRUNED_SESSIONS, - storage: "plaintext", - }), - setting({ - accessor: { name: "lastPrunedSumup" }, - key: CONFIG_KEYS.LAST_PRUNED_SUMUP, - storage: "plaintext", - }), - pruneSetting("lastPrunedStrings", CONFIG_KEYS.LAST_PRUNED_STRINGS), - pruneSetting("lastPrunedLogins", CONFIG_KEYS.LAST_PRUNED_LOGINS), - pruneSetting("lastPrunedTokens", CONFIG_KEYS.LAST_PRUNED_TOKENS), - pruneSetting("lastPrunedContacts", CONFIG_KEYS.LAST_PRUNED_CONTACTS), - pruneSetting("lastPrunedAddresses", CONFIG_KEYS.LAST_PRUNED_ADDRESSES), - pruneSetting("lastPrunedInvites", CONFIG_KEYS.LAST_PRUNED_INVITES), - pruneSetting("lastPrunedOrphans", CONFIG_KEYS.LAST_PRUNED_ORPHANS), setting({ accessor: { name: "smsGatewayBaseUrl" }, key: CONFIG_KEYS.SMS_GATEWAY_BASE_URL, storage: "plaintext", }), - setting({ - accessor: { name: "activityLogBackfillDone" }, - key: CONFIG_KEYS.ACTIVITY_LOG_BACKFILL_DONE, - storage: "plaintext", - }), - setting({ - accessor: { name: "lastActivityLogBackfill" }, - key: CONFIG_KEYS.LAST_ACTIVITY_LOG_BACKFILL, - storage: "plaintext", - }), setting({ accessor: { name: "businessEmail" }, key: CONFIG_KEYS.BUSINESS_EMAIL, @@ -335,5 +289,4 @@ const keysWithTag = (tag: StringTag): readonly StringSettingKey[] => export const PLAINTEXT_KEYS = keysWithStorage("plaintext"); export const ENCRYPTED_KEYS = keysWithStorage("encrypted"); -export const PRUNE_KEYS = keysWithTag("prune"); export const EMAIL_BODY_KEYS = keysWithTag("emailBody"); diff --git a/src/shared/site-assignment.ts b/src/shared/site-assignment.ts index 6042f892c..95897c788 100644 --- a/src/shared/site-assignment.ts +++ b/src/shared/site-assignment.ts @@ -6,16 +6,14 @@ import { sort } from "#fp"; import { isBuilderEnabled } from "#routes/admin/builder.ts"; -import { builderApi, resolveHostingProvider } from "#shared/builder.ts"; +import { resolveHostingProvider } from "#shared/builder.ts"; import { getEffectiveDomain } from "#shared/config.ts"; import { hmacHash } from "#shared/crypto/hashing.ts"; import { generateSecureToken } from "#shared/crypto/utils.ts"; import { addMonthsIso, parseDateMs } from "#shared/dates.ts"; +import type { BuiltSite } from "#shared/db/built-sites/types.ts"; import { assignBuiltSite, - type BuiltSite, - builtSites, - builtSitesCrudTable, getAssignableBuiltSites, siteBaseUrl, updateBuiltSiteRenewalState, @@ -30,6 +28,7 @@ import { import { ErrorCode, logError } from "#shared/logger.ts"; import { nowIso, nowMs } from "#shared/now.ts"; import { sendNtfyError } from "#shared/ntfy.ts"; +import { buildAssignableSite } from "#shared/site-build.ts"; import { parseEmail, type ValidEmail } from "#shared/validation/email.ts"; /** Entry with the fields needed for site assignment */ @@ -79,33 +78,6 @@ export type SiteAssignmentConfigValidation = listingId?: number; }; -/** Compute the next sequential site name based on total site count, zero-padded to 5 digits. */ -const nextSiteName = async (): Promise => - String((await builtSites.getAll()).length + 1).padStart(5, "0"); - -/** Build a new site on-demand and insert it as an assignable record. */ -const buildSiteForAssignment = async (): Promise => { - const name = await nextSiteName(); - const result = await builderApi.buildSite({ siteName: name }); - if (!result.ok) { - logError({ - code: ErrorCode.CDN_REQUEST, - detail: `Failed to auto-build site '${name}': ${result.error}`, - }); - return null; - } - return builtSitesCrudTable.insert({ - assignable: true, - dbProvider: result.dbProvider, - dbToken: result.dbToken, - dbUrl: result.dbUrl, - hostingId: result.hostingId, - hostingProvider: result.hostingProvider, - name, - siteUrl: result.defaultHostname, - }); -}; - /** Pick the cheapest qualifying tier listing (purchase_only=1, hidden=1, months_per_unit>0, active=1). */ export const isQualifyingTierListing = (listing: RenewalTierListing): boolean => listing.purchase_only && @@ -357,7 +329,7 @@ const assignSitesForEntries = async ( for (const { listing, attendee } of needsSite) { const qty = attendee.quantity; for (let i = 0; i < qty; i++) { - const site = available[idx] ?? (await buildSiteForAssignment()); + const site = available[idx] ?? (await buildAssignableSite()); if (!site) break; assignments.push( diff --git a/src/shared/site-build.ts b/src/shared/site-build.ts new file mode 100644 index 000000000..24b605de0 --- /dev/null +++ b/src/shared/site-build.ts @@ -0,0 +1,61 @@ +import { validateBootChecks } from "#shared/boot-checks.ts"; +import { + type BuildSiteInput, + type BuildSiteResult, + builderApi, +} from "#shared/builder.ts"; +import type { BuiltSite } from "#shared/db/built-sites/types.ts"; +import { + builtSites, + builtSitesCrudTable, + insertBuiltSite, +} from "#shared/db/built-sites.ts"; +import { initDb } from "#shared/db/migrations.ts"; +import { ErrorCode, logError } from "#shared/logger.ts"; + +const nextSiteName = async (): Promise => + String((await builtSites.getAll()).length + 1).padStart(5, "0"); + +export const buildRetainedSite = async ( + name: string, + input: BuildSiteInput, +): Promise<{ result: BuildSiteResult; retainedId: number }> => { + validateBootChecks(); + await initDb(); + const retainedId = { value: 0 }; + const result = await builderApi.buildSite(input, async (site) => { + const row = await insertBuiltSite( + name, + site.defaultHostname, + site.dbUrl, + site.dbToken, + false, + site.hostingId, + undefined, + site.hostingProvider, + site.dbProvider, + site.scheduledTaskKey, + ); + retainedId.value = row.id; + }); + if (result.ok && retainedId.value === 0) { + throw new Error("Built site was not retained"); + } + return { result, retainedId: retainedId.value }; +}; + +/** Build and retain one site before making it available for assignment. */ +export const buildAssignableSite = async (): Promise => { + const name = await nextSiteName(); + const { result, retainedId } = await buildRetainedSite(name, { + siteName: name, + }); + if (!result.ok) { + logError({ + code: ErrorCode.CDN_REQUEST, + detail: `Failed to auto-build site '${name}': ${result.error}`, + }); + return null; + } + return builtSitesCrudTable.update(retainedId, { assignable: true }); +}; diff --git a/src/shared/site-db.ts b/src/shared/site-db.ts index ed2a1aa26..551ae70dc 100644 --- a/src/shared/site-db.ts +++ b/src/shared/site-db.ts @@ -13,7 +13,7 @@ */ import { type Client, createClient } from "@libsql/client"; -import type { BuiltSite } from "#shared/db/built-sites.ts"; +import type { BuiltSite } from "#shared/db/built-sites/types.ts"; import type { Result } from "#shared/result.ts"; /** Credentials needed to open a read-only connection to a site's database. */ @@ -22,7 +22,7 @@ export type SiteDbCredentials = Pick; /** Stubbable client factory so tests can inject an in-memory database. */ export const siteDbApi = { createClient: (url: string, authToken: string): Client => - createClient(authToken ? { authToken, url } : { url }), + createClient({ authToken, url }), }; /** Do we hold both a URL and a read-only token for this site's database? */ diff --git a/src/shared/site-scheduler.ts b/src/shared/site-scheduler.ts new file mode 100644 index 000000000..49ca2cc41 --- /dev/null +++ b/src/shared/site-scheduler.ts @@ -0,0 +1,64 @@ +import { resolveHostingProvider, siteHostingAccess } from "#shared/builder.ts"; +import { ensureBuiltSiteSchedulerKey } from "#shared/db/built-site-scheduler.ts"; +import { findBuiltSiteByIdPrimary } from "#shared/db/built-sites.ts"; +import { fetchText } from "#shared/fetch.ts"; +import { errorResult, okResult, type Result } from "#shared/result.ts"; +import { SCHEDULED_TASK_KEY_ENV } from "#shared/scheduled-keys.ts"; + +type SiteSchedulerResult = Result; + +const getSite = async (siteId: number) => { + const site = await findBuiltSiteByIdPrimary(siteId); + if (!site) throw new Error(`Built site not found: ${siteId}`); + return site; +}; + +const verifyScheduledKey = async ( + siteUrl: string, + key: string, +): Promise => { + try { + const result = await fetchText( + `${new URL(/^https?:\/\//i.test(siteUrl) ? siteUrl : `https://${siteUrl}`).origin}/scheduled`, + { + headers: { authorization: `Bearer ${key}` }, + method: "POST", + redirect: "manual", + }, + ); + return result.status === 204 && result.text === "" + ? okResult(undefined) + : errorResult("The child did not accept the scheduled task key."); + } catch { + return errorResult("The child could not verify the scheduled task key."); + } +}; + +export const provisionSiteScheduler = async ( + siteId: number, +): Promise => { + const site = await getSite(siteId); + const access = siteHostingAccess( + site, + "its scheduled task key cannot be set", + ); + if (!access.ok) return access; + const provider = resolveHostingProvider(site.hostingProvider); + if (!site.scheduledTaskKey) { + const listed = await provider.getSecretNames(access.hostingId); + if (!listed.ok) return listed; + if (listed.value.includes(SCHEDULED_TASK_KEY_ENV)) { + return { + error: + "The child already has a scheduled task key that this site cannot read.", + ok: false, + }; + } + } + const key = await ensureBuiltSiteSchedulerKey(siteId); + const set = await provider.setSecrets(access.hostingId, [ + [SCHEDULED_TASK_KEY_ENV, key], + ]); + if (!set.ok) return set; + return verifyScheduledKey(site.siteUrl, key); +}; diff --git a/src/shared/site-secrets.ts b/src/shared/site-secrets.ts index 9dd674e67..26e96e3d5 100644 --- a/src/shared/site-secrets.ts +++ b/src/shared/site-secrets.ts @@ -21,7 +21,7 @@ import { resolveHostingProvider, siteHostingAccess, } from "#shared/builder.ts"; -import type { BuiltSite } from "#shared/db/built-sites.ts"; +import type { BuiltSite } from "#shared/db/built-sites/types.ts"; import type { Result } from "#shared/result.ts"; import { tryStep } from "#shared/try-step.ts"; diff --git a/src/shared/site-update.ts b/src/shared/site-update.ts index b1a506a98..9328bede9 100644 --- a/src/shared/site-update.ts +++ b/src/shared/site-update.ts @@ -12,7 +12,7 @@ /* jscpd:ignore-start */ import { resolveHostingProvider } from "#shared/builder.ts"; import { logActivity } from "#shared/db/activityLog.ts"; -import type { BuiltSite } from "#shared/db/built-sites.ts"; +import type { BuiltSite } from "#shared/db/built-sites/types.ts"; import { settings } from "#shared/db/settings.ts"; import { getEnv } from "#shared/env.ts"; import type { Result } from "#shared/result.ts"; diff --git a/src/shared/storage.ts b/src/shared/storage.ts index 2101b4e15..6a63f1096 100644 --- a/src/shared/storage.ts +++ b/src/shared/storage.ts @@ -24,6 +24,7 @@ import { import { ErrorCode, logError } from "#shared/logger.ts"; import { createScopedValue } from "#shared/request-scoped.ts"; import { streamChunks } from "#shared/stream-chunks.ts"; +import { countExternalSubrequest } from "#shared/subrequest-budget.ts"; import { getDeleteOverride } from "#shared/test-overrides.ts"; import type { NonEmptyString } from "#shared/validation/string.ts"; @@ -80,8 +81,8 @@ const getStorageConfig = (): StorageConfig => { const ctx = configOverride.read(); if (ctx) return ctx; return { - zoneKey: getEnv("STORAGE_ZONE_KEY") ?? "", - zoneName: getEnv("STORAGE_ZONE_NAME") ?? "", + zoneKey: getEnv("STORAGE_ZONE_KEY") || "", + zoneName: getEnv("STORAGE_ZONE_NAME") || "", }; }; @@ -94,7 +95,7 @@ const getLocalStoragePath = (): string | null => { if (ctx && "localPath" in ctx) { return ctx.localPath || null; } - return getEnv("LOCAL_STORAGE_PATH") ?? null; + return getEnv("LOCAL_STORAGE_PATH") || null; }; /** Derived lookup: file extension → MIME type, for serving stored files. */ @@ -130,7 +131,7 @@ export const getImageProxyUrl = (filename: string): string => export const getMimeTypeFromFilename = (filename: string): ImageMime | null => { const dotIndex = filename.lastIndexOf("."); if (dotIndex === -1) return null; - return EXT_TO_MIME[filename.slice(dotIndex)] ?? null; + return EXT_TO_MIME[filename.slice(dotIndex)] || null; }; /** @@ -370,6 +371,7 @@ export const uploadRaw = async ( controller.close(); }, }); + countExternalSubrequest("storage upload"); await sdk.file.upload(sz, `/${filename}`, stream as never, { contentType: "application/octet-stream", }); @@ -441,6 +443,7 @@ export const downloadRaw = async ( } try { const { sdk, sz } = await connectZone(); + countExternalSubrequest("storage download"); const { stream } = await sdk.file.download(sz, `/${filename}`); return collectStream(stream as ReadableStream); } catch (err) { @@ -472,6 +475,7 @@ export const deleteFile = async (filename: string): Promise => { return; } const { sdk, sz } = await connectZone(); + countExternalSubrequest("storage delete"); await sdk.file.remove(sz, `/${filename}`); }; @@ -565,7 +569,7 @@ const readDirSafe = async (dir: string): Promise => { export type StorageFileMeta = { name: string; size: number }; /** Sort stored files by name, ascending. */ -const byName = sort((a, b) => (a.name < b.name ? -1 : 1)); +const byName = sort((a, b) => a.name.localeCompare(b.name)); /** * Split a listing prefix into the directory to read and the leaf-name filter @@ -610,6 +614,7 @@ export const listFilesWithMeta = async ( } const config = getStorageConfig(); const url = `https://storage.bunnycdn.com/${config.zoneName}/${dir}`; + countExternalSubrequest("storage directory listing"); const response = await fetch(url, { headers: { AccessKey: config.zoneKey }, }); @@ -626,7 +631,7 @@ export const listFilesWithMeta = async ( // isFile for the same reason). .filter((item) => !item.IsDirectory) .map((item) => ({ - name: String(item.ObjectName ?? ""), + name: String(item.ObjectName || ""), size: Number(item.Length) || 0, })) .filter((f) => f.name !== "" && f.name.startsWith(namePrefix)) diff --git a/src/shared/subrequest-budget.ts b/src/shared/subrequest-budget.ts index 780964ccf..030c7a2ef 100644 --- a/src/shared/subrequest-budget.ts +++ b/src/shared/subrequest-budget.ts @@ -1,5 +1,112 @@ +import { createScope } from "#shared/request-scoped.ts"; + /** Bunny Edge Scripting stops an incoming request after this many subrequests. */ export const BUNNY_SUBREQUEST_LIMIT = 50; +export type SubrequestKind = "database" | "external"; + +export type SubrequestCounts = { + database: number; + external: number; + total: number; +}; + +type Counts = { database: number; external: number }; +type BudgetState = { + counts: Counts; + limits: Counts & { total: number }; +}; + +const budgetScope = createScope(); + +const freshBudget = (): BudgetState => ({ + counts: { database: 0, external: 0 }, + limits: { + database: BUNNY_SUBREQUEST_LIMIT, + external: BUNNY_SUBREQUEST_LIMIT, + total: BUNNY_SUBREQUEST_LIMIT, + }, +}); + +const usageOf = ({ database, external }: Counts): SubrequestCounts => ({ + database, + external, + total: database + external, +}); + +export const getSubrequestUsage = (): SubrequestCounts => + usageOf(budgetScope.current()?.counts ?? { database: 0, external: 0 }); + +export const getSubrequestRemaining = (): SubrequestCounts => { + const state = budgetScope.current(); + if (!state) { + return { + database: BUNNY_SUBREQUEST_LIMIT, + external: BUNNY_SUBREQUEST_LIMIT, + total: BUNNY_SUBREQUEST_LIMIT, + }; + } + const usage = usageOf(state.counts); + return { + database: Math.max(0, state.limits.database - usage.database), + external: Math.max(0, state.limits.external - usage.external), + total: Math.max(0, state.limits.total - usage.total), + }; +}; + +export const runWithSubrequestBudget = (fn: () => T): T => + budgetScope.run(freshBudget(), fn); + +export const withSubrequestAllowance = ( + allowance: SubrequestCounts, + fn: () => T, +): T => { + const parent = budgetScope.current(); + if (!parent) + return runWithSubrequestBudget(() => + withSubrequestAllowance(allowance, fn), + ); + const usage = usageOf(parent.counts); + return budgetScope.run( + { + counts: parent.counts, + limits: { + database: Math.min( + parent.limits.database, + usage.database + allowance.database, + ), + external: Math.min( + parent.limits.external, + usage.external + allowance.external, + ), + total: Math.min(parent.limits.total, usage.total + allowance.total), + }, + }, + fn, + ); +}; + +export const countSubrequest = ( + kind: SubrequestKind, + operation: string, +): void => { + const state = budgetScope.current(); + if (!state) return; + state.counts[kind] += 1; + const usage = usageOf(state.counts); + if ( + state.counts[kind] > state.limits[kind] || + usage.total > state.limits.total + ) { + throw new Error( + `Subrequest allowance exceeded: ${usage.database} database + ` + + `${usage.external} external calls. Blocked ${kind} operation: ${operation}`, + ); + } +}; + +export const countExternalSubrequest = (operation: string): void => + countSubrequest("external", operation); + /** A refund uses several database calls plus one payment-provider request. */ export const BULK_REFUND_LIMIT = 5; diff --git a/src/shared/update.ts b/src/shared/update.ts index de3a79e4e..8d8a58201 100644 --- a/src/shared/update.ts +++ b/src/shared/update.ts @@ -17,6 +17,7 @@ import { import { denoDeployApi } from "#shared/deno-deploy-api.ts"; import { logDebug } from "#shared/logger.ts"; import { type Result, requireSuccess } from "#shared/result.ts"; +import { countExternalSubrequest } from "#shared/subrequest-budget.ts"; /** GitHub repo URL — update here if the repo moves */ export const GITHUB_REPO = "chobbledotcom/tickets"; @@ -78,8 +79,10 @@ export const setBuildTimestampForTest = (ts: string | null): void => { }; /** Get the effective build timestamp (override or real). */ -const getEffectiveBuildTimestamp = (): string => - getBuildTimestampOverride() ?? BUILD_TIMESTAMP; +const getEffectiveBuildTimestamp = (): string => { + const override = getBuildTimestampOverride(); + return override === null ? BUILD_TIMESTAMP : override; +}; /** Override for BUILD_COMMIT in tests; null falls back to the compile-time constant. */ const [getBuildCommitOverride, setBuildCommitOverride] = lazyRef( @@ -92,8 +95,10 @@ export const setBuildCommitForTest = (commit: string | null): void => { }; /** Get the effective build commit (override or real). */ -const getEffectiveBuildCommit = (): string => - getBuildCommitOverride() ?? BUILD_COMMIT; +const getEffectiveBuildCommit = (): string => { + const override = getBuildCommitOverride(); + return override === null ? BUILD_COMMIT : override; +}; /** * Check if a release tag is newer than a build timestamp. Defaults to this @@ -192,6 +197,7 @@ export const formatBuildDate = (iso: string): string => { * Throws on network/API errors. */ export const fetchLatestRelease = async (): Promise => { + countExternalSubrequest("GitHub release lookup"); const response = await fetch(GITHUB_LATEST_RELEASE_URL, { headers: { Accept: "application/vnd.github.v3+json" }, }); @@ -212,6 +218,7 @@ export const fetchLatestRelease = async (): Promise => { /** Download a release asset URL and return the source code text. */ const downloadReleaseAsset = async (assetUrl: string): Promise => { + countExternalSubrequest("GitHub release download"); const assetResponse = await fetch(assetUrl); if (!assetResponse.ok) { throw new Error( diff --git a/src/ui/templates/admin/built-sites.tsx b/src/ui/templates/admin/built-sites.tsx index af7800035..25492d3dd 100644 --- a/src/ui/templates/admin/built-sites.tsx +++ b/src/ui/templates/admin/built-sites.tsx @@ -1,5 +1,8 @@ import { t } from "#i18n"; -import { type BuiltSite, DEFAULT_UPDATE_TIER } from "#shared/db/built-sites.ts"; +import { + type BuiltSite, + DEFAULT_UPDATE_TIER, +} from "#shared/db/built-sites/types.ts"; import type { FormRenderValuesFor } from "#shared/forms/definition.ts"; import { booleanToCheckbox } from "#shared/forms/values.ts"; import { escapeHtml, Raw } from "#shared/jsx/jsx-runtime.ts"; diff --git a/src/ui/templates/admin/built-sites/list-parts.tsx b/src/ui/templates/admin/built-sites/list-parts.tsx index f385ad865..47e901938 100644 --- a/src/ui/templates/admin/built-sites/list-parts.tsx +++ b/src/ui/templates/admin/built-sites/list-parts.tsx @@ -1,5 +1,5 @@ import { t } from "#i18n"; -import type { BuiltSite } from "#shared/db/built-sites.ts"; +import type { BuiltSite } from "#shared/db/built-sites/types.ts"; import { formatDeadlineLabel } from "#shared/renewal-helpers.ts"; import type { ListingWithCount } from "#shared/types.ts"; import { RenewalTierSummary } from "#templates/admin/built-sites/renewal-summary.tsx"; diff --git a/src/ui/templates/admin/built-sites/panels.tsx b/src/ui/templates/admin/built-sites/panels.tsx index fbc3ba8f7..182bba10a 100644 --- a/src/ui/templates/admin/built-sites/panels.tsx +++ b/src/ui/templates/admin/built-sites/panels.tsx @@ -1,5 +1,5 @@ import { t } from "#i18n"; -import type { BuiltSite } from "#shared/db/built-sites.ts"; +import type { BuiltSite } from "#shared/db/built-sites/types.ts"; import { CsrfForm } from "#shared/forms/csrf-form.tsx"; import { type Child, Raw } from "#shared/jsx/jsx-runtime.ts"; import { formatDeadlineLabel, isProvisioned } from "#shared/renewal-helpers.ts"; @@ -163,6 +163,31 @@ const unprovisionedPanel = (site: BuiltSite): JSX.Element => ( export const renewalPanelFor = (site: BuiltSite): JSX.Element => isProvisioned(site) ? provisionedPanel(site) : unprovisionedPanel(site); +export const MaintenancePanel = ({ + site, +}: { + site: BuiltSite; +}): JSX.Element => ( +
+

{t("built_sites.maintenance_intro")}

+ {site.scheduledTaskKey && ( +

+ {t("built_sites.maintenance_site_key")}{" "} + {site.scheduledTaskKey} +

+ )} + + + {t( + site.scheduledTaskKey + ? "built_sites.maintenance_resend" + : "built_sites.maintenance_provision", + )} + + +
+); + export const SecretsPanel = ({ site, view, diff --git a/src/ui/templates/admin/debug.tsx b/src/ui/templates/admin/debug.tsx index e21fc4731..33e110c50 100644 --- a/src/ui/templates/admin/debug.tsx +++ b/src/ui/templates/admin/debug.tsx @@ -86,13 +86,6 @@ export type DebugPageState = { runtime: RuntimeInfo; domain: string; limits: typeof LIMIT_ENTRIES; - prune: { - addresses: string; - payments: string; - sessions: string; - strings: string; - logins: string; - }; theme: Theme; }; @@ -503,43 +496,6 @@ const LimitsSection = ({ ), ); -/** Every pruned table, named as it appears in the schema, paired with the - * `DebugPageState["prune"]` field that carries its last-pruned time. */ -const PRUNE_ROWS: readonly [ - table: string, - field: keyof DebugPageState["prune"], -][] = [ - ["processed_payments", "payments"], - ["sessions", "sessions"], - ["strings", "strings"], - ["login_attempts", "logins"], - ["address_cache", "addresses"], -]; - -const PruneSection = ({ - prune, -}: { - prune: DebugPageState["prune"]; -}): JSX.Element => - proseTableSection( - t("debug.section.database_pruning"), - <> - Automatic cleanup of short-lived rows. Runs in the background on incoming - requests; frequency controlled by PRUNE_INTERVAL_HOURS. - , - )( - rowsTable( - [t("debug.field.table"), t("debug.field.last_pruned_utc")], - PRUNE_ROWS, - ([table, field]) => ( - - {table} - {prune[field]} - - ), - ), - ); - /** * Admin debug page */ @@ -558,6 +514,5 @@ export const adminDebugPage = ( ))} - , ); diff --git a/src/ui/templates/admin/settings-advanced.tsx b/src/ui/templates/admin/settings-advanced.tsx index f6b23fdc5..5f9a233f4 100644 --- a/src/ui/templates/admin/settings-advanced.tsx +++ b/src/ui/templates/admin/settings-advanced.tsx @@ -57,12 +57,13 @@ export type AdvancedSettingsPageState = { theme: Theme; listingColumnOrder: string; attendeeColumnOrder: string; - paymentProvider: string; + paymentProvider: string | null; smsGatewayUsername: string; smsGatewayBaseUrl: string; smsGatewayPasswordConfigured: boolean; smsGatewayPassphraseConfigured: boolean; smsGatewayWebhookConfigured: boolean; + scheduledTaskKey: string | undefined; }; export const adminAdvancedSettingsPage = ( @@ -95,6 +96,16 @@ export const adminAdvancedSettingsPage = ( {SmsGatewayForm(s)} {AddressLookupForm(s)} +
+

{t("settings.advanced.scheduled_title")}

+

{t("settings.advanced.scheduled_intro")}

+ {s.scheduledTaskKey ? ( + {s.scheduledTaskKey} + ) : ( +

{t("settings.advanced.scheduled_unset")}

+ )} +
+ { if (paymentProvider !== "square" && paymentProvider !== "stripe") return null; return ( diff --git a/src/ui/templates/fields/validators.ts b/src/ui/templates/fields/validators.ts index 171946809..15cdef2db 100644 --- a/src/ui/templates/fields/validators.ts +++ b/src/ui/templates/fields/validators.ts @@ -11,7 +11,7 @@ import * as v from "valibot"; import { t } from "#i18n"; import { VALID_DAY_NAMES } from "#shared/day-names.ts"; -import { isUpdateTier } from "#shared/db/built-sites.ts"; +import { isUpdateTier } from "#shared/db/built-sites/types.ts"; import type { ChoiceField, Field, diff --git a/test/integration/server/builder.test.ts b/test/features/admin/builder/server.test.ts similarity index 81% rename from test/integration/server/builder.test.ts rename to test/features/admin/builder/server.test.ts index 80a15467d..03376b59e 100644 --- a/test/integration/server/builder.test.ts +++ b/test/features/admin/builder/server.test.ts @@ -1,6 +1,7 @@ import { expect } from "@std/expect"; import { afterEach, it as test } from "@std/testing/bdd"; import { stub } from "@std/testing/mock"; +import { builderForm } from "#routes/admin/builder.ts"; import { builderApi } from "#shared/builder.ts"; import { builtSites } from "#shared/db/built-sites.ts"; import { ALL_SETTINGS_KEYS, settings } from "#shared/db/settings.ts"; @@ -25,7 +26,12 @@ import { } from "#test-utils/assertions.ts"; import { describeWithEnv } from "#test-utils/db.ts"; import { withEnv } from "#test-utils/env.ts"; -import { awaitTestRequest, withMocks } from "#test-utils/mocks.ts"; +import { + awaitTestRequest, + withExpectedError, + withMocks, +} from "#test-utils/mocks.ts"; +import { TEST_SCHEDULED_KEY } from "#test-utils/scheduled.ts"; import { adminFormPost, adminGet, testCookie } from "#test-utils/session.ts"; /** Stub all Bunny + GitHub APIs for a successful build */ @@ -47,13 +53,15 @@ const expectSiteCreated = async (response: Response) => { /** Stub `buildSite` to capture its input and return a success result. */ const stubBuildAndCapture = () => { const capture: { + currentTask: string | null; input: Parameters[0] | null; restore?: () => void; - } = { input: null }; + } = { currentTask: null, input: null }; return { - buildStub: stub(builderApi, "buildSite", (input) => { + buildStub: stub(builderApi, "buildSite", async (input, retain) => { + capture.currentTask = settings.currentTask; capture.input = input; - return Promise.resolve({ + const result = { dbProvider: "bunny" as const, dbToken: "tok", dbUrl: "libsql://test.io", @@ -61,7 +69,9 @@ const stubBuildAndCapture = () => { hostingId: "42", hostingProvider: "bunny" as const, ok: true as const, - }); + }; + await retain({ ...result, scheduledTaskKey: TEST_SCHEDULED_KEY }); + return result; }), capture, dbTestStub: stub(builderApi, "testDbConnection", () => @@ -70,6 +80,54 @@ const stubBuildAndCapture = () => { }; }; +test("builder form defines every field and option exactly", () => { + expect(builderForm.id).toBe("builder"); + expect(JSON.parse(JSON.stringify(builderForm.fields))).toEqual([ + { + label: "Site Name", + maxlength: 64, + minlength: 1, + name: "site_name", + placeholder: "My Listing Site", + required: true, + type: "text", + }, + { + label: "Hosting Provider", + name: "hosting_provider", + options: [ + { label: "Bunny Edge Scripting", value: "bunny" }, + { label: "Deno Deploy", value: "deno" }, + ], + type: "select", + }, + { + label: "Database Provider", + name: "db_provider", + options: [ + { label: "Bunny DB (auto-provision)", value: "bunny" }, + { label: "Turso (auto-provision)", value: "turso" }, + { label: "Manual (enter URL below)", value: "manual" }, + ], + type: "select", + }, + { + hint: "Leave blank to auto-provision a database", + label: "Database URL", + name: "db_url", + placeholder: "libsql://your-db.turso.io", + type: "url", + }, + { + hint: "Leave blank to auto-provision a database", + label: "Database Token", + name: "db_token", + placeholder: "Token for the database", + type: "password", + }, + ]); +}); + describeWithEnv( "server (admin builder)", { @@ -333,6 +391,49 @@ describeWithEnv( ); }); + test("fails if a provider reports success without retaining the site", async () => { + await withMocks( + () => ({ + buildStub: stub(builderApi, "buildSite", () => + Promise.resolve({ + dbProvider: "bunny" as const, + dbToken: "token", + dbUrl: "libsql://test.io", + defaultHostname: "child.example.test", + hostingId: "42", + hostingProvider: "bunny" as const, + ok: true as const, + }), + ), + dbTestStub: stubDbOk(), + }), + async () => { + const errorStub = stub(console, "error", () => {}); + try { + const { response } = await withExpectedError(() => + adminFormPost("/admin/builder", { + db_token: "token", + db_url: "libsql://test.io", + site_name: "Unretained", + }), + ); + expect(response.status).toBe(503); + expect( + errorStub.calls.some((call) => + call.args.some((arg) => + String(arg).includes( + "Built site was published before it was retained", + ), + ), + ), + ).toBe(true); + } finally { + errorStub.restore(); + } + }, + ); + }); + test("POST /admin/builder returns 404 when CAN_BUILD_SITES is not set", async () => { using _env = withEnv({ CAN_BUILD_SITES: undefined }); const { response } = await adminFormPost("/admin/builder", { @@ -403,6 +504,8 @@ describeWithEnv( }); expectRedirect(response, "/admin/builder"); expect(capture.input?.hostingProvider).toBe("deno"); + expect(capture.currentTask).toBe("builder"); + expect(settings.currentTask).toBe(""); }, ); }); diff --git a/test/features/admin/built-site-action.test.ts b/test/features/admin/built-site-action.test.ts new file mode 100644 index 000000000..76bc74f52 --- /dev/null +++ b/test/features/admin/built-site-action.test.ts @@ -0,0 +1,87 @@ +import { expect } from "@std/expect"; +import { describe, it as test } from "@std/testing/bdd"; +import { + builtSiteAction, + builtSiteTabResult, +} from "#routes/admin/built-site-action.ts"; +import { expectRedirectWithFlash } from "#test-utils/assertions.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; +import { mockFormRequest } from "#test-utils/mocks.ts"; +import { insertScheduledTestSite } from "#test-utils/scheduled.ts"; +import { getTestSession } from "#test-utils/session.ts"; + +describe("built-site tab results", () => { + const result = builtSiteTabResult( + "maintenance", + (error) => `Could not save: ${error}`, + )("Saved."); + + test("returns to the selected tab with a success message", () => { + expectRedirectWithFlash( + "/admin/built-sites/7/maintenance", + "Saved.", + )(result(7, { ok: true })); + }); + + test("returns to the selected tab with the transformed provider error", () => { + expectRedirectWithFlash( + "/admin/built-sites/7/maintenance", + "Could not save: provider failed", + false, + )(result(7, { error: "provider failed", ok: false })); + }); +}); + +describeWithEnv("built-site actions", { db: true }, () => { + test("loads the site and parsed form for an owner", async () => { + const site = await insertScheduledTestSite(); + const { cookie, csrfToken } = await getTestSession(); + const route = builtSiteAction((loaded, form, id) => + Promise.resolve( + Response.json({ + id, + name: loaded.name, + value: form.getString("value"), + }), + ), + ); + + const response = await route( + mockFormRequest( + `/admin/built-sites/${site.id}/action`, + { csrf_token: csrfToken, value: "saved" }, + cookie, + ), + { id: site.id }, + ); + + expect(await response.json()).toEqual({ + id: site.id, + name: "Child", + value: "saved", + }); + }); + + test("rejects an invalid CSRF token before running the action", async () => { + const site = await insertScheduledTestSite(); + const { cookie } = await getTestSession(); + let called = false; + const route = builtSiteAction(() => { + called = true; + return Promise.resolve(new Response()); + }); + + const response = await route( + mockFormRequest( + `/admin/built-sites/${site.id}/action`, + { csrf_token: "wrong" }, + cookie, + ), + { id: site.id }, + ); + + expect(response.status).toBe(403); + expect(await response.text()).toBe("CSRF token invalid"); + expect(called).toBe(false); + }); +}); diff --git a/test/features/admin/built-site-page/scheduler.test.ts b/test/features/admin/built-site-page/scheduler.test.ts new file mode 100644 index 000000000..2c729483c --- /dev/null +++ b/test/features/admin/built-site-page/scheduler.test.ts @@ -0,0 +1,75 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { describeWithEnv } from "#test-utils/db.ts"; +import { + insertScheduledTestSite, + SCHEDULED_OWNER_ENV, + TEST_SCHEDULED_KEY, +} from "#test-utils/scheduled.ts"; +import { adminGet } from "#test-utils/session.ts"; + +describeWithEnv( + "built-site maintenance page", + { db: true, env: SCHEDULED_OWNER_ENV }, + () => { + test("shows every built-site tab with its exact label and URL", async () => { + const site = await insertScheduledTestSite(); + + const response = await adminGet(`/admin/built-sites/${site.id}`); + const html = await response.text(); + + expect(response.status).toBe(200); + expect(html).toContain( + `Builds`, + ); + expect(html).toContain( + `href="/admin/built-sites/${site.id}/renewal">Renewal`, + ); + expect(html).toContain( + `href="/admin/built-sites/${site.id}/maintenance">Scheduled maintenance`, + ); + expect(html).toContain( + `href="/admin/built-sites/${site.id}/secrets">Secrets`, + ); + expect(html).toContain( + `href="/admin/built-sites/${site.id}/update">Software update`, + ); + }); + + test("shows the exact delete action on the actions tab", async () => { + const site = await insertScheduledTestSite(); + + const response = await adminGet(`/admin/built-sites/${site.id}/actions`); + const html = await response.text(); + + expect(response.status).toBe(200); + expect(html).toContain(`href="/admin/built-sites/${site.id}/delete"`); + expect(html).toContain("Delete this site"); + }); + + test("shows the child key without rotation controls", async () => { + const site = await insertScheduledTestSite(); + + const response = await adminGet( + `/admin/built-sites/${site.id}/maintenance`, + ); + const html = await response.text(); + + expect(response.status).toBe(200); + expect(response.headers.get("cache-control")).toBe("private, no-store"); + expect(html).toContain(TEST_SCHEDULED_KEY); + expect(html).not.toContain("stage-scheduler"); + expect(html).not.toContain("promote-scheduler"); + }); + + test("offers setup when a child has no retained key", async () => { + const site = await insertScheduledTestSite(null); + + const response = await adminGet( + `/admin/built-sites/${site.id}/maintenance`, + ); + + expect(await response.text()).toContain("provision-scheduler"); + }); + }, +); diff --git a/test/admin-built-sites-actions.test.ts b/test/features/admin/built-sites/actions.test.ts similarity index 88% rename from test/admin-built-sites-actions.test.ts rename to test/features/admin/built-sites/actions.test.ts index 75c9e2f20..803d56983 100644 --- a/test/admin-built-sites-actions.test.ts +++ b/test/features/admin/built-sites/actions.test.ts @@ -6,6 +6,7 @@ import { FakeTime } from "@std/testing/time"; import { handleRequest } from "#routes"; import { bunnyCdnApi } from "#shared/bunny-cdn.ts"; import { addMonthsIso } from "#shared/dates.ts"; +import type { BuiltSite } from "#shared/db/built-sites/types.ts"; import { builtSites, updateBuiltSiteRenewalState, @@ -24,9 +25,7 @@ import { adminFormPost, testCookie } from "#test-utils/session.ts"; const NOW_MS = 1_700_000_000_000; -const findSite = async ( - siteId: number, -): Promise => +const findSite = async (siteId: number): Promise => (await builtSites.getAll()).find((s) => s.id === siteId)!; type SecretStub = any; @@ -126,6 +125,25 @@ describeWithEnv( expect(secretStub.calls.length).toBe(0); }; + const reSyncDeadline = async ( + hostingId: string, + name: string, + readOnlyFrom: string, + provisioned: boolean, + ) => { + const site = await createTestBuiltSite({ hostingId, name }); + if (provisioned) await provisionTestBuiltSite(site.id); + await updateBuiltSiteRenewalState(site.id, { readOnlyFrom }); + resetSecretStub(); + + const { response } = await siteAction(site, "re-sync-deadline"); + await expectFlashRedirect( + `/admin/built-sites/${site.id}/renewal`, + "Deadline re-synced", + )(response); + return secretNamesOf(secretStub); + }; + describe("POST /admin/built-sites/:id/rotate-renewal-token", () => { test("rotates token on a provisioned site and pushes new RENEWAL_URL", async () => { const site = await createTestBuiltSite({ @@ -174,6 +192,23 @@ describeWithEnv( expect(secretStub.calls.length).toBe(0); }); + + test("shows the exact error when the replacement token cannot be pushed", async () => { + const site = await createTestBuiltSite({ + hostingId: "6002", + name: "Rotate Failure", + }); + await provisionTestBuiltSite(site.id); + + await withFailingSecretStub(async () => { + const { response } = await siteAction(site, "rotate-renewal-token"); + await expectFlashRedirect( + `/admin/built-sites/${site.id}/renewal`, + "Renewal token could not be pushed to the site", + false, + )(response); + }); + }); }); describe("POST /admin/built-sites/:id/bump-deadline", () => { @@ -187,13 +222,21 @@ describeWithEnv( readOnlyFrom: new Date(NOW_MS + 10 * 86400000).toISOString(), }); - await adminFormPost(`/admin/built-sites/${site.id}/bump-deadline`, { - months: "3", - }); + const { response } = await adminFormPost( + `/admin/built-sites/${site.id}/bump-deadline`, + { months: "3" }, + ); + await expectFlashRedirect( + `/admin/built-sites/${site.id}/renewal`, + "Deadline bumped", + )(response); const updated = await findSite(site.id); const expectedBase = new Date(NOW_MS + 10 * 86400000).toISOString(); expect(updated.readOnlyFrom).toBe(addMonthsIso(expectedBase, 3)); + expect( + (await getAllActivityLog()).map(({ message }) => message), + ).toContain("Admin bumped 'Bump Future' deadline by 3 month(s)"); }); test("bumps from now on expired site", async () => { @@ -274,6 +317,9 @@ describeWithEnv( test("clamps non-numeric months to 1", () => expectBumpClamps("6017", "Bump NaN", "abc", 1)); + + test("treats a hexadecimal month value as zero", () => + expectBumpClamps("6018", "Bump Hex", "0x10", 1)); }); describe("POST /admin/built-sites/:id/override-deadline", () => { @@ -287,10 +333,18 @@ describeWithEnv( `/admin/built-sites/${site.id}/override-deadline`, { date: "2027-06-15" }, ); - expect(response.status).toBe(302); + await expectFlashRedirect( + `/admin/built-sites/${site.id}/renewal`, + "Deadline updated", + )(response); const updated = await findSite(site.id); expect(updated.readOnlyFrom).toBe("2027-06-15T23:59:59Z"); + expect( + (await getAllActivityLog()).map(({ message }) => message), + ).toContain( + "Admin overrode 'Override Site' deadline to 2027-06-15T23:59:59Z", + ); }); test("works without a renewal token", async () => { @@ -340,47 +394,28 @@ describeWithEnv( describe("POST /admin/built-sites/:id/re-sync-deadline", () => { test("re-pushes stored deadline and RENEWAL_URL when provisioned", async () => { - const site = await createTestBuiltSite({ - hostingId: "6030", - name: "Resync Site", - }); - await provisionTestBuiltSite(site.id); - await updateBuiltSiteRenewalState(site.id, { - readOnlyFrom: "2027-03-15T00:00:00Z", - }); - - resetSecretStub(); - - const { response } = await adminFormPost( - `/admin/built-sites/${site.id}/re-sync-deadline`, + const secretNames = await reSyncDeadline( + "6030", + "Resync Site", + "2027-03-15T00:00:00Z", + true, ); - await expectFlashRedirect( - `/admin/built-sites/${site.id}/renewal`, - "Deadline re-synced", - )(response); - const secretNames = secretNamesOf(secretStub); expect(secretNames).toContain("READ_ONLY_FROM"); expect(secretNames).toContain("RENEWAL_URL"); + expect( + (await getAllActivityLog()).map(({ message }) => message), + ).toContain("Admin re-synced deadline for 'Resync Site'"); }); test("re-pushes deadline without RENEWAL_URL when unprovisioned", async () => { - const site = await createTestBuiltSite({ - hostingId: "6031", - name: "Resync Unprovisioned", - }); - await updateBuiltSiteRenewalState(site.id, { - readOnlyFrom: "2027-04-01T00:00:00Z", - }); - - resetSecretStub(); - - const { response } = await adminFormPost( - `/admin/built-sites/${site.id}/re-sync-deadline`, + const secretNames = await reSyncDeadline( + "6031", + "Resync Unprovisioned", + "2027-04-01T00:00:00Z", + false, ); - expect(response.status).toBe(302); - const secretNames = secretNamesOf(secretStub); expect(secretNames).toContain("READ_ONLY_FROM"); expect(secretNames).not.toContain("RENEWAL_URL"); }); @@ -422,7 +457,10 @@ describeWithEnv( `/admin/built-sites/${site.id}/provision-renewal`, { months: "3" }, ); - expect(response.status).toBe(302); + await expectFlashRedirect( + `/admin/built-sites/${site.id}/renewal`, + "Renewal provisioned", + )(response); const updated = await findSite(site.id); expect(updated.renewalTokenIndex).not.toBeNull(); @@ -433,6 +471,9 @@ describeWithEnv( mockRequest(`/renew/?t=${encodeURIComponent(updated.renewalToken!)}`), ); expect(renewResponse.status).toBe(200); + expect( + (await getAllActivityLog()).map(({ message }) => message), + ).toContain("Admin provisioned renewals for 'Provision Site' (3mo)"); }); test("rejects when no qualifying tier listing exists", async () => { @@ -596,7 +637,7 @@ describeWithEnv( ); await expectFlashRedirect( `/admin/built-sites/${site.id}/secrets`, - expect.stringContaining("missing secret(s)"), + expect.stringContaining(", "), )(response); const setNames = secrets.setCalls.map((c) => c.name); diff --git a/test/features/admin/built-sites/scheduler.test.ts b/test/features/admin/built-sites/scheduler.test.ts new file mode 100644 index 000000000..cdddee1e5 --- /dev/null +++ b/test/features/admin/built-sites/scheduler.test.ts @@ -0,0 +1,73 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { stub } from "@std/testing/mock"; +import { bunnyHostingProvider } from "#shared/bunny-cdn.ts"; +import { builtSitesCrudTable } from "#shared/db/built-sites.ts"; +import { isScheduledTaskKey } from "#shared/scheduled-keys.ts"; +import { expectFlash } from "#test-utils/assertions.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; +import { stubFetch } from "#test-utils/fetch-stub.ts"; +import { restoreStubsAfterEach } from "#test-utils/mocks.ts"; +import { + insertScheduledTestSite, + SCHEDULED_OWNER_ENV, + stubBunnySchedulerSecrets, +} from "#test-utils/scheduled.ts"; +import { adminFormPost } from "#test-utils/session.ts"; + +describeWithEnv( + "built-site scheduler action", + { db: true, env: SCHEDULED_OWNER_ENV }, + () => { + const stubs: { restore(): void }[] = []; + restoreStubsAfterEach(stubs); + + test("provisions and verifies one child key", async () => { + const site = await insertScheduledTestSite(null); + stubBunnySchedulerSecrets(stubs, [], { ok: true }); + using _fetch = stubFetch(new Response(null, { status: 204 })); + + const { response } = await adminFormPost( + `/admin/built-sites/${site.id}/provision-scheduler`, + ); + + expectFlash(response, "Scheduled maintenance key sent to the site."); + const key = (await builtSitesCrudTable.findById(site.id)) + ?.scheduledTaskKey; + expect(isScheduledTaskKey(key ?? "")).toBe(true); + }); + + test("retries a failed provider write with the retained key", async () => { + const site = await insertScheduledTestSite(null); + const pushed: string[] = []; + stubs.push( + stub(bunnyHostingProvider, "getSecretNames", () => + Promise.resolve({ ok: true, value: [] }), + ), + stub(bunnyHostingProvider, "setSecrets", (_hostingId, secrets) => { + pushed.push(secrets[0]![1]); + return Promise.resolve( + pushed.length === 1 + ? { error: "provider failed", ok: false as const } + : { ok: true as const, value: undefined }, + ); + }), + ); + using _fetch = stubFetch(new Response(null, { status: 204 })); + + const { response: failed } = await adminFormPost( + `/admin/built-sites/${site.id}/provision-scheduler`, + ); + expectFlash(failed, "provider failed", false); + const retained = (await builtSitesCrudTable.findById(site.id)) + ?.scheduledTaskKey; + + const { response: retried } = await adminFormPost( + `/admin/built-sites/${site.id}/provision-scheduler`, + ); + + expectFlash(retried, "Scheduled maintenance key sent to the site."); + expect(pushed).toEqual([retained, retained]); + }); + }, +); diff --git a/test/integration/server/built-sites.test.ts b/test/features/admin/built-sites/server.test.ts similarity index 93% rename from test/integration/server/built-sites.test.ts rename to test/features/admin/built-sites/server.test.ts index a21238821..3944341f5 100644 --- a/test/integration/server/built-sites.test.ts +++ b/test/features/admin/built-sites/server.test.ts @@ -17,7 +17,6 @@ import { updateTestBuiltSite, } from "#test-utils/db-helpers/built-sites.ts"; import { withEnv } from "#test-utils/env.ts"; -import { testBuiltSite } from "#test-utils/factories.ts"; import { awaitTestRequest } from "#test-utils/mocks.ts"; import { adminFormPost, @@ -600,50 +599,6 @@ describeWithEnv("server (admin built sites)", builtSitesTestEnv, () => { }); }); - describe("builtSiteToFieldValues", () => { - test("returns empty defaults when no site provided", async () => { - const { builtSiteToFieldValues } = await import( - "#templates/admin/built-sites.tsx" - ); - const values = builtSiteToFieldValues(); - expect(values.name).toBe(""); - expect(values.site_url).toBe(""); - expect(values.db_url).toBe(""); - expect(values.db_token).toBe(""); - expect(values.hosting_id).toBe(""); - expect(values.assignable).toBe(""); - }); - - test("returns site values when site provided", async () => { - const { builtSiteToFieldValues } = await import( - "#templates/admin/built-sites.tsx" - ); - const site = testBuiltSite({ - dbToken: "tok123", - dbUrl: "libsql://test.turso.io", - hostingId: "42", - name: "Test", - siteUrl: "https://test.b-cdn.net", - }); - const values = builtSiteToFieldValues(site); - expect(values.name).toBe("Test"); - expect(values.site_url).toBe("https://test.b-cdn.net"); - expect(values.db_url).toBe("libsql://test.turso.io"); - expect(values.db_token).toBe("tok123"); - expect(values.hosting_id).toBe("42"); - expect(values.assignable).toBe(""); - }); - - test("returns assignable=1 for assignable site", async () => { - const { builtSiteToFieldValues } = await import( - "#templates/admin/built-sites.tsx" - ); - const site = testBuiltSite({ assignable: true }); - const values = builtSiteToFieldValues(site); - expect(values.assignable).toBe("1"); - }); - }); - describe("update channel", () => { test("create form offers the update-channel selector", async () => { const response = await adminGet("/admin/built-sites/new"); diff --git a/test/integration/server/built-sites-update.test.ts b/test/features/admin/built-sites/update.test.ts similarity index 97% rename from test/integration/server/built-sites-update.test.ts rename to test/features/admin/built-sites/update.test.ts index f82be769d..5cc38aa57 100644 --- a/test/integration/server/built-sites-update.test.ts +++ b/test/features/admin/built-sites/update.test.ts @@ -19,7 +19,7 @@ import { type EnvScope, withEnv } from "#test-utils/env.ts"; import { type TempPath, tempDir } from "#test-utils/files.ts"; import { stubReleaseFetch } from "#test-utils/mocks.ts"; import { adminFormPost, testCookie } from "#test-utils/session.ts"; -import { useLocalStoragePath } from "../../lib/_shared-site-update.ts"; +import { useLocalStoragePath } from "../../../lib/_shared-site-update.ts"; /** A built site database URL whose backups land in a site-specific folder. */ const SITE_DB_URL = "libsql://01ABC-client-site.lite.bunnydb.net"; @@ -37,7 +37,7 @@ const expectNoHostingIdError = async (siteId: number): Promise => { ); await expectFlashRedirect( `/admin/built-sites/${siteId}/update`, - expect.stringContaining("no hosting ID"), + "This site has no hosting ID, so it can't be updated.", false, )(response); }; @@ -247,7 +247,7 @@ describeWithEnv( await seedSiteBackup(SITE_DB_URL); using _fetch = stubReleaseFetch(); const deployStub = stub(denoDeployApi, "deployCode", () => - Promise.resolve({ ok: true as const, value: "https://app.deno.dev" }), + Promise.resolve({ ok: true as const, value: undefined }), ); const getEnvVarNamesStub = stub(denoDeployApi, "getEnvVarNames", () => Promise.resolve({ ok: true as const, value: [] }), diff --git a/test/features/admin/debug/server.test.ts b/test/features/admin/debug/server.test.ts index 1bb3c12bf..bfbf0c5c4 100644 --- a/test/features/admin/debug/server.test.ts +++ b/test/features/admin/debug/server.test.ts @@ -614,21 +614,6 @@ describeWithEnv("server (admin debug)", { db: true }, () => { }); }); - describe("Database pruning section", () => { - test("renders the most recent prune timestamp as ISO", async () => { - // Set a recent timestamp so the request-handler's fire-and-forget - // maybeRunPrunes() sees it as not-yet-due and doesn't overwrite. - const ts = Date.now() - 1000; - await settings.update.lastPrunedPayments(String(ts)); - await settings.update.lastPrunedSessions(String(ts)); - await settings.update.lastPrunedLogins(String(ts)); - await settings.update.lastPrunedStrings(String(ts)); - await settings.update.lastPrunedAddresses(String(ts)); - const expected = new Date(ts).toISOString(); - await assertAdminHtml("/admin/debug", "Database pruning", expected); - }); - }); - describe("Limits section", () => { test("shows limits table with all entries", async () => { const html = await assertAdminHtml("/admin/debug", "Limits"); diff --git a/test/lib/server-settings/settings-page.test.ts b/test/features/admin/settings-page/page.test.ts similarity index 50% rename from test/lib/server-settings/settings-page.test.ts rename to test/features/admin/settings-page/page.test.ts index 9e7432dbe..1ac7f53b2 100644 --- a/test/lib/server-settings/settings-page.test.ts +++ b/test/features/admin/settings-page/page.test.ts @@ -1,5 +1,7 @@ import { expect } from "@std/expect"; import { afterEach, describe, it as test } from "@std/testing/bdd"; +import type { bunnyCdnApi } from "#shared/bunny-cdn.ts"; +import { buildFlashCookie } from "#shared/cookies.ts"; import { settings } from "#shared/db/settings.ts"; import { setDemoModeForTest } from "#shared/demo/mode.ts"; import { @@ -11,7 +13,8 @@ import { } from "#test-utils/assertions.ts"; import { describeWithEnv } from "#test-utils/db.ts"; import { withEnv } from "#test-utils/env.ts"; -import { awaitTestRequest } from "#test-utils/mocks.ts"; +import { awaitTestRequest, withMockBunnyCdnApi } from "#test-utils/mocks.ts"; +import { secureAdminCookie, secureAdminGet } from "#test-utils/secure-admin.ts"; import { adminGet, testCookie } from "#test-utils/session.ts"; describeWithEnv("server (admin settings)", { db: true }, () => { @@ -34,6 +37,35 @@ describeWithEnv("server (admin settings)", { db: true }, () => { return expectHtml(response, opts); }; + const advancedPageWithResult = async (result: string): Promise => { + const flashCookie = buildFlashCookie( + FLASH_TEST_ID, + "Subdomain available", + true, + result, + ).split(";")[0]!; + const response = await awaitTestRequest( + `/admin/settings-advanced?flash=${FLASH_TEST_ID}`, + { cookie: `${await testCookie()}; ${flashCookie}` }, + ); + expect(response.status).toBe(200); + return await response.text(); + }; + + const withCdnHostname = async ( + result: Awaited>, + body: () => Promise, + ): Promise => { + using _env = withEnv({ + BUNNY_API_KEY: "bunny-key", + BUNNY_SCRIPT_ID: "script-id", + }); + await withMockBunnyCdnApi( + { getCdnHostname: () => Promise.resolve(result) }, + body, + ); + }; + describe("GET /admin/settings", () => { testRequiresAuth("/admin/settings"); @@ -42,6 +74,28 @@ describeWithEnv("server (admin settings)", { db: true }, () => { await expectHtmlResponse(response, 200, "Settings", "Change Password"); }); + test("selects the exact empty payment state", async () => { + const html = await (await adminGet("/admin/settings")).text(); + expect(html).toMatch( + //, + ); + expect(html).toContain("None (payments disabled)"); + }); + + test("shows that an empty Square webhook key is not configured", async () => { + await settings.update.paymentProvider("square"); + await settings.update.square.accessToken("square-token"); + + const html = await (await adminGet("/admin/settings")).text(); + + expect(html).toContain( + "No webhook signature key is configured. Follow the steps above to set one up.", + ); + expect(html).not.toContain( + "A webhook signature key is currently configured.", + ); + }); + test("shows a flash with no form target as a page-level banner", async () => { const response = await awaitTestRequest( `/admin/settings?flash=${FLASH_TEST_ID}`, @@ -131,6 +185,146 @@ describeWithEnv("server (admin settings)", { db: true }, () => { ); }); + test("shows exact empty host settings and scheduled-key state", async () => { + using _env = withEnv({ + APPLE_WALLET_PASS_TYPE_ID: undefined, + BUNNY_API_KEY: undefined, + BUNNY_DNS_ZONE_ID: undefined, + BUNNY_SCRIPT_ID: undefined, + GOOGLE_WALLET_ISSUER_ID: undefined, + HOST_EMAIL_PROVIDER: undefined, + SCHEDULED_TASK_KEY: undefined, + }); + settings.appleWallet.setHostConfigForTest(null); + settings.googleWallet.setHostConfigForTest(null); + try { + const html = await (await adminGet("/admin/settings-advanced")).text(); + expect(html).toContain(">None (disabled)"); + expect(html).toContain( + "No scheduled maintenance key is set on this site.", + ); + expect(html).not.toContain("Currently using:"); + } finally { + settings.appleWallet.resetHostConfig(); + settings.googleWallet.resetHostConfig(); + } + }); + + test("shows the exact configured wallet host labels", async () => { + settings.appleWallet.setHostConfigForTest({ + passTypeId: "pass.com.host.tickets", + signingCert: "cert-data", + signingKey: "key-data", + teamId: "HOSTTEAM01", + wwdrCert: "wwdr-data", + }); + settings.googleWallet.setHostConfigForTest({ + issuerId: "3388000000012345678", + serviceAccountEmail: "wallet@example.com", + serviceAccountKey: "key-data", + }); + try { + const html = await (await adminGet("/admin/settings-advanced")).text(); + expect(html).toContain( + "Currently using: Host env (pass.com.host.tickets).", + ); + expect(html).toContain( + "Currently using: Host env (3388000000012345678).", + ); + } finally { + settings.appleWallet.resetHostConfig(); + settings.googleWallet.resetHostConfig(); + } + }); + + test("shows the exact DNS suffix without a pending preview", async () => { + using _env = withEnv({ + BUNNY_API_KEY: "bunny-key", + BUNNY_DNS_SUBDOMAIN_SUFFIX: ".tickets.test", + BUNNY_DNS_ZONE_ID: "zone-id", + BUNNY_SCRIPT_ID: undefined, + }); + + const html = await (await adminGet("/admin/settings-advanced")).text(); + + expect(html).toContain('.tickets.test'); + expect(html).not.toContain("is available."); + }); + + test("splits an exact subdomain preview from its full domain", async () => { + using _env = withEnv({ + BUNNY_API_KEY: "bunny-key", + BUNNY_DNS_SUBDOMAIN_SUFFIX: ".tickets.test", + BUNNY_DNS_ZONE_ID: "zone-id", + BUNNY_SCRIPT_ID: undefined, + }); + + const html = await advancedPageWithResult( + "my-site\nmy-site.tickets.test", + ); + + expect(html).toContain("my-site.tickets.test"); + expect(html).toContain( + '', + ); + }); + + test("uses an empty full domain when a preview result has one line", async () => { + using _env = withEnv({ + BUNNY_API_KEY: "bunny-key", + BUNNY_DNS_ZONE_ID: "zone-id", + BUNNY_SCRIPT_ID: undefined, + }); + + const html = await advancedPageWithResult("my-site"); + + expect(html).toContain(""); + expect(html).toContain( + '', + ); + }); + + test("shows an unvalidated custom domain and exact empty CDN target", () => + withCdnHostname({ error: "unavailable", ok: false }, async () => { + await settings.update.customDomain("tickets.example.com"); + + const html = await (await adminGet("/admin/settings-advanced")).text(); + + expect(html).toContain("Your custom domain is not yet validated."); + expect(html).toContain("Value: "); + expect(html).not.toContain("Last validated:"); + })); + + test("shows the exact validated custom domain and CDN target", () => + withCdnHostname({ hostname: "site.b-cdn.net", ok: true }, async () => { + const cookie = await secureAdminCookie(); + await settings.update.customDomain("tickets.example.com"); + await settings.update.customDomainLastValidated(); + const validatedAt = settings.customDomainLastValidated; + + const response = await secureAdminGet( + "/admin/settings-advanced", + "tickets.example.com", + cookie, + ); + const html = await response.text(); + + expect(response.status).toBe(200); + expect(html).toContain("tickets.example.com"); + expect(html).toContain( + "Value: site.b-cdn.net", + ); + expect(html).toContain(`Last validated: ${validatedAt}`); + })); + + test("keeps an empty custom domain empty when Bunny CDN is configured", () => + withCdnHostname({ hostname: "site.b-cdn.net", ok: true }, async () => { + const html = await (await adminGet("/admin/settings-advanced")).text(); + + expect(html).not.toContain("Your custom domain is not yet validated."); + expect(html).not.toContain("Last validated:"); + })); + test("shows warning about careful changes", async () => { const response = await adminGet("/admin/settings-advanced"); const html = await response.text(); diff --git a/test/features/admin/settings-page/scheduled-key.test.ts b/test/features/admin/settings-page/scheduled-key.test.ts new file mode 100644 index 000000000..70caccc23 --- /dev/null +++ b/test/features/admin/settings-page/scheduled-key.test.ts @@ -0,0 +1,36 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { handleRequest } from "#routes"; +import { describeWithEnv } from "#test-utils/db.ts"; +import { mockRequest } from "#test-utils/mocks.ts"; +import { + SCHEDULED_OWNER_ENV, + TEST_SCHEDULED_KEY, +} from "#test-utils/scheduled.ts"; +import { adminGet, createTestManagerSession } from "#test-utils/session.ts"; + +describeWithEnv( + "scheduled key owner settings", + { db: true, env: SCHEDULED_OWNER_ENV }, + () => { + test("shows the local active key on the owner advanced page", async () => { + const response = await adminGet("/admin/settings-advanced"); + + expect(response.status).toBe(200); + expect(response.headers.get("cache-control")).toBe("private, no-store"); + expect(await response.text()).toContain(TEST_SCHEDULED_KEY); + }); + + test("does not show the local key to a manager", async () => { + const managerCookie = await createTestManagerSession(); + const response = await handleRequest( + mockRequest("/admin/settings-advanced", { + headers: { cookie: managerCookie }, + }), + ); + + expect(response.status).toBe(403); + expect(await response.text()).not.toContain(TEST_SCHEDULED_KEY); + }); + }, +); diff --git a/test/features/app/read-only.test.ts b/test/features/app/read-only.test.ts index c5581f28b..ceddc0497 100644 --- a/test/features/app/read-only.test.ts +++ b/test/features/app/read-only.test.ts @@ -28,6 +28,7 @@ describe("read-only request guard", () => { }); test("blocks admin writes unless the route schema allows them", () => { + expect(readOnlyBlock("/admin", "POST")).toBe("page"); expect(readOnlyBlock("/admin/settings/email", "POST")).toBe("page"); expect(readOnlyBlock("/admin/logout", "POST")).toBeNull(); expect(readOnlyBlock("/admin/backup/create", "POST")).toBeNull(); @@ -50,6 +51,7 @@ describe("read-only request guard", () => { }); test("blocks every other public write by default", () => { + expect(readOnlyBlock("/scheduled", "POST")).toBe("page"); expect(readOnlyBlock("/ticket/listing", "POST")).toBe("page"); expect(readOnlyBlock("/read-only", "POST")).toBe("page"); expect(readOnlyBlock("/unknown", "DELETE")).toBe("page"); diff --git a/test/features/app/request.test.ts b/test/features/app/request.test.ts index 580c0af29..769f79130 100644 --- a/test/features/app/request.test.ts +++ b/test/features/app/request.test.ts @@ -10,11 +10,13 @@ import { } from "#shared/db/settings.ts"; import { getHeader } from "#test-utils/assertions.ts"; import { describeWithEnv } from "#test-utils/db.ts"; +import { createTestListing } from "#test-utils/db-helpers/listings.ts"; import { withEnv } from "#test-utils/env.ts"; import { setupErrorSpy } from "#test-utils/error-spy.ts"; import { mockFormRequest, mockRequest } from "#test-utils/mocks.ts"; import { recordQueries } from "#test-utils/record-queries.ts"; import { testCookie } from "#test-utils/session.ts"; +import { enablePublicSite } from "#test-utils/settings.ts"; const deleteSetting = async (key: string): Promise => { const { getDb } = await import("#shared/db/client.ts"); @@ -28,6 +30,34 @@ const deleteSetting = async (key: string): Promise => { describeWithEnv("request pipeline", { db: true }, () => { const errors = setupErrorSpy(); + test("runs an ordinary page through routing and response security", async () => { + await enablePublicSite(); + const response = await handleRequest(mockRequest("/")); + expect(response.status).toBe(200); + expect(response.headers.get("x-frame-options")).toBe("DENY"); + expect(response.headers.get("x-content-type-options")).toBe("nosniff"); + expect(response.headers.get("cache-control")).toBe("private, no-store"); + }); + + test("keeps ticket pages embeddable inside the request scopes", async () => { + const listing = await createTestListing({ maxAttendees: 50 }); + const response = await handleRequest( + mockRequest(`/ticket/${listing.slug}`), + ); + expect(response.status).toBe(200); + expect(response.headers.get("x-frame-options")).toBeNull(); + expect(response.headers.get("x-robots-tag")).toBe("index, follow"); + }); + + test("uses the configured payment provider in response security", async () => { + await settings.update.paymentProvider("square"); + await settings.update.square.sandbox(true); + const response = await handleRequest(mockRequest("/")); + expect(response.headers.get("content-security-policy")).toContain( + "https://connect.squareupsandbox.com", + ); + }); + test("rejects a body-bearing POST with no content type", async () => { const request = new Request("http://localhost/admin/login", { body: new Uint8Array([1]), diff --git a/test/features/app/request/organic-maintenance.test.ts b/test/features/app/request/organic-maintenance.test.ts new file mode 100644 index 000000000..c5a4161d2 --- /dev/null +++ b/test/features/app/request/organic-maintenance.test.ts @@ -0,0 +1,26 @@ +// test-groups: run-alone +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { execute, queryOne } from "#shared/db/client.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; +import { setupErrorSpy } from "#test-utils/error-spy.ts"; +import { adminGet } from "#test-utils/session.ts"; + +describeWithEnv("organic maintenance failure isolation", { db: true }, () => { + const errors = setupErrorSpy(); + + test("reports maintenance failure without replacing a successful response", async () => { + await execute("DROP TABLE maintenance_tasks"); + + const response = await adminGet("/admin"); + + expect(response.status).toBe(200); + expect(errors.contains("organic maintenance failed")).toBe(true); + expect(errors.calls.length).toBe(1); + expect( + await queryOne<{ name: string }>( + "SELECT name FROM sqlite_master WHERE type = 'table' AND name = 'maintenance_tasks'", + ), + ).toBeNull(); + }); +}); diff --git a/test/features/app/routes.test.ts b/test/features/app/routes.test.ts index 0dad3b445..d7009b607 100644 --- a/test/features/app/routes.test.ts +++ b/test/features/app/routes.test.ts @@ -1,28 +1,48 @@ import { expect } from "@std/expect"; import { it as test } from "@std/testing/bdd"; import { routeMainApp } from "#routes/app/routes.ts"; +import { signCsrfToken } from "#shared/csrf.ts"; import { setAdminFeatureEnabled } from "#shared/db/admin-features.ts"; +import { builtSites, insertBuiltSite } from "#shared/db/built-sites.ts"; import { settings } from "#shared/db/settings.ts"; import { describeWithEnv } from "#test-utils/db.ts"; +import { provisionTestBuiltSite } from "#test-utils/db-helpers/built-sites.ts"; import { withEnv } from "#test-utils/env.ts"; -import { mockRequest } from "#test-utils/mocks.ts"; +import { mockFormRequest, mockRequest } from "#test-utils/mocks.ts"; import { enablePublicApi, enablePublicSite } from "#test-utils/settings.ts"; -const route = async (path: string, method = "GET"): Promise => { +const routeRequest = async ( + request: Request, + path: string, + method: string, +): Promise => { const response = await routeMainApp({ method, path, - request: mockRequest(path, { method }), + request, server: undefined, }); return response; }; -describeWithEnv("main app router", { db: true }, () => { - test("returns not found for an inherited object property", async () => { - expect((await route("/constructor")).status).toBe(404); - }); +const route = (path: string, method = "GET"): Promise => + routeRequest(mockRequest(path, { method }), path, method); + +const routeRenewal = async (method: string): Promise => { + await insertBuiltSite("Route Renewal Site", "route-renewal.b-cdn.net"); + const site = (await builtSites.getAll()).find( + ({ name }) => name === "Route Renewal Site", + ); + if (!site) throw new Error("Route renewal site not found"); + const { token } = await provisionTestBuiltSite(site.id); + return routeRequest( + mockRequest(`/renew?t=${encodeURIComponent(token)}`, { method }), + "/renew", + method, + ); +}; +describeWithEnv("main app router", { db: true }, () => { test("returns not found for an inherited object property", async () => { expect((await route("/constructor")).status).toBe(404); }); @@ -52,6 +72,10 @@ describeWithEnv("main app router", { db: true }, () => { expect((await route("/listings/extra")).status).toBe(404); }); + test("does not expose scheduled maintenance through the app router", async () => { + expect((await route("/scheduled", "POST")).status).toBe(404); + }); + test("turns a null result from a known prefix into not found", async () => { expect((await route("/custom.css/extra")).status).toBe(404); }); @@ -98,6 +122,35 @@ describeWithEnv("main app router", { db: true }, () => { expect((await route("/contact", "POST")).status).toBe(302); }); + test("routes configured address lookup on its exact path", async () => { + await settings.update.addressLookup.provider("easypostcodes"); + await settings.update.addressLookup.apiKey("test-api-key"); + + expect((await route("/address-lookup")).status).toBe(400); + }); + + test("routes renewal GET requests", async () => { + expect((await routeRenewal("GET")).status).toBe(200); + }); + + test("routes renewal POST requests", async () => { + expect((await routeRenewal("POST")).status).toBe(200); + }); + + test("routes unsubscribe GET requests", async () => { + expect((await route("/unsubscribe")).status).toBe(200); + }); + + test("routes unsubscribe POST requests", async () => { + const request = mockFormRequest("/unsubscribe", { + csrf_token: await signCsrfToken(), + }); + + expect((await routeRequest(request, "/unsubscribe", "POST")).status).toBe( + 302, + ); + }); + test("serves the read-only information page only for GET", async () => { expect((await route("/read-only")).status).toBe(200); expect((await route("/read-only", "HEAD")).status).toBe(404); diff --git a/test/lib/i18n-route-loading.test.ts b/test/features/app/routes/messages.test.ts similarity index 97% rename from test/lib/i18n-route-loading.test.ts rename to test/features/app/routes/messages.test.ts index c6cf74f10..b4dead200 100644 --- a/test/lib/i18n-route-loading.test.ts +++ b/test/features/app/routes/messages.test.ts @@ -143,10 +143,13 @@ describeWithEnv("route message loading", { db: true }, () => { const { handleRequest } = await import("#routes"); const response = await handleRequest( - mockRequest("/scheduled", { method: "POST" }), + mockRequest("/scheduled", { + headers: { "content-type": "application/x-www-form-urlencoded" }, + method: "POST", + }), ); - expect(response.status).toBe(200); + expect(response.status).toBe(404); expect(() => t("builder.site_builder_title")).toThrow( 'Missing translation for key "builder.site_builder_title"', ); diff --git a/test/features/app/rules.test.ts b/test/features/app/rules.test.ts index a1e725715..d3473d9d6 100644 --- a/test/features/app/rules.test.ts +++ b/test/features/app/rules.test.ts @@ -1,14 +1,15 @@ +// test-groups: run-alone - this suite verifies isolate-lived wake throttling. import { expect } from "@std/expect"; import { describe, it as test } from "@std/testing/bdd"; import { bufferRequestIfNeeded, ensureCustomCssResponse, isSetupPath, + runOrganicMaintenanceWhenDue, shouldBufferRequestBody, shouldLogQueries, shouldPrefetchSettings, shouldRetryBusyRequest, - shouldRunPrunes, trackingRedirectLocation, } from "#routes/app/rules.ts"; @@ -61,10 +62,60 @@ describe("request rules", () => { expect(shouldLogQueries("GET", "ticket")).toBe(false); }); - test("skips background prunes only for the orphan settings POST", () => { - expect(shouldRunPrunes("POST", "/admin/privacy/orphans")).toBe(false); - expect(shouldRunPrunes("GET", "/admin/privacy/orphans")).toBe(true); - expect(shouldRunPrunes("POST", "/admin/privacy/erase")).toBe(true); + test("runs organic maintenance only after safe successful reads", async () => { + const calls: string[] = []; + const run = (name: string) => () => { + calls.push(name); + }; + await runOrganicMaintenanceWhenDue("GET", "/", 200, run("get"), 0); + await runOrganicMaintenanceWhenDue( + "HEAD", + "/admin", + 204, + run("head"), + 60_000, + ); + await runOrganicMaintenanceWhenDue("POST", "/", 200, run("post"), 120_000); + await runOrganicMaintenanceWhenDue("GET", "/", 199, run("early"), 120_000); + await runOrganicMaintenanceWhenDue("GET", "/", 404, run("error"), 120_000); + expect(calls).toEqual(["get", "head"]); + }); + + test("excludes checkout, payment, webhook, and setup paths", async () => { + const calls: string[] = []; + for (const path of [ + "/api/listing/book", + "/calculate/listing", + "/order", + "/pay/token", + "/payment/webhook", + "/renew", + "/sms/webhook", + "/ticket/listing", + "/setup", + ]) { + await runOrganicMaintenanceWhenDue( + "GET", + path, + 200, + () => { + calls.push(path); + }, + 120_000, + ); + } + expect(calls).toEqual([]); + }); + + test("throttles repeated organic wakes in one warm isolate", async () => { + const calls: number[] = []; + const run = (time: number) => () => { + calls.push(time); + }; + await runOrganicMaintenanceWhenDue("GET", "/", 200, run(120_000), 120_000); + await runOrganicMaintenanceWhenDue("GET", "/", 200, run(179_999), 179_999); + await runOrganicMaintenanceWhenDue("GET", "/", 200, run(180_000), 180_000); + expect(calls).toEqual([120_000, 180_000]); }); test("retries busy requests only when they are safe to repeat", () => { diff --git a/test/features/index.test.ts b/test/features/index.test.ts deleted file mode 100644 index 2d30cdcea..000000000 --- a/test/features/index.test.ts +++ /dev/null @@ -1,38 +0,0 @@ -import { expect } from "@std/expect"; -import { it as test } from "@std/testing/bdd"; -import { handleRequest } from "#routes"; -import { settings } from "#shared/db/settings.ts"; -import { describeWithEnv } from "#test-utils/db.ts"; -import { createTestListing } from "#test-utils/db-helpers/listings.ts"; -import { mockRequest } from "#test-utils/mocks.ts"; -import { enablePublicSite } from "#test-utils/settings.ts"; - -describeWithEnv("request entry point", { db: true }, () => { - test("runs an ordinary page through routing and response security", async () => { - await enablePublicSite(); - const response = await handleRequest(mockRequest("/")); - expect(response.status).toBe(200); - expect(response.headers.get("x-frame-options")).toBe("DENY"); - expect(response.headers.get("x-content-type-options")).toBe("nosniff"); - expect(response.headers.get("cache-control")).toBe("private, no-store"); - }); - - test("keeps ticket pages embeddable inside the request scopes", async () => { - const listing = await createTestListing({ maxAttendees: 50 }); - const response = await handleRequest( - mockRequest(`/ticket/${listing.slug}`), - ); - expect(response.status).toBe(200); - expect(response.headers.get("x-frame-options")).toBeNull(); - expect(response.headers.get("x-robots-tag")).toBe("index, follow"); - }); - - test("uses the configured payment provider in response security", async () => { - await settings.update.paymentProvider("square"); - await settings.update.square.sandbox(true); - const response = await handleRequest(mockRequest("/")); - expect(response.headers.get("content-security-policy")).toContain( - "https://connect.squareupsandbox.com", - ); - }); -}); diff --git a/test/lib/server-instance.test.ts b/test/features/instance.test.ts similarity index 83% rename from test/lib/server-instance.test.ts rename to test/features/instance.test.ts index 31945873e..d8a6094b4 100644 --- a/test/lib/server-instance.test.ts +++ b/test/features/instance.test.ts @@ -10,7 +10,8 @@ import { expect } from "@std/expect"; import { describe, it as test } from "@std/testing/bdd"; import { handleRequest } from "#routes"; -import { insertBuiltSite, type UpdateTier } from "#shared/db/built-sites.ts"; +import type { UpdateTier } from "#shared/db/built-sites/types.ts"; +import { insertBuiltSite } from "#shared/db/built-sites.ts"; import { describeWithEnv } from "#test-utils/db.ts"; import { withEnv } from "#test-utils/env.ts"; import { mockRequest } from "#test-utils/mocks.ts"; @@ -71,17 +72,30 @@ describeWithEnv("server (instance site-credentials)", { db: true }, () => { test("returns 404 when MAIN_INSTANCE_KEY is not configured", async () => { const response = await post({ authorization: `Bearer ${KEY}` }); expect(response.status).toBe(404); + expect(await response.json()).toEqual({ error: "not_found" }); }); test("returns 401 when the bearer key is missing", async () => { using _env = withEnv({ MAIN_INSTANCE_KEY: KEY }); - expect((await post()).status).toBe(401); + const response = await post(); + expect(response.status).toBe(401); + expect(await response.json()).toEqual({ error: "unauthorized" }); }); test("returns 401 when the bearer key is wrong", async () => { using _env = withEnv({ MAIN_INSTANCE_KEY: KEY }); const response = await post({ authorization: "Bearer not-the-key" }); expect(response.status).toBe(401); + expect(await response.json()).toEqual({ error: "unauthorized" }); + }); + + test("rejects a key without the exact bearer scheme", async () => { + using _env = withEnv({ MAIN_INSTANCE_KEY: KEY }); + for (const authorization of [KEY, `bearer ${KEY}`, `Token ${KEY}`]) { + const response = await post({ authorization }); + expect(response.status).toBe(401); + expect(await response.json()).toEqual({ error: "unauthorized" }); + } }); test("returns read-only credentials for sites that have them", async () => { @@ -96,6 +110,26 @@ describeWithEnv("server (instance site-credentials)", { db: true }, () => { ); // A half-provisioned site (no script id / credentials) is omitted. await insertBuiltSite("Pending", "pending.b-cdn.net"); + await insertBuiltSite( + "Deno", + "deno.example", + "libsql://deno", + "deno-token", + false, + "deno-app", + "release", + "deno", + "bunny", + ); + await insertBuiltSite("No URL", "no-url.example", "", "token", false, "1"); + await insertBuiltSite( + "No token", + "no-token.example", + "libsql://no-token", + "", + false, + "2", + ); const response = await post({ authorization: `Bearer ${KEY}` }); expect(response.status).toBe(200); diff --git a/test/features/middleware.test.ts b/test/features/middleware.test.ts index b4089b771..ed104e3a7 100644 --- a/test/features/middleware.test.ts +++ b/test/features/middleware.test.ts @@ -10,6 +10,7 @@ import { isJsonApiPath, isValidContentType, isWebhookPath, + lowerContentType, } from "#routes/middleware.ts"; import { resetEffectiveDomain, @@ -121,17 +122,27 @@ describe("isEmbeddablePath", () => { }); describe("isValidContentType", () => { + test("normalizes a missing content type to an empty string", () => { + expect(lowerContentType(requestWithType("POST"))).toBe(""); + }); + test("non-POST requests need no content type", () => { expect(isValidContentType(requestWithType("PATCH"), "/admin/login")).toBe( true, ); }); - for (const path of ["/scheduled", "/instance/site-credentials"]) { - test(`${path} accepts a bodyless POST`, () => { - expect(isValidContentType(requestWithType("POST"), path)).toBe(true); - }); - } + test("normal middleware does not exempt the specialized scheduled route", () => { + expect(isValidContentType(requestWithType("POST"), "/scheduled")).toBe( + false, + ); + }); + + test("/instance/site-credentials accepts a bodyless POST", () => { + expect( + isValidContentType(requestWithType("POST"), "/instance/site-credentials"), + ).toBe(true); + }); for (const path of [ "/payment/webhook", diff --git a/test/features/request-scopes.test.ts b/test/features/request-scopes.test.ts new file mode 100644 index 000000000..254e7b1d2 --- /dev/null +++ b/test/features/request-scopes.test.ts @@ -0,0 +1,83 @@ +import { expect } from "@std/expect"; +import { describe, it as test } from "@std/testing/bdd"; +import { getLocale } from "#i18n"; +import { + requestScopedHandler, + runWithRequestScopes, +} from "#routes/request-scopes.ts"; +import { getRequestClientIp } from "#shared/client-context.ts"; +import { addPendingWork } from "#shared/pending-work.ts"; +import { + BUNNY_SUBREQUEST_LIMIT, + countExternalSubrequest, +} from "#shared/subrequest-budget.ts"; + +describe("request scopes", () => { + test("binds request values while building the response", async () => { + const request = new Request("https://example.com/path"); + const server = { requestIP: () => ({ address: "203.0.113.9" }) }; + + const response = await runWithRequestScopes(request, server, () => + Promise.resolve( + Response.json({ ip: getRequestClientIp(), locale: getLocale() }), + ), + ); + + expect(await response.json()).toEqual({ ip: "203.0.113.9", locale: "en" }); + expect(getRequestClientIp()).toBe("direct"); + }); + + test("passes the request and server through the scoped handler", async () => { + const request = new Request("https://example.com/scoped"); + const server = { requestIP: () => ({ address: "198.51.100.4" }) }; + const handler = requestScopedHandler((receivedRequest, receivedServer) => + Promise.resolve( + Response.json({ + ip: getRequestClientIp(), + requestMatches: receivedRequest === request, + serverMatches: receivedServer === server, + }), + ), + ); + + const response = await handler(request, server); + + expect(await response.json()).toEqual({ + ip: "198.51.100.4", + requestMatches: true, + serverMatches: true, + }); + }); + + test("keeps queued work inside the request subrequest budget", async () => { + let blocked = ""; + // Pending work accepts promises in production. This lazy thenable makes its + // work begin only when the request scope drains the queue. + const queuedWork = { + // biome-ignore lint/suspicious/noThenProperty: the regression requires work that starts during promise assimilation + then: (resolve: () => void): void => { + try { + for (let call = 0; call <= BUNNY_SUBREQUEST_LIMIT; call += 1) { + countExternalSubrequest("queued test work"); + } + } catch (error) { + blocked = String(error); + } + resolve(); + }, + } as unknown as Promise; + + await runWithRequestScopes( + new Request("https://example.com/queued"), + undefined, + () => { + addPendingWork(queuedWork); + return Promise.resolve(new Response()); + }, + ); + + expect(blocked).toContain( + `Subrequest allowance exceeded: 0 database + ${BUNNY_SUBREQUEST_LIMIT + 1} external calls`, + ); + }); +}); diff --git a/test/features/scheduled/server.test.ts b/test/features/scheduled/server.test.ts new file mode 100644 index 000000000..e5ea9c8d3 --- /dev/null +++ b/test/features/scheduled/server.test.ts @@ -0,0 +1,147 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { handleRequest } from "#routes"; +import { settings } from "#shared/db/settings.ts"; +import { initSentry } from "#shared/sentry.ts"; +import { serveHandler } from "#src/serve-app.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; +import { withEnv } from "#test-utils/env.ts"; +import { setupErrorSpy } from "#test-utils/error-spy.ts"; +import { stubFetch } from "#test-utils/fetch-stub.ts"; +import { mockRequest } from "#test-utils/mocks.ts"; +import { + expectScheduledResponse, + scheduledAuthorization, + TEST_SCHEDULED_KEY, +} from "#test-utils/scheduled.ts"; +import { resetSentry } from "#test-utils/sentry.ts"; + +const scheduled = (key = TEST_SCHEDULED_KEY): Promise => + serveHandler( + mockRequest("/scheduled", { + headers: scheduledAuthorization(key), + method: "POST", + }), + ); + +describeWithEnv( + "server (scheduled maintenance)", + { + db: true, + env: { + SCHEDULED_TASK_KEY: TEST_SCHEDULED_KEY, + }, + }, + () => { + const errors = setupErrorSpy(); + + test("runs authenticated local maintenance through the production handler", async () => { + await expectScheduledResponse(await scheduled(), 204); + }); + + test("returns 503 until site setup is complete", async () => { + using _env = withEnv({ + SENTRY_URL: "https://scheduled@bugs.example.test/2", + }); + using fetchStub = stubFetch(new Response("{}", { status: 200 })); + await initSentry(); + try { + await settings.setRaw("setup_complete", "false"); + settings.setup.clearCache(); + settings.invalidateCache(); + + await expectScheduledResponse(await scheduled(), 503); + expect(errors.contains("scheduled maintenance failed")).toBe(true); + const [, options] = fetchStub.calls[0]!.args as [string, RequestInit]; + const body = + typeof options.body === "string" + ? options.body + : new TextDecoder().decode(options.body as Uint8Array); + expect(body).toContain( + "Scheduled maintenance requires completed setup", + ); + } finally { + resetSentry(); + } + }); + + test("does not read an authenticated request body", async () => { + const request = mockRequest("/scheduled", { + body: "ignored", + headers: scheduledAuthorization(), + method: "POST", + }); + await expectScheduledResponse(await serveHandler(request), 204); + expect(request.bodyUsed).toBe(false); + }); + + test("returns a bearer challenge for missing credentials", async () => { + const response = await serveHandler( + mockRequest("/scheduled", { method: "POST" }), + ); + expect(response.headers.get("www-authenticate")).toBe("Bearer"); + await expectScheduledResponse(response, 401); + }); + + test("returns a bearer challenge for malformed credentials", async () => { + const response = await serveHandler( + mockRequest("/scheduled", { + headers: { authorization: `Basic ${TEST_SCHEDULED_KEY}` }, + method: "POST", + }), + ); + expect(response.headers.get("www-authenticate")).toBe("Bearer"); + await expectScheduledResponse(response, 401); + }); + + test("returns a bearer challenge for the wrong key", async () => { + const response = await scheduled("wrong"); + expect(response.headers.get("www-authenticate")).toBe("Bearer"); + await expectScheduledResponse(response, 401); + }); + + test("does not expose the endpoint through the normal app router", async () => { + const response = await handleRequest( + mockRequest("/scheduled", { + headers: { + ...scheduledAuthorization(), + "content-type": "application/x-www-form-urlencoded", + }, + method: "POST", + }), + ); + expect(response.status).toBe(404); + }); + + test("does not make outbound scheduler requests on a builder", async () => { + const originalFetch = globalThis.fetch; + let calls = 0; + globalThis.fetch = () => { + calls += 1; + return Promise.reject(new Error("unexpected scheduler fetch")); + }; + try { + await expectScheduledResponse(await scheduled(), 204); + expect(calls).toBe(0); + } finally { + globalThis.fetch = originalFetch; + } + }); + }, +); + +describeWithEnv( + "server (scheduled maintenance in read-only mode)", + { + db: true, + env: { + READ_ONLY_FROM: "2020-01-01T00:00:00.000Z", + SCHEDULED_TASK_KEY: TEST_SCHEDULED_KEY, + }, + }, + () => { + test("permits an authenticated local run", async () => { + await expectScheduledResponse(await scheduled(), 204); + }); + }, +); diff --git a/test/features/settings-bundles.test.ts b/test/features/settings-bundles.test.ts index f3ac679b9..05fb40cf5 100644 --- a/test/features/settings-bundles.test.ts +++ b/test/features/settings-bundles.test.ts @@ -23,4 +23,10 @@ describe("settings bundles", () => { CONFIG_KEYS.APPLE_WALLET_SIGNING_KEY, ); }); + + test("does not reserve a settings bundle for the early scheduled route", () => { + expect(settingsForPath("/scheduled")).toContain( + CONFIG_KEYS.APPLE_WALLET_SIGNING_KEY, + ); + }); }); diff --git a/test/features/url.test.ts b/test/features/url.test.ts new file mode 100644 index 000000000..65f1f6b97 --- /dev/null +++ b/test/features/url.test.ts @@ -0,0 +1,59 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { + getBaseUrl, + getClientIp, + getSearchParam, + parseCookies, + parseRequest, +} from "../../src/features/url.ts"; + +test("parses request cookies", () => { + const request = new Request("https://example.test", { + headers: { cookie: "first=one; second=two" }, + }); + expect(parseCookies(request)).toEqual( + new Map([ + ["first", "one"], + ["second", "two"], + ]), + ); +}); + +test("uses the direct client address when available", () => { + const request = new Request("https://example.test"); + expect( + getClientIp(request, { + requestIP: () => ({ address: "192.0.2.10", family: "IPv4", port: 42 }), + }), + ).toBe("192.0.2.10"); +}); + +test("uses the direct fallback without a client address", () => { + const request = new Request("https://example.test"); + expect(getClientIp(request)).toBe("direct"); + expect(getClientIp(request, { requestIP: () => null })).toBe("direct"); +}); + +test("extracts the origin from a request", () => { + expect(getBaseUrl(new Request("https://example.test:8443/path"))).toBe( + "https://example.test:8443", + ); +}); + +test("parses method, URL, and a normalized path", () => { + const request = new Request("https://example.test/scheduled///?page=2", { + method: "POST", + }); + const parsed = parseRequest(request); + expect(parsed.method).toBe("POST"); + expect(parsed.path).toBe("/scheduled"); + expect(parsed.url.searchParams.get("page")).toBe("2"); +}); + +test("returns a query value or an empty string", () => { + const request = new Request("https://example.test/?present=value&empty="); + expect(getSearchParam(request, "present")).toBe("value"); + expect(getSearchParam(request, "empty")).toBe(""); + expect(getSearchParam(request, "missing")).toBe(""); +}); diff --git a/test/integration/kek-v2.test.ts b/test/integration/kek-v2.test.ts index b8bae7b83..dbfbe3986 100644 --- a/test/integration/kek-v2.test.ts +++ b/test/integration/kek-v2.test.ts @@ -17,6 +17,7 @@ import { } from "#shared/crypto/keys.ts"; import { signCsrfToken } from "#shared/csrf.ts"; import { getDb, insert } from "#shared/db/client.ts"; +import { runDatabasePruning } from "#shared/db/prune.ts"; import { createSession } from "#shared/db/sessions.ts"; import { settings } from "#shared/db/settings.ts"; import { @@ -26,7 +27,6 @@ import { hashInviteCode, invalidateUsersCache, migrateUserToV2Kek, - pruneExpiredInvites, verifyUserPassword, } from "#shared/db/users.ts"; import type { User } from "#shared/types.ts"; @@ -226,8 +226,7 @@ describeWithEnv("KEK v2 (password-bound DATA_KEY)", { db: true }, () => { await wrapKeyWithToken(dataKey, "valid-code"), ); - const pruned = await pruneExpiredInvites(); - expect(pruned).toBe(1); + await runDatabasePruning(); // The expired invite (and its DATA_KEY handoff) is gone; the valid one // remains so the invitee can still join. expect(await getUserByUsername("expired-invitee")).toBeNull(); @@ -256,8 +255,7 @@ describeWithEnv("KEK v2 (password-bound DATA_KEY)", { db: true }, () => { ); invalidateUsersCache(); - const pruned = await pruneExpiredInvites(); - expect(pruned).toBe(0); + await runDatabasePruning(); expect(await getUserByUsername("has-password")).not.toBeNull(); }); }); diff --git a/test/integration/migration-restore-verify.test.ts b/test/integration/migration-restore-verify.test.ts index 1081e7608..615c24c89 100644 --- a/test/integration/migration-restore-verify.test.ts +++ b/test/integration/migration-restore-verify.test.ts @@ -61,12 +61,14 @@ describeWithEnv( // rebuild), the attendees.kind NOT NULL tightening (an empty-`requires` // constraint rebuild owning no additive objects to drop/restore), and the // attendee-listings-tag settings rewrite (data-only; covered by its own - // data test), listing_image_thumb (historically added a column that + // data test), the historical built-site prune marker, listing_image_thumb + // (historically added a column that // first_class_images now drops), remove_broken_image_records (data-only; // covered by its own data test), and removal-only migrations whose // absent-table checks cannot be rebuilt by a restore case. enabled_features // now owns the triggers that keep saved feature data and visibility in step. - expect(additiveMigrations.length).toBe(MIGRATIONS.length - 19); + // The built-site marker drop is also removal-only. + expect(additiveMigrations.length).toBe(MIGRATIONS.length - 21); }); test("restores triggers attached to a dropped table", async () => { diff --git a/test/integration/migration-round-trip-budget.test.ts b/test/integration/migration-round-trip-budget.test.ts index e3e653e91..5465fc941 100644 --- a/test/integration/migration-round-trip-budget.test.ts +++ b/test/integration/migration-round-trip-budget.test.ts @@ -119,7 +119,7 @@ describeWithEnv("migration request round-trip budget", { db: true }, () => { } } expect(finished).toBe(true); - expect(continuations).toBe(3); + expect(continuations).toBe(4); const result = await getDb().execute({ args: pendingIds, diff --git a/test/integration/questions-attendee-answers.test.ts b/test/integration/questions-attendee-answers.test.ts index 301c93383..7b4e2f7cf 100644 --- a/test/integration/questions-attendee-answers.test.ts +++ b/test/integration/questions-attendee-answers.test.ts @@ -1,7 +1,7 @@ import { expect } from "@std/expect"; import { describe, it as test } from "@std/testing/bdd"; import { execute, queryAll } from "#shared/db/client.ts"; -import { pruneUnusedStrings } from "#shared/db/prune.ts"; +import { runDatabasePruning } from "#shared/db/prune.ts"; import { getAttendeeAnswersBatch, getAttendeeTextAnswers, @@ -316,7 +316,7 @@ describeWithEnv("custom questions", { db: true }, () => { await execute( "UPDATE strings SET created = '2000-01-01T00:00:00Z' WHERE used_count = 0", ); - await pruneUnusedStrings(); + await runDatabasePruning(); expect(await queryAll("SELECT id FROM strings")).toEqual([]); }); diff --git a/test/integration/renewals.test.ts b/test/integration/renewals.test.ts index 7379d2dc1..88c0ee2cc 100644 --- a/test/integration/renewals.test.ts +++ b/test/integration/renewals.test.ts @@ -4,11 +4,8 @@ import { stub } from "@std/testing/mock"; import { FakeTime } from "@std/testing/time"; import { bunnyCdnApi } from "#shared/bunny-cdn.ts"; import { addMonthsIso } from "#shared/dates.ts"; -import { - type BuiltSite, - builtSites, - insertBuiltSite, -} from "#shared/db/built-sites.ts"; +import type { BuiltSite } from "#shared/db/built-sites/types.ts"; +import { builtSites, insertBuiltSite } from "#shared/db/built-sites.ts"; import type { Listing } from "#shared/types.ts"; import { applyRenewalsForEntries } from "#shared/webhook.ts"; import { getAllActivityLog } from "#test-utils/activity-log.ts"; diff --git a/test/integration/routes/read-only.test.ts b/test/integration/routes/read-only.test.ts index 85a652cf8..9115aa9b4 100644 --- a/test/integration/routes/read-only.test.ts +++ b/test/integration/routes/read-only.test.ts @@ -261,7 +261,6 @@ describeWithEnv( "/contact", "/admin/support", "/instance/site-credentials", - "/scheduled", "/admin/backup/create", "/checkin/abc123+def456", "/admin/listing/42/attendee/99/checkin", diff --git a/test/integration/server-balance-webhook.test.ts b/test/integration/server-balance-webhook.test.ts index 0d6f0258a..d1097cb8c 100644 --- a/test/integration/server-balance-webhook.test.ts +++ b/test/integration/server-balance-webhook.test.ts @@ -9,7 +9,7 @@ import { } from "#shared/accounting/queries.ts"; import { getAttendeeBalanceState } from "#shared/db/attendees/balance.ts"; import { execute } from "#shared/db/client.ts"; -import { prunePayments } from "#shared/db/prune.ts"; +import { runDatabasePruning } from "#shared/db/prune.ts"; import { stripeApi } from "#shared/stripe.ts"; import { describeWithEnv } from "#test-utils/db.ts"; import { mockRequest } from "#test-utils/mocks.ts"; @@ -89,7 +89,7 @@ describeWithEnv("server (public balance page) > webhook", { db: true }, () => { "UPDATE processed_payments SET processed_at = ? WHERE payment_session_id = ?", ["2000-01-01T00:00:00.000Z", "cs_balance_replay"], ); - await prunePayments(); + await runDatabasePruning(); // The replay: the balance is already paid (owed 0), so without the ledger // preflight settleAttendeeBalance reports nothing_owed and refunds the diff --git a/test/integration/server/scheduled.test.ts b/test/integration/server/scheduled.test.ts deleted file mode 100644 index fc7948853..000000000 --- a/test/integration/server/scheduled.test.ts +++ /dev/null @@ -1,129 +0,0 @@ -/** - * Tests for the public maintenance-ping endpoint (GET/POST /scheduled). - * - * Pruning runs as interval-gated pending work on every request, so hitting - * /scheduled (like any request) prunes this site. On a builder, POST /scheduled - * also pokes the least-recently-poked built site with a plain GET to trigger - * its prune (the outbound call is stubbed here). There is no auth: the only - * side effects are interval-gated prunes of already-expired data. - */ - -import { expect } from "@std/expect"; -import { it as test } from "@std/testing/bdd"; -import { handleRequest } from "#routes"; -import { insertBuiltSite } from "#shared/db/built-sites.ts"; -import { queryOne } from "#shared/db/client.ts"; -import { describeWithEnv } from "#test-utils/db.ts"; -import { - attendeeExists, - insertOrphanAttendee, -} from "#test-utils/db-helpers/attendees.ts"; -import { stubFetch } from "#test-utils/fetch-stub.ts"; -import { expectFetchSilent, mockRequest } from "#test-utils/mocks.ts"; - -/** GET or POST /scheduled. */ -const scheduled = (method: "GET" | "POST"): Promise => - handleRequest(mockRequest("/scheduled", { method })); - -/** POST /scheduled and assert `poked` is null and no fetch was made. */ -const expectPostScheduledPokesNothing = async (): Promise => { - await expectFetchSilent(async () => { - const response = await scheduled("POST"); - expect((await response.json()).poked).toBe(null); - }); -}; - -/** Insert an orphaned attendee created `days` ago (no listing booking). */ -const insertOldOrphan = (days: number): Promise => - insertOrphanAttendee(days, "sched-orphan"); - -const lastPrunedOf = async (siteId: number): Promise => - ( - await queryOne<{ last_pruned: string }>( - "SELECT last_pruned FROM built_sites WHERE id = ?", - [siteId], - ) - )?.last_pruned ?? ""; - -describeWithEnv( - "server (scheduled tasks): self-prune", - { db: true, env: { CAN_BUILD_SITES: undefined } }, - () => { - test("pinging /scheduled prunes this site", async () => { - const orphanId = await insertOldOrphan(365); - - const response = await scheduled("GET"); - - expect(response.status).toBe(200); - expect(await response.json()).toEqual({ ok: true, poked: null }); - // Per-request pruning (auto-purge on by default) reaped the year-old orphan. - expect(await attendeeExists(orphanId)).toBe(false); - }); - - test("needs no auth — a bare POST is accepted", async () => { - const response = await scheduled("POST"); - expect(response.status).toBe(200); - expect(await response.json()).toEqual({ ok: true, poked: null }); - }); - }, -); - -describeWithEnv( - "server (scheduled tasks): built forwarding", - { db: true, env: { CAN_BUILD_SITES: "true" } }, - () => { - test("POST pokes the least-recently-poked built site with a GET", async () => { - const site = await insertBuiltSite("Client", "client.b-cdn.net"); - using fetchStub = stubFetch(new Response("ok", { status: 200 })); - const response = await scheduled("POST"); - - expect(response.status).toBe(200); - const body = await response.json(); - // No client hostname in the response — this endpoint is public. - expect(body.poked).toEqual({ ok: true, status: 200 }); - - // It poked the client's /scheduled with a plain unauthenticated GET. - expect(fetchStub.calls.length).toBe(1); - const [url, init] = fetchStub.calls[0]!.args as [string, RequestInit]; - expect(url).toBe("https://client.b-cdn.net/scheduled"); - expect(init.method).toBe("GET"); - expect(init.headers).toBeUndefined(); - // Redirects are followed only after SSRF re-validation, never blindly. - expect(init.redirect).toBe("manual"); - - // The site's rotation stamp was bumped so the next call walks onward. - expect(await lastPrunedOf(site.id)).not.toBe(""); - }); - - test("GET does not walk the fleet", async () => { - await insertBuiltSite("Client", "client.b-cdn.net"); - using fetchStub = stubFetch(new Response("ok")); - const response = await scheduled("GET"); - expect((await response.json()).poked).toBe(null); - expect(fetchStub.calls.length).toBe(0); - }); - - test("reports poked null when the builder has no built sites", async () => { - await expectPostScheduledPokesNothing(); - }); - - test("reports an error when the built site is unreachable", async () => { - await insertBuiltSite("Client", "client.b-cdn.net"); - using _fetch = stubFetch(new Error("network down")); - const response = await scheduled("POST"); - // The failure is reported without leaking the client hostname. - expect((await response.json()).poked).toEqual({ failed: true }); - }); - }, -); - -describeWithEnv( - "server (scheduled tasks): not a builder", - { db: true, env: { CAN_BUILD_SITES: undefined } }, - () => { - test("POST does not poke built sites when not a builder", async () => { - await insertBuiltSite("Client", "client.b-cdn.net"); - await expectPostScheduledPokesNothing(); - }); - }, -); diff --git a/test/integration/server/site-assignment.test.ts b/test/integration/server/site-assignment.test.ts index ea08ca5ad..813fcc949 100644 --- a/test/integration/server/site-assignment.test.ts +++ b/test/integration/server/site-assignment.test.ts @@ -4,8 +4,8 @@ import { type Stub, stub } from "@std/testing/mock"; import { type BuildSiteInput, builderApi } from "#shared/builder.ts"; import { bunnyCdnApi } from "#shared/bunny-cdn.ts"; import { addMonthsIso } from "#shared/dates.ts"; +import type { BuiltSite } from "#shared/db/built-sites/types.ts"; import { - type BuiltSite, builtSites, getAssignableBuiltSites, insertBuiltSite, @@ -17,6 +17,7 @@ import { } from "#shared/email.ts"; import { ErrorCode } from "#shared/logger.ts"; import { nowIso } from "#shared/now.ts"; +import { generateScheduledTaskKey } from "#shared/scheduled-keys.ts"; import { assignAndNotifyBuiltSites, parseReadOnlyFromMs, @@ -33,19 +34,25 @@ import { stubFetch } from "#test-utils/fetch-stub.ts"; const stubBuildSiteSuccess = (onCall?: (input: BuildSiteInput) => void) => { let counter = 0; - return stub(builderApi, "buildSite", (input: BuildSiteInput) => { - counter++; - onCall?.(input); - return Promise.resolve({ - dbProvider: "bunny" as const, - dbToken: `token-${counter}`, - dbUrl: `libsql://auto-${counter}.test`, - defaultHostname: `auto-${counter}.b-cdn.net`, - hostingId: String(1000 + counter), - hostingProvider: "bunny" as const, - ok: true as const, - }); - }); + return stub( + builderApi, + "buildSite", + async (input: BuildSiteInput, retain) => { + counter++; + onCall?.(input); + const result = { + dbProvider: "bunny" as const, + dbToken: `token-${counter}`, + dbUrl: `libsql://auto-${counter}.test`, + defaultHostname: `auto-${counter}.b-cdn.net`, + hostingId: String(1000 + counter), + hostingProvider: "bunny" as const, + ok: true as const, + }; + await retain({ ...result, scheduledTaskKey: generateScheduledTaskKey() }); + return result; + }, + ); }; const stubBuildSiteFailure = () => @@ -244,6 +251,27 @@ describeWithEnv( } }); + test("fails if an auto-build succeeds without retaining its site", async () => { + const buildStub = stub(builderApi, "buildSite", () => + Promise.resolve({ + dbProvider: "bunny" as const, + dbToken: "token", + dbUrl: "libsql://auto.test", + defaultHostname: "auto.b-cdn.net", + hostingId: "42", + hostingProvider: "bunny" as const, + ok: true as const, + }), + ); + try { + await expect( + assignAndNotifyBuiltSites([siteEntry()]), + ).rejects.toThrow("Built site was not retained"); + } finally { + buildStub.restore(); + } + }); + test("no-ops for empty entries", async () => { await assignAndNotifyBuiltSites([]); expect(fetchStub.calls.length).toBe(0); diff --git a/test/integration/webhook-price-signature-trusted-and-mismatch.test.ts b/test/integration/webhook-price-signature-trusted-and-mismatch.test.ts index c88e6bd96..320110918 100644 --- a/test/integration/webhook-price-signature-trusted-and-mismatch.test.ts +++ b/test/integration/webhook-price-signature-trusted-and-mismatch.test.ts @@ -8,7 +8,7 @@ import { listingChildren } from "#shared/db/listing-parents.ts"; import { deleteListing } from "#shared/db/listings/delete.ts"; import { listingsTable } from "#shared/db/listings/records.ts"; import { isSessionProcessed } from "#shared/db/processed-payments.ts"; -import { prunePayments } from "#shared/db/prune.ts"; +import { runDatabasePruning } from "#shared/db/prune.ts"; import { assertJson } from "#test-utils/assertions.ts"; import { describeWithEnv } from "#test-utils/db.ts"; import { createTestListing } from "#test-utils/db-helpers/listings.ts"; @@ -33,7 +33,7 @@ const pruneReplayRowWithoutRefundReference = async (sessionId: string) => { WHERE payment_session_id = ?`, ["2000-01-01T00:00:00.000Z", sessionId], ); - await prunePayments(); + await runDatabasePruning(); expect(await isSessionProcessed(sessionId)).toBe(null); }; diff --git a/test/integration/with-transaction.test.ts b/test/integration/with-transaction.test.ts index 1717a851e..b426bb63a 100644 --- a/test/integration/with-transaction.test.ts +++ b/test/integration/with-transaction.test.ts @@ -13,6 +13,7 @@ import { enableQueryLog, getQueryLog, runWithQueryLogContext, + setN1GuardNotifyOnly, TRANSACTION_ROUNDTRIP_THRESHOLD, } from "#shared/db/query-log.ts"; import { @@ -92,17 +93,22 @@ describe("withTransaction", () => { // A chatty interactive transaction (many sequential round-trips holding the // write lock) is the "Transaction timed-out" shape; the guard fails it loudly // in dev/test so it gets restructured into a batch. - await withFileDb(async () => { - await expect( - runWithQueryLogContext(async () => { - await withTransaction(async (tx) => { - for (let i = 0; i <= TRANSACTION_ROUNDTRIP_THRESHOLD; i++) { - await tx.execute("INSERT INTO t VALUES (1)"); - } - }); - }), - ).rejects.toThrow(/Interactive transaction too chatty/); - }); + setN1GuardNotifyOnly(false); + try { + await withFileDb(async () => { + await expect( + runWithQueryLogContext(async () => { + await withTransaction(async (tx) => { + for (let i = 0; i <= TRANSACTION_ROUNDTRIP_THRESHOLD; i++) { + await tx.execute("INSERT INTO t VALUES (1)"); + } + }); + }), + ).rejects.toThrow(/Interactive transaction too chatty/); + }); + } finally { + setN1GuardNotifyOnly(null); + } }); test("fires cache invalidation for each written statement after the commit", async () => { diff --git a/test/lib/db/migration-restore/helpers.ts b/test/lib/db/migration-restore/helpers.ts index 9de714e7d..df0874350 100644 --- a/test/lib/db/migration-restore/helpers.ts +++ b/test/lib/db/migration-restore/helpers.ts @@ -342,6 +342,9 @@ const columnsRemovedByMigration: Partial> = { "2026-07-05_first_class_images": [ "ALTER TABLE listings ADD COLUMN image_url TEXT NOT NULL DEFAULT ''", ], + "2026-07-18_drop_built_sites_last_pruned": [ + "ALTER TABLE built_sites ADD COLUMN last_pruned TEXT NOT NULL DEFAULT ''", + ], }; /** Wind the live schema back to just before these migrations ran. */ diff --git a/test/lib/server-public/listings-query-scaling.test.ts b/test/lib/server-public/listings-query-scaling.test.ts index 1eb071ffd..414f02b17 100644 --- a/test/lib/server-public/listings-query-scaling.test.ts +++ b/test/lib/server-public/listings-query-scaling.test.ts @@ -78,7 +78,9 @@ describeWithEnv( { db: true, triggers: true }, () => { test("checks all regular groups with one batched classification", async () => { - const first = await recordGroupPageQueries("group", 0, 1); + const firstNames = await addGroupPageFixtures("group", 0, 1); + await recordListingsPage(firstNames); + const first = await recordListingsPage(firstNames); const seen = await recordGroupPageQueries("group", 1, 3); const childLinkBatches = seen.filter((sql) => diff --git a/test/lib/serve-app.test.ts b/test/serve-app.test.ts similarity index 64% rename from test/lib/serve-app.test.ts rename to test/serve-app.test.ts index 4f857284a..9695d5ad6 100644 --- a/test/lib/serve-app.test.ts +++ b/test/serve-app.test.ts @@ -11,6 +11,7 @@ import { expect } from "@std/expect"; import { describe, it as test } from "@std/testing/bdd"; import { stub } from "@std/testing/mock"; +import { getEffectiveDomain } from "#shared/config.ts"; import { N_PLUS_ONE_THRESHOLD, runWithQueryLogContext, @@ -20,13 +21,88 @@ import { setSuppressDebugLogs } from "#shared/log-settings.ts"; import { devServerPort, serveHandler } from "#src/serve-app.ts"; import { describeWithEnv } from "#test-utils/db.ts"; import { withEnv } from "#test-utils/env.ts"; +import { stubFetch } from "#test-utils/fetch-stub.ts"; import { withExpectedError } from "#test-utils/mocks.ts"; +import { + expectScheduledResponse, + scheduledAuthorization, + TEST_SCHEDULED_KEY, +} from "#test-utils/scheduled.ts"; const request = (path: string): Request => new Request(`http://localhost${path}`, { headers: { host: "localhost" } }); describeWithEnv("serve-app", { db: true }, () => { describe("serveHandler", () => { + test("rejects an unset scheduled endpoint before broken boot and Sentry", async () => { + using _env = withEnv({ + MAIN_INSTANCE_KEY: "too-short", + SCHEDULED_TASK_KEY: undefined, + SENTRY_URL: "https://abc123@bugs.example.test/2", + }); + using fetchStub = stubFetch(new Error("Sentry must not start")); + + const response = await serveHandler( + new Request("http://localhost/scheduled", { method: "POST" }), + ); + + await expectScheduledResponse(response, 404); + expect(fetchStub.calls.length).toBe(0); + }); + + test("rejects a wrong scheduled key without reading the body", async () => { + using _env = withEnv({ + MAIN_INSTANCE_KEY: "too-short", + SCHEDULED_TASK_KEY: TEST_SCHEDULED_KEY, + }); + const request = new Request("http://localhost/scheduled", { + body: "caller-selected work", + headers: { + ...scheduledAuthorization("wrong"), + "content-type": "application/json", + }, + method: "POST", + }); + + const response = await serveHandler(request); + + expect(response.headers.get("www-authenticate")).toBe("Bearer"); + await expectScheduledResponse(response, 401); + expect(request.bodyUsed).toBe(false); + }); + + test("hides every non-POST scheduled method before boot", async () => { + using _env = withEnv({ + MAIN_INSTANCE_KEY: "too-short", + SCHEDULED_TASK_KEY: TEST_SCHEDULED_KEY, + }); + const response = await serveHandler( + new Request("http://localhost/scheduled", { + headers: scheduledAuthorization(), + method: "GET", + }), + ); + + await expectScheduledResponse(response, 404); + }); + + test("returns an empty scheduled 503 when authorized boot fails", async () => { + using _env = withEnv({ + MAIN_INSTANCE_KEY: "too-short", + SCHEDULED_TASK_KEY: TEST_SCHEDULED_KEY, + }); + await withExpectedError(async () => { + const response = await serveHandler( + new Request("https://scheduled-site.example/scheduled", { + headers: scheduledAuthorization(), + method: "POST", + }), + ); + await expectScheduledResponse(response, 503); + expect(getEffectiveDomain()).toBe("scheduled-site.example"); + }); + }); + test("a failing boot check returns 503 and does not poison later boots", async () => { // MAIN_INSTANCE_KEY must be unset or ≥32 bytes; a short one fails the // boot checks inside the handler, which must answer with the generic diff --git a/test/shared/admin-surface/routes/built-sites.test.ts b/test/shared/admin-surface/routes/built-sites.test.ts new file mode 100644 index 000000000..150d66fda --- /dev/null +++ b/test/shared/admin-surface/routes/built-sites.test.ts @@ -0,0 +1,28 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { ADMIN_SURFACE } from "#shared/admin-surface.ts"; +import { jsonHash } from "#test-utils/hash.ts"; + +test("keeps the complete built-site route catalog exact", async () => { + const routes = ADMIN_SURFACE.routes.filter( + ({ area }) => area === "builtSites", + ); + + expect(await jsonHash(routes)).toBe( + "ad669c386d0694be463bbe43624a6d7bc0cd6364043d9b27e540f9a6db43a6a6", + ); +}); + +test("declares the scheduler provisioning action as a blocked POST route", () => { + expect( + ADMIN_SURFACE.routes.filter(({ id }) => id.includes("Scheduler")), + ).toEqual([ + { + area: "builtSites", + id: "postBuiltSitesByIdProvisionScheduler", + method: "POST", + pattern: "/admin/built-sites/:id/provision-scheduler", + readOnly: "block", + }, + ]); +}); diff --git a/test/shared/boot-checks.test.ts b/test/shared/boot-checks.test.ts index 2a9b687c5..eebd10bdc 100644 --- a/test/shared/boot-checks.test.ts +++ b/test/shared/boot-checks.test.ts @@ -13,6 +13,7 @@ describeWithEnv("boot checks", { encryptionKey: true }, () => { expect(BOOT_CHECKS.map((check) => check.name)).toEqual([ "DB_ENCRYPTION_KEY", "MAIN_INSTANCE_KEY", + "SCHEDULED_TASK_KEY", ]); }); @@ -67,4 +68,11 @@ describeWithEnv("boot checks", { encryptionKey: true }, () => { "MAIN_INSTANCE_KEY must be at least 32 bytes when set, got 9 bytes", ); }); + + test("runs scheduled key validation during boot checks", () => { + using _env = withEnv({ SCHEDULED_TASK_KEY: "invalid" }); + expect(() => validateBootChecks()).toThrow( + "SCHEDULED_TASK_KEY must be canonical unpadded base64url for exactly 32 bytes", + ); + }); }); diff --git a/test/shared/builder.test.ts b/test/shared/builder.test.ts index 7c1b513dc..970d188bc 100644 --- a/test/shared/builder.test.ts +++ b/test/shared/builder.test.ts @@ -26,6 +26,9 @@ const BUILD_INPUT = { siteName: "Test", } as const; +const buildSite = (input: Parameters[0]) => + builderApi.buildSite(input, () => Promise.resolve()); + type Restorable = { restore(): void }; /** Each error path: install mocks, call buildSite, expect a matching failure. */ @@ -152,7 +155,7 @@ describeWithEnv( for (const { name, mocks, input, error } of ERROR_CASES) { test(name, async () => { await withMocks(mocks, async () => { - const result = await builderApi.buildSite(input); + const result = await buildSite(input); expectBuildError(result, error); }); }); @@ -160,7 +163,7 @@ describeWithEnv( test("buildSite copies host secrets when env vars are set", () => withBuildSiteMocks(async ({ secretStub }) => { - const result = await builderApi.buildSite(BUILD_INPUT); + const result = await buildSite(BUILD_INPUT); expect(result.ok).toBe(true); const secretsSet = secretsFrom(secretStub); @@ -172,7 +175,7 @@ describeWithEnv( test("buildSite succeeds with all steps", () => withBuildSiteMocks( async ({ createStub, updatePzStub, publishStub, secretStub }) => { - const result = await builderApi.buildSite({ + const result = await buildSite({ ...BUILD_INPUT, siteName: "My Site", }); @@ -208,7 +211,7 @@ describeWithEnv( test("buildSite auto-creates database when dbUrl is not provided", () => withBuildSiteMocks(async ({ createDbStub, secretStub }) => { - const result = await builderApi.buildSite({ siteName: "Auto Site" }); + const result = await buildSite({ siteName: "Auto Site" }); expect(result.ok).toBe(true); expect(createDbStub.calls.length).toBe(1); @@ -226,7 +229,7 @@ describeWithEnv( test("buildSite uses provided dbUrl and dbToken without calling createDatabase", () => withBuildSiteMocks(async ({ createDbStub }) => { - await builderApi.buildSite({ + await buildSite({ dbToken: "provided-token", dbUrl: "libsql://provided.io", siteName: "Provided", @@ -236,7 +239,7 @@ describeWithEnv( test("buildSite uses provided code without fetching from GitHub", () => withBuildSiteMocks(async ({ createStub, fetchStub }) => { - const result = await builderApi.buildSite({ + const result = await buildSite({ code: "console.log('local-bundle')", siteName: "Local", }); @@ -251,7 +254,7 @@ describeWithEnv( test("buildSite uses empty string dbToken when dbUrl provided without dbToken", () => withBuildSiteMocks(async ({ createDbStub, secretStub }) => { - await builderApi.buildSite({ + await buildSite({ dbUrl: "libsql://provided.io", siteName: "NoToken", }); @@ -263,7 +266,7 @@ describeWithEnv( withMocks( () => stubDenoBuilderApis(), async ({ deployStub }) => { - const result = await builderApi.buildSite({ + const result = await buildSite({ dbToken: "tok", dbUrl: "libsql://test.turso.io", hostingProvider: "deno", @@ -323,7 +326,7 @@ describeWithEnv( for (const { name, mocks, error } of DENO_ERROR_CASES) { test(name, () => withMocks(mocks, async () => { - const result = await builderApi.buildSite({ + const result = await buildSite({ dbToken: "tok", dbUrl: "libsql://test.io", hostingProvider: "deno", @@ -389,7 +392,7 @@ describeWithEnv( ), }), async ({ tursoStub }) => { - const result = await builderApi.buildSite({ + const result = await buildSite({ dbProvider: "turso", siteName: "Turso Auto", }); @@ -411,7 +414,7 @@ describeWithEnv( await withMocks( () => stubDenoBuilderApis(), async ({ setEnvStub }) => { - const result = await builderApi.buildSite({ + const result = await buildSite({ dbToken: "tok", dbUrl: "libsql://test.turso.io", hostingProvider: "deno", diff --git a/test/shared/builder/access.test.ts b/test/shared/builder/access.test.ts new file mode 100644 index 000000000..5991c3392 --- /dev/null +++ b/test/shared/builder/access.test.ts @@ -0,0 +1,31 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { siteHostingAccess } from "#shared/builder.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; + +describeWithEnv("builder hosting access", {}, () => { + test("rejects a site with no hosting ID", () => { + expect( + siteHostingAccess( + { hostingId: "", hostingProvider: "bunny" }, + "its secrets can't be read", + ), + ).toEqual({ + error: "This site has no hosting ID, so its secrets can't be read.", + ok: false, + }); + }); + + test("rejects a site when its provider key is missing", () => { + expect( + siteHostingAccess( + { hostingId: "42", hostingProvider: "bunny" }, + "its secrets can't be read", + ), + ).toEqual({ + error: + "BUNNY_API_KEY is not configured on this host, so its secrets can't be read.", + ok: false, + }); + }); +}); diff --git a/test/shared/builder/host-secrets.test.ts b/test/shared/builder/host-secrets.test.ts new file mode 100644 index 000000000..a945f407c --- /dev/null +++ b/test/shared/builder/host-secrets.test.ts @@ -0,0 +1,56 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { collectHostSecrets, HOST_INFRA_SECRET_KEYS } from "#shared/builder.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; + +test("lists every high-privilege host credential", () => { + expect(HOST_INFRA_SECRET_KEYS).toEqual([ + "STORAGE_ZONE_NAME", + "STORAGE_ZONE_KEY", + "HOST_EMAIL_API_KEY", + "BUNNY_API_KEY", + "BUNNY_DNS_ZONE_ID", + "APPLE_WALLET_SIGNING_CERT", + "APPLE_WALLET_SIGNING_KEY", + "APPLE_WALLET_WWDR_CERT", + "GOOGLE_WALLET_SERVICE_ACCOUNT_KEY", + ]); +}); + +describeWithEnv( + "builder host secrets by provider", + { + env: { + BUNNY_API_KEY: "host-key", + BUNNY_DNS_SUBDOMAIN_SUFFIX: ".tickets", + BUNNY_DNS_ZONE_ID: "zone-1", + NTFY_URL: "https://ntfy.example.com/t", + }, + }, + () => { + test("copies the exact allowed configured values", () => { + const relevantNames = new Set([ + "BUNNY_API_KEY", + "BUNNY_DNS_SUBDOMAIN_SUFFIX", + "BUNNY_DNS_ZONE_ID", + "NTFY_URL", + ]); + const selected = (provider: "bunny" | "deno") => + Object.fromEntries( + collectHostSecrets(provider).filter(([name]) => + relevantNames.has(name), + ), + ); + + expect({ bunny: selected("bunny"), deno: selected("deno") }).toEqual({ + bunny: { + BUNNY_API_KEY: "host-key", + BUNNY_DNS_SUBDOMAIN_SUFFIX: ".tickets", + BUNNY_DNS_ZONE_ID: "zone-1", + NTFY_URL: "https://ntfy.example.com/t", + }, + deno: { NTFY_URL: "https://ntfy.example.com/t" }, + }); + }); + }, +); diff --git a/test/shared/builder/scheduler.test.ts b/test/shared/builder/scheduler.test.ts new file mode 100644 index 000000000..e64879c01 --- /dev/null +++ b/test/shared/builder/scheduler.test.ts @@ -0,0 +1,113 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { stub } from "@std/testing/mock"; +import { builderApi, type PreparedBuildSite } from "#shared/builder.ts"; +import { bunnyHostingProvider } from "#shared/bunny-cdn.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; +import { restoreStubsAfterEach } from "#test-utils/mocks.ts"; + +const input = { + code: "addEventListener('fetch', () => {})", + dbToken: "token", + dbUrl: "libsql://child.example.test", + siteName: "Child", +} as const; + +describeWithEnv("builder scheduled keys", { encryptionKey: true }, () => { + const stubs: { restore(): void }[] = []; + restoreStubsAfterEach(stubs); + + const stubHosting = (calls: string[]): void => { + stubs.push( + stub(bunnyHostingProvider, "prepareSite", (_name, _code, secrets) => { + calls.push(`prepare:${secrets.map(([name]) => name).join(",")}`); + return Promise.resolve({ + ok: true, + value: { + defaultHostname: "child.b-cdn.net", + hostingId: "42", + }, + }); + }), + stub(bunnyHostingProvider, "publishSite", () => { + calls.push("publish"); + return Promise.resolve({ ok: true, value: undefined }); + }), + ); + }; + + test("creates a distinct 256-bit key for every child", async () => { + const calls: string[] = []; + const retained: PreparedBuildSite[] = []; + stubHosting(calls); + + await builderApi.buildSite(input, (site) => { + retained.push(site); + return Promise.resolve(); + }); + await builderApi.buildSite({ ...input, siteName: "Other" }, (site) => { + retained.push(site); + return Promise.resolve(); + }); + + expect(retained.length).toBe(2); + expect(retained[0]!.scheduledTaskKey).toMatch(/^[A-Za-z0-9_-]{43}$/); + expect(retained[1]!.scheduledTaskKey).not.toBe( + retained[0]!.scheduledTaskKey, + ); + }); + + test("retains the generated key before publishing", async () => { + const calls: string[] = []; + stubHosting(calls); + + const result = await builderApi.buildSite(input, (site) => { + calls.push(`retain:${site.scheduledTaskKey.length}`); + return Promise.resolve(); + }); + + expect(result.ok).toBe(true); + expect(calls).toEqual([ + "prepare:DB_URL,DB_TOKEN,DB_ENCRYPTION_KEY,SCHEDULED_TASK_KEY", + "retain:43", + "publish", + ]); + }); + + test("does not publish when durable retention fails", async () => { + const calls: string[] = []; + stubHosting(calls); + + const result = await builderApi.buildSite(input, () => + Promise.reject(new Error("database write failed")), + ); + + expect(result).toEqual({ + error: "Failed to retain site: database write failed", + ok: false, + }); + expect(calls.some((call) => call === "publish")).toBe(false); + }); + + test("keeps the durable record when publishing fails", async () => { + const calls: string[] = []; + const retained: PreparedBuildSite[] = []; + stubHosting(calls); + stubs.at(-1)!.restore(); + stubs.pop(); + stubs.push( + stub(bunnyHostingProvider, "publishSite", () => + Promise.resolve({ error: "publish failed", ok: false }), + ), + ); + + const result = await builderApi.buildSite(input, (site) => { + retained.push(site); + return Promise.resolve(); + }); + + expect(result.ok).toBe(false); + expect(retained.length).toBe(1); + expect(retained[0]!.scheduledTaskKey.length).toBe(43); + }); +}); diff --git a/test/shared/bunny-cdn/domain.test.ts b/test/shared/bunny-cdn/domain.test.ts new file mode 100644 index 000000000..120073a4d --- /dev/null +++ b/test/shared/bunny-cdn/domain.test.ts @@ -0,0 +1,352 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { stub } from "@std/testing/mock"; +import { buildSubdomainRecordName, bunnyCdnApi } from "#shared/bunny-cdn.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; +import { setupErrorSpy } from "#test-utils/error-spy.ts"; +import { stubFetch } from "#test-utils/fetch-stub.ts"; +import { restoreStubsAfterEach } from "#test-utils/mocks.ts"; + +const ENV = { + BUNNY_API_KEY: "test-key", + BUNNY_DNS_SUBDOMAIN_SUFFIX: ".tickets", + BUNNY_DNS_ZONE_ID: "7", + BUNNY_SCRIPT_ID: "42", +}; + +const zone = ( + records: { Id: number; Name: string; Type: number; Value: string }[] = [], +) => ({ + Domain: "example.test", + Id: 7, + Records: records, +}); + +type Restorable = { restore(): void }; + +const stubPullZoneId = ( + stubs: Restorable[], + result: { id: number; ok: true } | { error: string; ok: false } = { + id: 9, + ok: true, + }, +): void => { + stubs.push( + stub(bunnyCdnApi, "findPullZoneId", () => Promise.resolve(result)), + ); +}; + +const stubAvailableSubdomain = ( + stubs: Restorable[], + target: { hostname: string; ok: true } | { error: string; ok: false } = { + hostname: "child.b-cdn.net", + ok: true, + }, +): void => { + stubs.push( + stub(bunnyCdnApi, "checkSubdomainAvailable", () => + Promise.resolve({ + available: true, + fullDomain: "child.tickets.example.test", + ok: true, + }), + ), + stub(bunnyCdnApi, "getCdnHostname", () => Promise.resolve(target)), + ); +}; + +const noContent = (): Response => new Response(null, { status: 204 }); + +const stubCustomDomainRequests = ( + first = noContent(), +): ReturnType => stubFetch(first, noContent(), noContent()); + +describeWithEnv("Bunny domain API", { env: ENV }, () => { + const stubs: Restorable[] = []; + restoreStubsAfterEach(stubs); + const errors = setupErrorSpy(); + + test("loads a DNS zone with the configured credential", async () => { + using fetchStub = stubFetch(new Response(JSON.stringify(zone()))); + + expect(await bunnyCdnApi.getDnsZone()).toEqual({ ok: true, zone: zone() }); + const [url, init] = fetchStub.calls[0]!.args as [string, RequestInit]; + expect(url).toBe("https://api.bunny.net/dnszone/7"); + expect(init.headers).toEqual({ AccessKey: "test-key" }); + expect(init.method).toBeUndefined(); + }); + + test("labels a DNS zone provider failure", async () => { + using _fetch = stubFetch(new Response("failed", { status: 500 })); + + expect(await bunnyCdnApi.getDnsZone()).toEqual({ + error: "Get DNS zone failed (500): failed", + ok: false, + }); + }); + + test("builds and checks the full subdomain name", async () => { + stubs.push( + stub(bunnyCdnApi, "getDnsZone", () => + Promise.resolve({ + ok: true, + zone: zone([ + { Id: 1, Name: "used.tickets", Type: 2, Value: "target" }, + ]), + }), + ), + ); + + expect(buildSubdomainRecordName("used")).toBe("used.tickets"); + expect(await bunnyCdnApi.checkSubdomainAvailable("used")).toEqual({ + available: false, + fullDomain: "used.tickets.example.test", + ok: true, + }); + expect(await bunnyCdnApi.checkSubdomainAvailable("free")).toEqual({ + available: true, + fullDomain: "free.tickets.example.test", + ok: true, + }); + }); + + test("returns a DNS lookup failure before checking records", async () => { + stubs.push( + stub(bunnyCdnApi, "getDnsZone", () => + Promise.resolve({ error: "zone failed", ok: false }), + ), + ); + expect(await bunnyCdnApi.checkSubdomainAvailable("free")).toEqual({ + error: "zone failed", + ok: false, + }); + }); + + test("registers a custom hostname and enables forced SSL", async () => { + stubPullZoneId(stubs); + using fetchStub = stubCustomDomainRequests(); + + expect( + await bunnyCdnApi.validateCustomDomain("custom.example.test"), + ).toEqual({ ok: true }); + const calls = fetchStub.calls.map( + ({ args }) => args as [string, RequestInit], + ); + expect(calls.map(([url]) => url)).toEqual([ + "https://api.bunny.net/pullzone/9/addHostname", + "https://api.bunny.net/pullzone/loadFreeCertificate?hostname=custom.example.test", + "https://api.bunny.net/pullzone/9/setForceSSL", + ]); + expect(calls.map(([, init]) => init.method)).toEqual([ + "POST", + "GET", + "POST", + ]); + expect(JSON.parse(calls[0]![1].body as string)).toEqual({ + Hostname: "custom.example.test", + }); + expect(JSON.parse(calls[2]![1].body as string)).toEqual({ + ForceSSL: true, + Hostname: "custom.example.test", + }); + expect(new Headers(calls[0]![1].headers).get("content-type")).toBe( + "application/json", + ); + }); + + test("continues when the custom hostname is already registered", async () => { + stubPullZoneId(stubs); + using _fetch = stubCustomDomainRequests( + new Response( + JSON.stringify({ + ErrorKey: "pullzone.hostname_already_registered", + Message: "already registered", + }), + { status: 400 }, + ), + ); + + expect( + await bunnyCdnApi.validateCustomDomain("custom.example.test"), + ).toEqual({ ok: true }); + }); + + test("reports a pull-zone lookup failure", async () => { + stubPullZoneId(stubs, { error: "zone failed", ok: false }); + + expect( + await bunnyCdnApi.validateCustomDomain("custom.example.test"), + ).toEqual({ error: "zone failed", ok: false }); + expect(errors.contains("zone failed")).toBe(true); + }); + + for (const [label, responses, expected] of [ + [ + "hostname", + [new Response("add failed", { status: 500 })], + "Add hostname failed (500): add failed", + ], + [ + "certificate", + [ + new Response(null, { status: 204 }), + new Response("cert failed", { status: 500 }), + ], + "Load free certificate failed (500): cert failed", + ], + [ + "forced SSL", + [ + new Response(null, { status: 204 }), + new Response(null, { status: 204 }), + new Response("ssl failed", { status: 500 }), + ], + "Set force SSL failed (500): ssl failed", + ], + ] as const) { + test(`reports a ${label} setup failure`, async () => { + stubPullZoneId(stubs); + using _fetch = stubFetch(responses[0], ...responses.slice(1)); + + const result = await bunnyCdnApi.validateCustomDomain( + "custom.example.test", + ); + + expect(result).toMatchObject({ error: expected, ok: false }); + expect(errors.contains(expected)).toBe(true); + }); + } + + test("registers a CNAME with the exact DNS payload", async () => { + stubAvailableSubdomain(stubs); + stubs.push( + stub(bunnyCdnApi, "validateCustomDomain", () => + Promise.resolve({ ok: true }), + ), + ); + using fetchStub = stubFetch(new Response(JSON.stringify({ Id: 12 }))); + + expect(await bunnyCdnApi.registerBunnySubdomain("child")).toEqual({ + fullDomain: "child.tickets.example.test", + ok: true, + }); + const [url, init] = fetchStub.calls[0]!.args as [string, RequestInit]; + expect(url).toBe("https://api.bunny.net/dnszone/7/records"); + expect(init.method).toBe("PUT"); + expect(JSON.parse(init.body as string)).toEqual({ + Name: "child.tickets", + Ttl: 300, + Type: 2, + Value: "child.b-cdn.net", + }); + }); + + test("rejects a taken subdomain before provider writes", async () => { + stubs.push( + stub(bunnyCdnApi, "checkSubdomainAvailable", () => + Promise.resolve({ + available: false, + fullDomain: "used.tickets.example.test", + ok: true, + }), + ), + ); + using fetchStub = stubFetch(() => { + throw new Error("Unexpected provider request"); + }); + + expect(await bunnyCdnApi.registerBunnySubdomain("used")).toEqual({ + error: 'Subdomain "used" is already taken', + ok: false, + }); + expect(fetchStub.calls.length).toBe(0); + }); + + test("reports a CDN target lookup failure", async () => { + stubAvailableSubdomain(stubs, { error: "target failed", ok: false }); + + expect(await bunnyCdnApi.registerBunnySubdomain("child")).toEqual({ + error: "target failed", + ok: false, + }); + expect(errors.contains("target failed")).toBe(true); + }); + + test("reports a DNS record write failure", async () => { + stubAvailableSubdomain(stubs); + using _fetch = stubFetch(new Response("write failed", { status: 500 })); + + expect(await bunnyCdnApi.registerBunnySubdomain("child")).toEqual({ + error: "Add DNS CNAME record failed (500): write failed", + ok: false, + }); + expect(errors.contains("Add DNS CNAME record failed")).toBe(true); + }); + + test("retries certificate setup with bounded backoff and cleans up", async () => { + const delays: number[] = []; + const validations: string[] = []; + const deleted: [string, number][] = []; + stubAvailableSubdomain(stubs); + stubs.push( + stub(bunnyCdnApi, "validateCustomDomain", (hostname) => { + validations.push(hostname); + return Promise.resolve({ error: "certificate pending", ok: false }); + }), + stub(bunnyCdnApi, "delay", (milliseconds) => { + delays.push(milliseconds); + return Promise.resolve(); + }), + stub(bunnyCdnApi, "deleteDnsRecord", (zoneId, recordId) => { + deleted.push([zoneId, recordId]); + return Promise.resolve({ ok: true }); + }), + ); + using _fetch = stubFetch(new Response(JSON.stringify({ Id: 12 }))); + + expect(await bunnyCdnApi.registerBunnySubdomain("child")).toEqual({ + error: "certificate pending", + ok: false, + }); + expect(validations).toEqual(Array(5).fill("child.tickets.example.test")); + expect(delays).toEqual([5_000, 10_000, 15_000, 20_000]); + expect(deleted).toEqual([["7", 12]]); + }); + + test("does not clean up an unknown DNS record id", async () => { + const deleted: number[] = []; + stubAvailableSubdomain(stubs); + stubs.push( + stub(bunnyCdnApi, "validateCustomDomain", () => + Promise.resolve({ error: "certificate pending", ok: false }), + ), + stub(bunnyCdnApi, "delay", () => Promise.resolve()), + stub(bunnyCdnApi, "deleteDnsRecord", (_zoneId, recordId) => { + deleted.push(recordId); + return Promise.resolve({ ok: true }); + }), + ); + using _fetch = stubFetch(new Response("not json")); + + await bunnyCdnApi.registerBunnySubdomain("child"); + + expect(deleted).toEqual([]); + }); + + test("deletes a DNS record with the exact provider request", async () => { + using fetchStub = stubFetch(new Response(null, { status: 204 })); + + expect(await bunnyCdnApi.deleteDnsRecord("7", 12)).toEqual({ ok: true }); + const [url, init] = fetchStub.calls[0]!.args as [string, RequestInit]; + expect(url).toBe("https://api.bunny.net/dnszone/7/records/12"); + expect(init.method).toBe("DELETE"); + }); + + test("labels a DNS record delete failure", async () => { + using _fetch = stubFetch(new Response("failed", { status: 500 })); + expect(await bunnyCdnApi.deleteDnsRecord("7", 12)).toEqual({ + error: "Delete DNS record failed (500): failed", + ok: false, + }); + }); +}); diff --git a/test/shared/bunny-cdn/edge-script.test.ts b/test/shared/bunny-cdn/edge-script.test.ts index a4f728df1..8cce3d997 100644 --- a/test/shared/bunny-cdn/edge-script.test.ts +++ b/test/shared/bunny-cdn/edge-script.test.ts @@ -175,7 +175,7 @@ describeWithEnv( }), ), ), - async () => { + async (fetchStub) => { const result = await bunnyCdnApi.createEdgeScript( "Test Script", "console.log('test')", @@ -186,6 +186,18 @@ describeWithEnv( pullZoneId: 99, scriptId: 42, }); + const [url, init] = fetchStub.calls[0]!.args as [string, RequestInit]; + expect(url).toBe("https://api.bunny.net/compute/script"); + expect(init.method).toBe("POST"); + expect(new Headers(init.headers).get("content-type")).toBe( + "application/json", + ); + expect(JSON.parse(init.body as string)).toEqual({ + Code: "console.log('test')", + CreateLinkedPullZone: true, + Name: "Test Script", + ScriptType: 1, + }); }, ); }); @@ -244,6 +256,15 @@ describeWithEnv( expect(String(fetchStub.calls[1]!.args[0])).toContain( "/compute/script/99/publish", ); + expect( + fetchStub.calls.map(({ args }) => (args[1] as RequestInit).method), + ).toEqual(["POST", "POST"]); + expect( + JSON.parse( + (fetchStub.calls[0]!.args[1] as RequestInit).body as string, + ), + ).toEqual({ Code: "console.log(1)" }); + expect((fetchStub.calls[1]!.args[1] as RequestInit).body).toBe("{}"); }, ); }); @@ -308,6 +329,7 @@ describeWithEnv( "/compute/script/42/publish", ); expect((fetchStub.calls[0]!.args[1] as RequestInit).method).toBe("POST"); + expect((fetchStub.calls[0]!.args[1] as RequestInit).body).toBe("{}"); }); test("returns error on API failure", async () => { @@ -410,9 +432,14 @@ describeWithEnv( DisableCookies: false, }); expect(result.ok).toBe(true); - expect(String(fetchStub.calls[0]!.args[0])).toContain("/pullzone/99"); + expect(String(fetchStub.calls[0]!.args[0])).toBe( + "https://api.bunny.net/pullzone/99", + ); const init = fetchStub.calls[0]!.args[1] as RequestInit; expect(init.method).toBe("POST"); + expect(new Headers(init.headers).get("content-type")).toBe( + "application/json", + ); const body = JSON.parse(init.body as string); expect(body.DisableCookies).toBe(false); }); diff --git a/test/shared/bunny-cdn/provider.test.ts b/test/shared/bunny-cdn/provider.test.ts new file mode 100644 index 000000000..c37853391 --- /dev/null +++ b/test/shared/bunny-cdn/provider.test.ts @@ -0,0 +1,89 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { stub } from "@std/testing/mock"; +import { bunnyCdnApi, bunnyHostingProvider } from "#shared/bunny-cdn.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; +import { restoreStubsAfterEach } from "#test-utils/mocks.ts"; + +describeWithEnv( + "Bunny hosting provider", + { env: { BUNNY_API_KEY: "test-key" } }, + () => { + const stubs: { restore(): void }[] = []; + restoreStubsAfterEach(stubs); + + test("names its required credential", () => { + expect(bunnyHostingProvider.configEnvVar).toBe("BUNNY_API_KEY"); + }); + + test("prepares pull-zone settings and every native secret", async () => { + const pullZoneSettings: Record[] = []; + const secrets: [number, string, string][] = []; + stubs.push( + stub(bunnyCdnApi, "createEdgeScript", () => + Promise.resolve({ + defaultHostname: "child.b-cdn.net", + ok: true, + pullZoneId: 9, + scriptId: 42, + }), + ), + stub(bunnyCdnApi, "updatePullZone", (_id, settings) => { + pullZoneSettings.push(settings); + return Promise.resolve({ ok: true }); + }), + stub(bunnyCdnApi, "setEdgeScriptSecret", (id, name, value) => { + secrets.push([id, name, value]); + return Promise.resolve({ ok: true }); + }), + ); + + expect( + await bunnyHostingProvider.prepareSite("Child", "code", [ + ["DB_URL", "libsql://child"], + ]), + ).toEqual({ + ok: true, + value: { + defaultHostname: "child.b-cdn.net", + hostingId: "42", + }, + }); + expect(pullZoneSettings).toEqual([{ DisableCookies: false }]); + expect(secrets).toEqual([ + [42, "DB_URL", "libsql://child"], + [42, "BUNNY_SCRIPT_ID", "42"], + ]); + }); + + test("rejects a non-numeric hosting id", async () => { + expect(await bunnyHostingProvider.setSecrets("missing", [])).toEqual({ + error: "No hostingId", + ok: false, + }); + }); + + test("stops setting secrets at the first provider failure", async () => { + const names: string[] = []; + stubs.push( + stub(bunnyCdnApi, "setEdgeScriptSecret", (_id, name) => { + names.push(name); + return Promise.resolve( + name === "SECOND" + ? { error: "set failed", ok: false } + : { ok: true }, + ); + }), + ); + + expect( + await bunnyHostingProvider.setSecrets("42", [ + ["FIRST", "1"], + ["SECOND", "2"], + ["THIRD", "3"], + ]), + ).toEqual({ error: "set failed", ok: false }); + expect(names).toEqual(["FIRST", "SECOND"]); + }); + }, +); diff --git a/test/shared/db/activity-log-backfill.test.ts b/test/shared/db/activity-log-backfill.test.ts index c5a9bcf22..959203179 100644 --- a/test/shared/db/activity-log-backfill.test.ts +++ b/test/shared/db/activity-log-backfill.test.ts @@ -1,49 +1,60 @@ import { expect } from "@std/expect"; import { it as test } from "@std/testing/bdd"; -import { spy } from "@std/testing/mock"; -import { FakeTime } from "@std/testing/time"; -import { ENCRYPTION_PREFIX, encrypt } from "#shared/crypto/encryption.ts"; +import { stub } from "@std/testing/mock"; +import { ENCRYPTION_PREFIX } from "#shared/crypto/encryption.ts"; import { HYBRID_PREFIX } from "#shared/crypto/keys.ts"; import { backfillActivityLogBatch, - maybeBackfillActivityLog, + hasLegacyActivityLog, + runActivityLogBackfill, } from "#shared/db/activity-log-backfill.ts"; import { getAllActivityLog, logActivity } from "#shared/db/activityLog.ts"; -import { execute, queryOne } from "#shared/db/client.ts"; +import { execute } from "#shared/db/client.ts"; import { settings } from "#shared/db/settings.ts"; -import { ACTIVITY_LOG_BACKFILL_INTERVAL_MS } from "#shared/limits.ts"; import { setSuppressDebugLogs } from "#shared/log-settings.ts"; +import { MAINTENANCE_TASKS } from "#shared/maintenance/registry.ts"; +import { maintenance } from "#shared/maintenance/runner.ts"; import { nowIso } from "#shared/now.ts"; +import { + insertLegacyActivity, + rawActivityMessage, +} from "#test-utils/activity-log.ts"; import { describeWithEnv } from "#test-utils/db.ts"; import { withTestSession } from "#test-utils/session.ts"; -/** Insert a row encrypted with DB_ENCRYPTION_KEY (the pre-migration format). */ -const insertLegacyRow = async (message: string): Promise => { - const result = await execute( - "INSERT INTO activity_log (message, created, listing_id, attendee_id) VALUES (?, ?, NULL, NULL)", - [await encrypt(message), nowIso()], - ); - return Number(result.lastInsertRowid); +const captureBackfillLogs = async ( + operation: () => Promise, +): Promise => { + setSuppressDebugLogs(false); + const debugStub = stub(console, "debug"); + try { + await operation(); + return debugStub.calls + .map((call) => String(call.args[0])) + .filter((line) => line.includes("[Backfill]")); + } finally { + debugStub.restore(); + setSuppressDebugLogs(null); + } }; -/** Raw (still-encrypted) stored message for a row. */ -const rawMessage = async (id: number): Promise => - (await queryOne<{ message: string }>( - "SELECT message FROM activity_log WHERE id = ?", - [id], - ))!.message; - describeWithEnv("db > activity log backfill", { db: true }, () => { test("re-encrypts legacy rows to the owner key, preserving the plaintext", async () => { - const id1 = await insertLegacyRow("legacy one"); - const id2 = await insertLegacyRow("legacy two"); - expect((await rawMessage(id1)).startsWith(ENCRYPTION_PREFIX)).toBe(true); + const id1 = await insertLegacyActivity("legacy one"); + const id2 = await insertLegacyActivity("legacy two"); + expect((await rawActivityMessage(id1)).startsWith(ENCRYPTION_PREFIX)).toBe( + true, + ); const converted = await backfillActivityLogBatch(settings.publicKey); expect(converted).toBe(2); - expect((await rawMessage(id1)).startsWith(HYBRID_PREFIX)).toBe(true); - expect((await rawMessage(id2)).startsWith(HYBRID_PREFIX)).toBe(true); + expect((await rawActivityMessage(id1)).startsWith(HYBRID_PREFIX)).toBe( + true, + ); + expect((await rawActivityMessage(id2)).startsWith(HYBRID_PREFIX)).toBe( + true, + ); // Re-encrypted rows still read back as the original plaintext for an admin. const messages = (await withTestSession(() => getAllActivityLog())).map( (e) => e.message, @@ -53,20 +64,22 @@ describeWithEnv("db > activity log backfill", { db: true }, () => { }); test("leaves owner-key rows untouched", async () => { - const legacyId = await insertLegacyRow("legacy"); + const legacyId = await insertLegacyActivity("legacy"); const owner = await logActivity("already owner-key"); - const ownerBefore = await rawMessage(owner.id); + const ownerBefore = await rawActivityMessage(owner.id); const converted = await backfillActivityLogBatch(settings.publicKey); expect(converted).toBe(1); // only the legacy row matched - expect(await rawMessage(owner.id)).toBe(ownerBefore); // byte-for-byte - expect((await rawMessage(legacyId)).startsWith(HYBRID_PREFIX)).toBe(true); + expect(await rawActivityMessage(owner.id)).toBe(ownerBefore); // byte-for-byte + expect((await rawActivityMessage(legacyId)).startsWith(HYBRID_PREFIX)).toBe( + true, + ); }); test("writes each re-encrypted message back to its own row (by id)", async () => { - const id1 = await insertLegacyRow("first plaintext"); - const id2 = await insertLegacyRow("second plaintext"); + const id1 = await insertLegacyActivity("first plaintext"); + const id2 = await insertLegacyActivity("second plaintext"); await backfillActivityLogBatch(settings.publicKey); @@ -86,113 +99,50 @@ describeWithEnv("db > activity log backfill", { db: true }, () => { expect(await backfillActivityLogBatch(settings.publicKey)).toBe(0); }); - test("scheduler converts a due batch and records the run timestamp", async () => { - const id = await insertLegacyRow("convert me"); - // A last-run a full day ago is unambiguously past the interval, so the run - // is due — pinning the elapsed-time subtraction in the interval check - // (now - last >= interval), not merely the last=0 boundary. - const dayAgo = Date.now() - 24 * 60 * 60 * 1000; - await settings.update.lastActivityLogBackfill(String(dayAgo)); - - await maybeBackfillActivityLog(); - - expect((await rawMessage(id)).startsWith(HYBRID_PREFIX)).toBe(true); - // Work may remain, so it is not marked done after a non-empty batch. - expect(settings.activityLogBackfillDone).not.toBe("true"); - // The run stamped a fresh, newer timestamp over the day-old one. - expect(Number(settings.lastActivityLogBackfill)).toBeGreaterThan(dayAgo); - }); - - test("scheduler treats an unset last-run as never run (epoch 0), so the first interval is exactly due", async () => { - // Freeze the clock at exactly one interval past the epoch. With no stored - // last-run the fallback must be 0 — "never ran" — making the batch due on - // the >= boundary; any later fallback would still be inside the interval - // and silently delay the very first run. - const id = await insertLegacyRow("due at the exact boundary"); - using _time = new FakeTime(ACTIVITY_LOG_BACKFILL_INTERVAL_MS); - - await maybeBackfillActivityLog(); + test("logs the exact number of converted rows", async () => { + const logs = await captureBackfillLogs(async () => { + await insertLegacyActivity("legacy"); + await runActivityLogBackfill(settings.publicKey); + }); - expect((await rawMessage(id)).startsWith(HYBRID_PREFIX)).toBe(true); + expect(logs).toEqual(["[Backfill] activity_log: re-encrypted 1 rows"]); }); - /** Run the scheduler with debug logs on, capturing every console.debug line. */ - const debugLinesFromRun = async (): Promise => { - setSuppressDebugLogs(false); - using debug = spy(console, "debug"); - try { - await maybeBackfillActivityLog(); - } finally { - setSuppressDebugLogs(null); - } - return debug.calls.map((call) => String(call.args[0])); - }; - - test("scheduler logs the converted row count under the Backfill category", async () => { - await insertLegacyRow("logged"); - - expect(await debugLinesFromRun()).toContain( - "[Backfill] activity_log: re-encrypted 1 rows", - ); - }); - - test("scheduler logs a failed batch under the Backfill category", async () => { - // A corrupt env-key payload makes the batch throw; the swallowed failure - // must still surface as a categorised debug line for the operator. - await execute( - "INSERT INTO activity_log (message, created, listing_id, attendee_id) VALUES (?, ?, NULL, NULL)", - [`${ENCRYPTION_PREFIX}AAAA:BBBB`, nowIso()], - ); - - const failureLine = (await debugLinesFromRun()).find((line) => - line.includes("activity_log failed"), + test("does not log when no rows need conversion", async () => { + const logs = await captureBackfillLogs(() => + runActivityLogBackfill(settings.publicKey), ); - expect(failureLine).toMatch(/^\[Backfill\] activity_log failed: /); - }); - - test("scheduler marks itself done once nothing remains", async () => { - await maybeBackfillActivityLog(); - - expect(settings.activityLogBackfillDone).toBe("true"); - }); - test("scheduler is a no-op once done", async () => { - await settings.update.activityLogBackfillDone("true"); - const id = await insertLegacyRow("should stay legacy"); - - await maybeBackfillActivityLog(); - - expect((await rawMessage(id)).startsWith(ENCRYPTION_PREFIX)).toBe(true); + expect(logs).toEqual([]); }); - test("scheduler is a no-op before a key pair is configured", async () => { - const id = await insertLegacyRow("no key yet"); - settings.setForTest({ public_key: "" }); - - await maybeBackfillActivityLog(); - - expect((await rawMessage(id)).startsWith(ENCRYPTION_PREFIX)).toBe(true); - expect(settings.activityLogBackfillDone).not.toBe("true"); + test("reports whether legacy work remains", async () => { + expect(await hasLegacyActivityLog()).toBe(false); + await insertLegacyActivity("legacy"); + expect(await hasLegacyActivityLog()).toBe(true); + await runActivityLogBackfill(settings.publicKey); + expect(await hasLegacyActivityLog()).toBe(false); }); - test("scheduler skips when the interval has not elapsed", async () => { - await settings.update.lastActivityLogBackfill(String(Date.now())); - const id = await insertLegacyRow("too soon"); + test("runs the legacy backfill through the maintenance registry", async () => { + const id = await insertLegacyActivity("registry legacy"); - await maybeBackfillActivityLog(); + await maintenance.run(MAINTENANCE_TASKS); - expect((await rawMessage(id)).startsWith(ENCRYPTION_PREFIX)).toBe(true); + expect((await rawActivityMessage(id)).startsWith(HYBRID_PREFIX)).toBe(true); }); - test("scheduler swallows a failing batch without marking done", async () => { - // A corrupt env-key payload makes decrypt throw partway through the batch. + test("surfaces a corrupt legacy message to the task runner", async () => { await execute( "INSERT INTO activity_log (message, created, listing_id, attendee_id) VALUES (?, ?, NULL, NULL)", [`${ENCRYPTION_PREFIX}AAAA:BBBB`, nowIso()], ); + await expect(runActivityLogBackfill(settings.publicKey)).rejects.toThrow(); + }); - await maybeBackfillActivityLog(); // must not throw - - expect(settings.activityLogBackfillDone).not.toBe("true"); + test("fails loudly when a raw activity row does not exist", async () => { + await expect(rawActivityMessage(999_999)).rejects.toThrow( + "Activity log entry not found: 999999", + ); }); }); diff --git a/test/shared/db/address-cache.test.ts b/test/shared/db/address-cache.test.ts index ed6e0471c..bea9cb0c4 100644 --- a/test/shared/db/address-cache.test.ts +++ b/test/shared/db/address-cache.test.ts @@ -11,7 +11,7 @@ import { storeCachedAddresses, } from "#shared/db/address-cache.ts"; import { queryOne } from "#shared/db/client.ts"; -import { pruneAddressCache } from "#shared/db/prune.ts"; +import { runDatabasePruning } from "#shared/db/prune.ts"; import { ADDRESS_CACHE_MS } from "#shared/limits.ts"; import { nowMs } from "#shared/now.ts"; import { describeWithEnv } from "#test-utils/db.ts"; @@ -133,7 +133,7 @@ describeWithEnv("address cache", { db: true }, () => { await storeCachedAddresses(fresh, ADDRESSES); await backdateRow(stale, ADDRESS_CACHE_MS + 1000); - expect(await pruneAddressCache()).toBe(1); + await runDatabasePruning(); const count = await queryOne<{ n: number }>( "SELECT COUNT(*) AS n FROM address_cache", diff --git a/test/shared/db/built-site-scheduler.test.ts b/test/shared/db/built-site-scheduler.test.ts new file mode 100644 index 000000000..ea56db435 --- /dev/null +++ b/test/shared/db/built-site-scheduler.test.ts @@ -0,0 +1,92 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { ensureBuiltSiteSchedulerKey } from "#shared/db/built-site-scheduler.ts"; +import { + builtSitesCrudTable, + insertBuiltSite, + updateBuiltSiteRenewalState, +} from "#shared/db/built-sites.ts"; +import { queryOne } from "#shared/db/client.ts"; +import { isScheduledTaskKey } from "#shared/scheduled-keys.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; +import { TEST_SCHEDULED_KEY } from "#test-utils/scheduled.ts"; + +describeWithEnv("built-site scheduler keys", { db: true }, () => { + test("stores the active key only inside encrypted site data", async () => { + const row = await insertBuiltSite( + "Child", + "child.example.test", + "", + "", + false, + "42", + undefined, + "bunny", + "bunny", + TEST_SCHEDULED_KEY, + ); + + const raw = await queryOne<{ site_data: string }>( + "SELECT site_data FROM built_sites WHERE id = ?", + [row.id], + ); + expect(raw?.site_data).not.toContain(TEST_SCHEDULED_KEY); + expect(JSON.stringify(raw)).not.toContain(TEST_SCHEDULED_KEY); + expect((await builtSitesCrudTable.findById(row.id))?.scheduledTaskKey).toBe( + TEST_SCHEDULED_KEY, + ); + }); + + test("fails loudly when the built site does not exist", async () => { + await expect(ensureBuiltSiteSchedulerKey(999_999)).rejects.toThrow( + "Built site not found", + ); + }); + + test("creates one active key across concurrent provisioning", async () => { + const site = await insertBuiltSite("Child", "child.example.test"); + + const [first, second] = await Promise.all([ + ensureBuiltSiteSchedulerKey(site.id), + ensureBuiltSiteSchedulerKey(site.id), + ]); + + expect(first).toBe(second); + expect(isScheduledTaskKey(first)).toBe(true); + expect( + (await builtSitesCrudTable.findById(site.id))?.scheduledTaskKey, + ).toBe(first); + }); + + test("keeps an edit and concurrent key provisioning", async () => { + const site = await insertBuiltSite("Child", "child.example.test"); + + await Promise.all([ + ensureBuiltSiteSchedulerKey(site.id), + builtSitesCrudTable.update(site.id, { name: "Edited child" }), + ]); + + const updated = await builtSitesCrudTable.findById(site.id); + expect(updated?.name).toBe("Edited child"); + expect(isScheduledTaskKey(updated?.scheduledTaskKey ?? "")).toBe(true); + }); + + test("keeps renewal state and concurrent key provisioning", async () => { + const site = await insertBuiltSite("Child", "child.example.test"); + + await Promise.all([ + ensureBuiltSiteSchedulerKey(site.id), + updateBuiltSiteRenewalState(site.id, { + readOnlyFrom: "2026-08-01T00:00:00.000Z", + renewalToken: "renewal-token", + renewalTokenIndex: "renewal-index", + }), + ]); + + const updated = await builtSitesCrudTable.findById(site.id); + expect(updated?.readOnlyFrom).toBe("2026-08-01T00:00:00.000Z"); + expect(updated?.renewalToken).toBe("renewal-token"); + expect(updated?.renewalTokenIndex).toBe("renewal-index"); + expect(isScheduledTaskKey(updated?.scheduledTaskKey ?? "")).toBe(true); + }); +}); diff --git a/test/shared/db/built-sites.test.ts b/test/shared/db/built-sites.test.ts deleted file mode 100644 index 18af03e71..000000000 --- a/test/shared/db/built-sites.test.ts +++ /dev/null @@ -1,796 +0,0 @@ -import { expect } from "@std/expect"; -import { describe, it as test } from "@std/testing/bdd"; -import { getAllCacheStats } from "#shared/cache-registry.ts"; -import { - assignBuiltSite, - builtSites, - builtSitesCrudTable, - claimNextBuiltSiteForPrune, - DEFAULT_UPDATE_TIER, - getAssignableBuiltSites, - getBuiltSiteByRenewalTokenIndex, - insertBuiltSite, - isUpdateTier, - parseSiteDataBlob, - providerOrBunny, - siteAcceptsDeployTier, - siteBaseUrl, - UPDATE_TIERS, - updateBuiltSiteRenewalState, -} from "#shared/db/built-sites.ts"; -import { execute } from "#shared/db/client.ts"; -import { describeWithEnv } from "#test-utils/db.ts"; - -const formBlob = async ( - input: Parameters[0], -) => { - const values = await builtSitesCrudTable.toDbValues(input); - return parseSiteDataBlob(values.site_data as string); -}; - -describe("providerOrBunny", () => { - test("keeps named providers and defaults other values to Bunny", () => { - expect(providerOrBunny("deno", "deno")).toBe("deno"); - expect(providerOrBunny("turso", "turso")).toBe("turso"); - expect(providerOrBunny("", "deno")).toBe("bunny"); - }); -}); - -describe("siteBaseUrl", () => { - test("prepends https:// to a bare hostname", () => { - expect(siteBaseUrl("site.b-cdn.net")).toBe("https://site.b-cdn.net"); - }); - - test("keeps an existing scheme", () => { - expect(siteBaseUrl("http://example.com")).toBe("http://example.com"); - }); - - test("strips a trailing slash so a path can be appended", () => { - expect(siteBaseUrl("https://example.com/")).toBe("https://example.com"); - }); - - test("collapses a path, query, and hash to the origin", () => { - expect(siteBaseUrl("https://example.com/admin?x=1#y")).toBe( - "https://example.com", - ); - }); - - test("normalizes an uppercase scheme to a lowercase origin", () => { - expect(siteBaseUrl("HTTPS://example.com")).toBe("https://example.com"); - }); -}); - -describe("update tiers", () => { - test("UPDATE_TIERS is ordered most- to least-eager", () => { - expect(UPDATE_TIERS).toEqual(["alpha", "beta", "release"]); - }); - - test("DEFAULT_UPDATE_TIER is the most conservative channel", () => { - expect(DEFAULT_UPDATE_TIER).toBe("release"); - }); - - test("isUpdateTier accepts known channels and rejects anything else", () => { - for (const tier of UPDATE_TIERS) expect(isUpdateTier(tier)).toBe(true); - for (const bad of ["", "ALPHA", "stable", "rel", "release "]) { - expect(isUpdateTier(bad)).toBe(false); - } - }); - - test("a release deploy reaches every channel", () => { - for (const siteTier of UPDATE_TIERS) { - expect(siteAcceptsDeployTier(siteTier, "release")).toBe(true); - } - }); - - test("a beta deploy reaches beta + alpha sites but not release-only", () => { - expect(siteAcceptsDeployTier("alpha", "beta")).toBe(true); - expect(siteAcceptsDeployTier("beta", "beta")).toBe(true); - expect(siteAcceptsDeployTier("release", "beta")).toBe(false); - }); - - test("an alpha deploy reaches only alpha sites", () => { - expect(siteAcceptsDeployTier("alpha", "alpha")).toBe(true); - expect(siteAcceptsDeployTier("beta", "alpha")).toBe(false); - expect(siteAcceptsDeployTier("release", "alpha")).toBe(false); - }); -}); - -describeWithEnv("claimNextBuiltSiteForPrune", { db: true }, () => { - test("returns null when there are no built sites", async () => { - expect(await claimNextBuiltSiteForPrune()).toBe(null); - }); - - test("walks sites least-recently-pruned first, then round-robins", async () => { - await insertBuiltSite("A", "a.example.com"); - await insertBuiltSite("B", "b.example.com"); - - // Both start never-pruned (''), so the lowest id goes first; after each is - // stamped, the other (still '') is next; then it cycles back to the oldest. - const first = await claimNextBuiltSiteForPrune(); - const second = await claimNextBuiltSiteForPrune(); - const third = await claimNextBuiltSiteForPrune(); - - expect(first?.siteUrl).toBe("a.example.com"); - expect(second?.siteUrl).toBe("b.example.com"); - expect(third?.siteUrl).toBe("a.example.com"); - }); -}); - -describeWithEnv("built-sites", { db: true }, () => { - test("toDbValues creates valid site-data JSON", async () => { - const parsed = await formBlob({ - name: "Test Site", - siteUrl: "test.b-cdn.net", - }); - expect(parsed.n).toBe("Test Site"); - expect(parsed.u).toBe("test.b-cdn.net"); - expect(parsed.v).toBe(1); - }); - - test("toDbValues includes db credentials when provided", async () => { - const parsed = await formBlob({ - dbToken: "secret-token", - dbUrl: "libsql://db.turso.io", - name: "Test Site", - siteUrl: "test.b-cdn.net", - }); - expect(parsed.d).toBe("libsql://db.turso.io"); - expect(parsed.t).toBe("secret-token"); - }); - - test("toDbValues omits db keys when empty", async () => { - const parsed = await formBlob({ - name: "Test Site", - siteUrl: "test.b-cdn.net", - }); - expect(parsed.d).toBeUndefined(); - expect(parsed.t).toBeUndefined(); - }); - - test("toDbValues includes bunny script id when provided", async () => { - const parsed = await formBlob({ - hostingId: "98765", - name: "Test Site", - siteUrl: "test.b-cdn.net", - }); - expect(parsed.s).toBe("98765"); - }); - - test("toDbValues omits bunny script id when empty", async () => { - const parsed = await formBlob({ - name: "Test Site", - siteUrl: "test.b-cdn.net", - }); - expect(parsed.s).toBeUndefined(); - }); - - test("parseSiteDataBlob decodes stored site-data JSON", async () => { - const parsed = await formBlob({ - name: "My Site", - siteUrl: "my.b-cdn.net", - }); - expect(parsed.n).toBe("My Site"); - expect(parsed.u).toBe("my.b-cdn.net"); - expect(parsed.v).toBe(1); - }); - - test("parseSiteDataBlob handles legacy blobs without db keys or a version", () => { - const legacyBlob = JSON.stringify({ - n: "Old Site", - u: "old.b-cdn.net", - }); - const parsed = parseSiteDataBlob(legacyBlob); - expect(parsed.d).toBeUndefined(); - expect(parsed.t).toBeUndefined(); - expect(parsed.s).toBeUndefined(); - expect(parsed.v).toBe(1); - - const providers = parseSiteDataBlob( - JSON.stringify({ - dp: "turso", - hp: "deno", - n: "Provider Site", - u: "provider.b-cdn.net", - }), - ); - expect(providers.dp).toBe("turso"); - expect(providers.hp).toBe("deno"); - }); - - test("parseSiteDataBlob rejects malformed JSON and invalid fields", () => { - expect(() => parseSiteDataBlob("{")).toThrow( - "Invalid stored JSON in built_sites.site_data", - ); - expect(() => - parseSiteDataBlob( - JSON.stringify({ hp: "unknown", n: "Site", u: "site.test", v: 1 }), - ), - ).toThrow("Invalid stored JSON in built_sites.site_data"); - }); - - test("toDbValues identifies invalid site data", () => { - expect(() => - builtSitesCrudTable.toDbValues({ - assignable: false, - dbToken: "", - dbUrl: "", - hostingId: "", - hostingProvider: "invalid" as never, - name: "Invalid Provider", - siteUrl: "invalid.b-cdn.net", - }), - ).toThrow("Invalid value for stored JSON in built_sites.site_data"); - }); - - test("insertBuiltSite creates a row with encrypted site_data", async () => { - const row = await insertBuiltSite("Alpha Site", "alpha.b-cdn.net"); - expect(row.id).toBe(1); - expect(row.created).toBeTruthy(); - }); - - test("insertBuiltSite stores db credentials when provided", async () => { - await insertBuiltSite( - "DB Site", - "db.b-cdn.net", - "libsql://db.turso.io", - "secret-token", - ); - const sites = await builtSites.getAll(); - const site = sites.find((s) => s.name === "DB Site")!; - expect(site.dbUrl).toBe("libsql://db.turso.io"); - expect(site.dbToken).toBe("secret-token"); - }); - - test("insertBuiltSite defaults db credentials to empty strings", async () => { - await insertBuiltSite("No DB Site", "nodb.b-cdn.net"); - const sites = await builtSites.getAll(); - const site = sites.find((s) => s.name === "No DB Site")!; - expect(site.dbUrl).toBe(""); - expect(site.dbToken).toBe(""); - }); - - test("insertBuiltSite stores bunny script id when provided", async () => { - await insertBuiltSite( - "Script Site", - "script.b-cdn.net", - "", - "", - false, - "12345", - ); - const sites = await builtSites.getAll(); - const site = sites.find((s) => s.name === "Script Site")!; - expect(site.hostingId).toBe("12345"); - }); - - test("insertBuiltSite defaults bunny script id to empty string", async () => { - await insertBuiltSite("No Script Site", "noscript.b-cdn.net"); - const sites = await builtSites.getAll(); - const site = sites.find((s) => s.name === "No Script Site")!; - expect(site.hostingId).toBe(""); - }); - - test("getAllBuiltSites returns decrypted sites sorted by name", async () => { - const charlie = await insertBuiltSite("Charlie", "charlie.b-cdn.net"); - const alpha = await insertBuiltSite("Alpha", "alpha.b-cdn.net"); - const bravo = await insertBuiltSite("Bravo", "bravo.b-cdn.net"); - await execute( - `UPDATE built_sites SET created = CASE id - WHEN ? THEN '2026-01-03T00:00:00.000Z' - WHEN ? THEN '2026-01-02T00:00:00.000Z' - WHEN ? THEN '2026-01-01T00:00:00.000Z' - ELSE created END`, - [charlie.id, alpha.id, bravo.id], - ); - - const sites = await builtSites.getAll(); - expect(sites).toHaveLength(3); - expect(sites[0]!.name).toBe("Alpha"); - expect(sites[0]!.siteUrl).toBe("alpha.b-cdn.net"); - expect(sites[1]!.name).toBe("Bravo"); - expect(sites[2]!.name).toBe("Charlie"); - }); - - test("getAllBuiltSites returns empty array when no sites exist", async () => { - const sites = await builtSites.getAll(); - expect(sites).toHaveLength(0); - }); - - describe("builtSitesCrudTable", () => { - const insertOriginalSite = () => - builtSitesCrudTable.insert({ - assignable: false, - dbToken: "", - dbUrl: "", - hostingId: "", - name: "Original", - siteUrl: "original.bunny.run", - }); - - test("findAll returns all built sites", async () => { - await insertBuiltSite("Site A", "a.bunny.run"); - await insertBuiltSite("Site B", "b.bunny.run"); - - const sites = await builtSitesCrudTable.findAll(); - expect(sites).toHaveLength(2); - }); - - test("fromDb returns the row unchanged", async () => { - const site = { - assignable: false, - assignedAttendeeId: null, - assignedListingId: null, - created: "2026-01-01", - dbProvider: "bunny" as const, - dbToken: "", - dbUrl: "", - hostingId: "", - hostingProvider: "bunny" as const, - id: 1, - name: "Test", - readOnlyFrom: "", - renewalToken: null, - renewalTokenIndex: null, - siteUrl: "test.bunny.run", - updates: "release" as const, - }; - const result = await builtSitesCrudTable.fromDb(site); - expect(result).toEqual(site); - }); - - test("readColumn returns the stored value unchanged", async () => { - const value = await builtSitesCrudTable.readColumn("name", "Test"); - expect(value).toBe("Test"); - }); - - test("raw nullable columns preserve falsy stored values", async () => { - expect(await builtSites.table.readColumn("assigned_attendee_id", 0)).toBe( - 0, - ); - expect(await builtSites.table.readColumn("renewal_token_index", "")).toBe( - "", - ); - }); - - test("exposes CRUD table metadata", () => { - expect(builtSitesCrudTable.name).toBe("built_sites"); - expect(builtSitesCrudTable.inputKeyMap).toEqual({ - assignable: "assignable", - db_provider: "dbProvider", - db_token: "dbToken", - db_url: "dbUrl", - hosting_id: "hostingId", - hosting_provider: "hostingProvider", - name: "name", - site_url: "siteUrl", - updates: "updates", - }); - }); - - test("rowToInput exposes form-input fields for reuse", () => { - const site = { - assignable: true, - assignedAttendeeId: null, - assignedListingId: null, - created: "2026-01-01", - dbProvider: "bunny" as const, - dbToken: "token", - dbUrl: "libsql://db", - hostingId: "script-123", - hostingProvider: "bunny" as const, - id: 42, - name: "Mirror", - readOnlyFrom: "", - renewalToken: null, - renewalTokenIndex: null, - siteUrl: "example.bunny.run", - updates: "beta" as const, - }; - expect(builtSitesCrudTable.rowToInput(site)).toEqual({ - assignable: true, - dbProvider: "bunny", - dbToken: "token", - dbUrl: "libsql://db", - hostingId: "script-123", - hostingProvider: "bunny", - name: "Mirror", - siteUrl: "example.bunny.run", - updates: "beta", - }); - }); - - test("toDbValues builds encrypted blob from input", async () => { - const values = await builtSitesCrudTable.toDbValues({ - assignable: false, - dbToken: "tok123", - dbUrl: "libsql://test.turso.io", - hostingId: "777", - name: "Test", - siteUrl: "test.bunny.run", - }); - expect(values.site_data).toBeTruthy(); - const parsed = parseSiteDataBlob(values.site_data as string); - expect(parsed.n).toBe("Test"); - expect(parsed.u).toBe("test.bunny.run"); - expect(parsed.d).toBe("libsql://test.turso.io"); - expect(parsed.t).toBe("tok123"); - expect(parsed.s).toBe("777"); - }); - - test("update preserves existing name when only siteUrl provided", async () => { - const site = await insertOriginalSite(); - const updated = await builtSitesCrudTable.update(site.id, { - siteUrl: "new.bunny.run", - }); - expect(updated!.name).toBe("Original"); - expect(updated!.siteUrl).toBe("new.bunny.run"); - }); - - test("update preserves existing siteUrl when only name provided", async () => { - const site = await insertOriginalSite(); - const updated = await builtSitesCrudTable.update(site.id, { - name: "Updated", - }); - expect(updated!.name).toBe("Updated"); - expect(updated!.siteUrl).toBe("original.bunny.run"); - }); - - test("update preserves existing db credentials when not provided", async () => { - const site = await builtSitesCrudTable.insert({ - assignable: false, - dbToken: "tok123", - dbUrl: "libsql://db.turso.io", - hostingId: "", - name: "Original", - siteUrl: "original.bunny.run", - }); - const updated = await builtSitesCrudTable.update(site.id, { - name: "Updated", - }); - expect(updated!.dbUrl).toBe("libsql://db.turso.io"); - expect(updated!.dbToken).toBe("tok123"); - }); - - test("update preserves existing bunny script id when not provided", async () => { - const site = await builtSitesCrudTable.insert({ - assignable: false, - dbToken: "", - dbUrl: "", - hostingId: "98765", - name: "Original", - siteUrl: "original.bunny.run", - }); - const updated = await builtSitesCrudTable.update(site.id, { - name: "Updated", - }); - expect(updated!.hostingId).toBe("98765"); - }); - - test("update changes bunny script id when provided", async () => { - const site = await builtSitesCrudTable.insert({ - assignable: false, - dbToken: "", - dbUrl: "", - hostingId: "111", - name: "Original", - siteUrl: "original.bunny.run", - }); - const updated = await builtSitesCrudTable.update(site.id, { - hostingId: "222", - }); - expect(updated!.hostingId).toBe("222"); - }); - - test("update returns null for non-existent id", async () => { - const result = await builtSitesCrudTable.update(999, { - name: "Test", - }); - expect(result).toBeNull(); - }); - - test("toDbValues handles partial input with missing fields", async () => { - const values = await builtSitesCrudTable.toDbValues({}); - const parsed = parseSiteDataBlob(values.site_data as string); - expect(values.assignable).toBe(0); - expect(parsed.n).toBe(""); - expect(parsed.u).toBe(""); - }); - - test("legacy rows default to Bunny providers", async () => { - const row = await builtSites.table.insert({ - siteData: JSON.stringify({ n: "Legacy", u: "legacy.b-cdn.net" }), - }); - const site = await builtSitesCrudTable.findById(row.id); - expect(site!.hostingProvider).toBe("bunny"); - expect(site!.dbProvider).toBe("bunny"); - }); - - test("cache stats identify the built sites cache", () => { - expect( - getAllCacheStats().filter(({ name }) => name === "built_sites"), - ).toHaveLength(1); - }); - - test("toDbValues sets assignable to 1 when true", async () => { - const values = await builtSitesCrudTable.toDbValues({ - assignable: true, - dbToken: "", - dbUrl: "", - hostingId: "", - name: "Test", - siteUrl: "test.bunny.run", - }); - expect(values.assignable).toBe(1); - }); - }); - - describe("assignable sites", () => { - test("insertBuiltSite with assignable flag", async () => { - await insertBuiltSite("Assignable Site", "a.b-cdn.net", "", "", true); - const sites = await builtSites.getAll(); - const site = sites.find((s) => s.name === "Assignable Site")!; - expect(site.assignable).toBe(true); - }); - - test("insertBuiltSite defaults to not assignable", async () => { - await insertBuiltSite("Default Site", "d.b-cdn.net"); - const sites = await builtSites.getAll(); - const site = sites.find((s) => s.name === "Default Site")!; - expect(site.assignable).toBe(false); - }); - - test("getAssignableBuiltSites filters to assignable only", async () => { - await insertBuiltSite("Site A", "a.b-cdn.net", "", "", true); - await insertBuiltSite("Site B", "b.b-cdn.net", "", "", false); - await insertBuiltSite("Site C", "c.b-cdn.net", "", "", true); - const sites = await getAssignableBuiltSites(); - expect(sites).toHaveLength(2); - expect(sites.every((s) => s.assignable)).toBe(true); - }); - - test("assignBuiltSite marks site as not assignable and stores IDs in columns", async () => { - await insertBuiltSite("To Assign", "assign.b-cdn.net", "", "", true); - const sites = await builtSites.getAll(); - const site = sites.find((s) => s.name === "To Assign")!; - - const updated = await assignBuiltSite(site.id, 42, 7); - expect(updated).not.toBeNull(); - expect(updated!.assignable).toBe(false); - expect(updated!.assignedAttendeeId).toBe(42); - expect(updated!.assignedListingId).toBe(7); - }); - - test("assignBuiltSite returns null for non-existent site", async () => { - const result = await assignBuiltSite(999, 1, 1); - expect(result).toBeNull(); - }); - - test("unassigned sites have null attendee and listing IDs", async () => { - await insertBuiltSite("Unassigned", "u.b-cdn.net", "", "", true); - const sites = await builtSites.getAll(); - const site = sites.find((s) => s.name === "Unassigned")!; - expect(site.assignedAttendeeId).toBeNull(); - expect(site.assignedListingId).toBeNull(); - }); - }); - - describe("renewal columns", () => { - test("new sites have empty readOnlyFrom and null renewal fields", async () => { - await insertBuiltSite("Renewal Site", "renewal.b-cdn.net"); - const sites = await builtSites.getAll(); - const site = sites.find((s) => s.name === "Renewal Site")!; - expect(site.readOnlyFrom).toBe(""); - expect(site.renewalTokenIndex).toBeNull(); - }); - - test("getBuiltSiteByRenewalTokenIndex returns matching site", async () => { - await insertBuiltSite("Token Site", "token.b-cdn.net"); - const sites = await builtSites.getAll(); - const site = sites.find((s) => s.name === "Token Site")!; - - await updateBuiltSiteRenewalState(site.id, { - readOnlyFrom: "2026-07-01T00:00:00Z", - renewalToken: "raw-token-123", - renewalTokenIndex: "test-index-abc", - }); - - const found = await getBuiltSiteByRenewalTokenIndex("test-index-abc"); - expect(found).not.toBeNull(); - expect(found!.name).toBe("Token Site"); - expect(found!.readOnlyFrom).toBe("2026-07-01T00:00:00Z"); - }); - - test("getBuiltSiteByRenewalTokenIndex returns null when no match", async () => { - const found = await getBuiltSiteByRenewalTokenIndex("nonexistent"); - expect(found).toBeNull(); - }); - - test("multiple sites with null renewalTokenIndex are allowed", async () => { - await insertBuiltSite("Site 1", "s1.b-cdn.net"); - await insertBuiltSite("Site 2", "s2.b-cdn.net"); - const sites = await builtSites.getAll(); - const nullIndexSiteNames = sites - .filter((s) => s.renewalTokenIndex === null) - .map((s) => s.name) - .sort(); - expect(nullIndexSiteNames).toEqual(["Site 1", "Site 2"]); - }); - - test("legacy blob still decodes correctly (no rt field)", () => { - const legacyBlob = JSON.stringify({ - n: "Old Site", - u: "old.b-cdn.net", - v: 1, - }); - const parsed = parseSiteDataBlob(legacyBlob); - expect(parsed.rt).toBeUndefined(); - expect(parsed.n).toBe("Old Site"); - }); - - test("CRUD update preserves existing renewal token", async () => { - const site = await builtSitesCrudTable.insert({ - assignable: false, - dbToken: "", - dbUrl: "", - hostingId: "100", - name: "Token Preserve", - siteUrl: "preserve.b-cdn.net", - }); - - await updateBuiltSiteRenewalState(site.id, { - readOnlyFrom: "2026-08-01T00:00:00Z", - renewalToken: "secret-token", - renewalTokenIndex: "idx-123", - }); - - const updated = await builtSitesCrudTable.update(site.id, { - name: "Token Preserve Updated", - }); - - expect(updated!.name).toBe("Token Preserve Updated"); - expect(updated!.renewalTokenIndex).toBe("idx-123"); - expect(updated!.readOnlyFrom).toBe("2026-08-01T00:00:00Z"); - expect(updated!.renewalToken).toBe("secret-token"); - }); - - test("updateBuiltSiteRenewalState updates individual fields", async () => { - await insertBuiltSite("Renewal Update", "ru.b-cdn.net"); - const sites = await builtSites.getAll(); - const site = sites.find((s) => s.name === "Renewal Update")!; - - await updateBuiltSiteRenewalState(site.id, { - readOnlyFrom: "2027-01-01T00:00:00Z", - }); - const afterFirst = await builtSites.getAll(); - const updatedAfterFirst = afterFirst.find((s) => s.id === site.id)!; - expect(updatedAfterFirst.readOnlyFrom).toBe("2027-01-01T00:00:00Z"); - expect(updatedAfterFirst.renewalTokenIndex).toBeNull(); - - await updateBuiltSiteRenewalState(site.id, { - renewalToken: "tok-456", - renewalTokenIndex: "idx-456", - }); - const afterSecond = await getBuiltSiteByRenewalTokenIndex("idx-456"); - expect(afterSecond).not.toBeNull(); - expect(afterSecond!.renewalTokenIndex).toBe("idx-456"); - expect(afterSecond!.renewalToken).toBe("tok-456"); - expect(afterSecond!.readOnlyFrom).toBe("2027-01-01T00:00:00Z"); - }); - }); - - describe("update channel", () => { - const crudInput = ( - overrides: Partial[0]> = {}, - ): Parameters[0] => ({ - assignable: false, - dbToken: "", - dbUrl: "", - hostingId: "", - name: "Channel Site", - siteUrl: "chan.b-cdn.net", - ...overrides, - }); - - test("insertBuiltSite defaults the channel to release", async () => { - await insertBuiltSite("Defaulted", "defaulted.b-cdn.net"); - const site = (await builtSites.getAll()).find( - (s) => s.name === "Defaulted", - )!; - expect(site.updates).toBe("release"); - }); - - test("insertBuiltSite stores an explicit channel that round-trips", async () => { - await insertBuiltSite( - "Alpha Chan", - "ac.b-cdn.net", - "", - "", - false, - "", - "alpha", - ); - const site = (await builtSites.getAll()).find( - (s) => s.name === "Alpha Chan", - )!; - expect(site.updates).toBe("alpha"); - }); - - test("CRUD insert defaults the channel to release when omitted", async () => { - const site = await builtSitesCrudTable.insert( - crudInput({ name: "Crud Default" }), - ); - expect(site.updates).toBe("release"); - }); - - test("CRUD insert persists an explicit channel through the DB", async () => { - const site = await builtSitesCrudTable.insert( - crudInput({ name: "Crud Beta", updates: "beta" }), - ); - expect(site.updates).toBe("beta"); - const reloaded = (await builtSites.getAll()).find( - (s) => s.id === site.id, - )!; - expect(reloaded.updates).toBe("beta"); - }); - - test("CRUD update changes the channel", async () => { - const site = await builtSitesCrudTable.insert( - crudInput({ name: "Chan Change" }), - ); - const updated = await builtSitesCrudTable.update(site.id, { - updates: "alpha", - }); - expect(updated!.updates).toBe("alpha"); - }); - - test("CRUD update preserves the channel when other fields change", async () => { - const site = await builtSitesCrudTable.insert( - crudInput({ name: "Keep Chan", updates: "beta" }), - ); - const updated = await builtSitesCrudTable.update(site.id, { - name: "Keep Chan Renamed", - }); - expect(updated!.name).toBe("Keep Chan Renamed"); - expect(updated!.updates).toBe("beta"); - }); - - test("assigning a site preserves its update channel", async () => { - await insertBuiltSite( - "Assign Chan", - "ach.b-cdn.net", - "", - "", - true, - "", - "beta", - ); - const site = (await builtSites.getAll()).find( - (s) => s.name === "Assign Chan", - )!; - const updated = await assignBuiltSite(site.id, 1, 2); - expect(updated!.updates).toBe("beta"); - }); - - test("updating renewal state preserves the update channel", async () => { - await insertBuiltSite( - "Renew Chan", - "rch.b-cdn.net", - "", - "", - false, - "", - "alpha", - ); - const site = (await builtSites.getAll()).find( - (s) => s.name === "Renew Chan", - )!; - await updateBuiltSiteRenewalState(site.id, { - readOnlyFrom: "2027-01-01T00:00:00Z", - }); - const reloaded = (await builtSites.getAll()).find( - (s) => s.id === site.id, - )!; - expect(reloaded.updates).toBe("alpha"); - }); - }); -}); diff --git a/test/shared/db/built-sites/assignment-renewal.test.ts b/test/shared/db/built-sites/assignment-renewal.test.ts new file mode 100644 index 000000000..f601f959c --- /dev/null +++ b/test/shared/db/built-sites/assignment-renewal.test.ts @@ -0,0 +1,195 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { ensureBuiltSiteSchedulerKey } from "#shared/db/built-site-scheduler.ts"; +import { parseSiteDataBlob } from "#shared/db/built-sites/blob.ts"; +import { + assignBuiltSite, + builtSites, + builtSitesCrudTable, + getAssignableBuiltSites, + getBuiltSiteByRenewalTokenIndex, + insertBuiltSite, + updateBuiltSiteRenewalState, +} from "#shared/db/built-sites.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; + +describeWithEnv("assignable built sites", { db: true }, () => { + test("insertBuiltSite stores the assignable flag", async () => { + await insertBuiltSite("Assignable Site", "a.b-cdn.net", "", "", true); + expect((await builtSites.getAll())[0]?.assignable).toBe(true); + }); + + test("insertBuiltSite defaults to not assignable", async () => { + await insertBuiltSite("Default Site", "d.b-cdn.net"); + expect((await builtSites.getAll())[0]?.assignable).toBe(false); + }); + + test("getAssignableBuiltSites filters to assignable sites", async () => { + await insertBuiltSite("Site A", "a.b-cdn.net", "", "", true); + await insertBuiltSite("Site B", "b.b-cdn.net", "", "", false); + await insertBuiltSite("Site C", "c.b-cdn.net", "", "", true); + expect((await getAssignableBuiltSites()).map(({ name }) => name)).toEqual([ + "Site A", + "Site C", + ]); + }); + + test("assignBuiltSite stores the assignment", async () => { + const row = await insertBuiltSite( + "To Assign", + "assign.b-cdn.net", + "", + "", + true, + ); + expect(await assignBuiltSite(row.id, 42, 7)).toMatchObject({ + assignable: false, + assignedAttendeeId: 42, + assignedListingId: 7, + }); + }); + + test("assignBuiltSite returns null for a missing site", async () => { + expect(await assignBuiltSite(999, 1, 1)).toBeNull(); + }); + + test("keeps an assignment made during scheduler-key provisioning", async () => { + const site = await insertBuiltSite( + "Concurrent assignment", + "concurrent.example.test", + "", + "", + true, + ); + + await Promise.all([ + ensureBuiltSiteSchedulerKey(site.id), + assignBuiltSite(site.id, 42, 7), + ]); + + expect(await builtSitesCrudTable.findById(site.id)).toMatchObject({ + assignable: false, + assignedAttendeeId: 42, + assignedListingId: 7, + siteDataRevision: 2, + }); + }); + + test("unassigned sites have null assignment ids", async () => { + await insertBuiltSite("Unassigned", "u.b-cdn.net", "", "", true); + expect((await builtSites.getAll())[0]).toMatchObject({ + assignedAttendeeId: null, + assignedListingId: null, + }); + }); +}); + +describeWithEnv("built-site renewal storage", { db: true }, () => { + test("new sites have empty renewal state", async () => { + await insertBuiltSite("Renewal Site", "renewal.b-cdn.net"); + expect((await builtSites.getAll())[0]).toMatchObject({ + readOnlyFrom: "", + renewalToken: null, + renewalTokenIndex: null, + }); + }); + + test("getBuiltSiteByRenewalTokenIndex returns the matching site", async () => { + const row = await insertBuiltSite("Token Site", "token.b-cdn.net"); + await updateBuiltSiteRenewalState(row.id, { + readOnlyFrom: "2026-07-01T00:00:00Z", + renewalToken: "raw-token-123", + renewalTokenIndex: "test-index-abc", + }); + expect( + await getBuiltSiteByRenewalTokenIndex("test-index-abc"), + ).toMatchObject({ + name: "Token Site", + readOnlyFrom: "2026-07-01T00:00:00Z", + }); + }); + + test("getBuiltSiteByRenewalTokenIndex returns null when no site matches", async () => { + expect(await getBuiltSiteByRenewalTokenIndex("nonexistent")).toBeNull(); + }); + + test("multiple sites may have a null renewal index", async () => { + await insertBuiltSite("Site 1", "s1.b-cdn.net"); + await insertBuiltSite("Site 2", "s2.b-cdn.net"); + expect( + (await builtSites.getAll()) + .filter(({ renewalTokenIndex }) => renewalTokenIndex === null) + .map(({ name }) => name), + ).toEqual(["Site 1", "Site 2"]); + }); + + test("legacy blobs omit the renewal token", () => { + const parsed = parseSiteDataBlob( + JSON.stringify({ n: "Old Site", u: "old.b-cdn.net", v: 1 }), + ); + expect(parsed.rt).toBeUndefined(); + }); + + test("CRUD edits preserve renewal state", async () => { + const site = await builtSitesCrudTable.insert({ + assignable: false, + dbToken: "", + dbUrl: "", + hostingId: "100", + name: "Token Preserve", + siteUrl: "preserve.b-cdn.net", + }); + await updateBuiltSiteRenewalState(site.id, { + readOnlyFrom: "2026-08-01T00:00:00Z", + renewalToken: "secret-token", + renewalTokenIndex: "idx-123", + }); + expect( + await builtSitesCrudTable.update(site.id, { + name: "Token Preserve Updated", + }), + ).toMatchObject({ + name: "Token Preserve Updated", + readOnlyFrom: "2026-08-01T00:00:00Z", + renewalToken: "secret-token", + renewalTokenIndex: "idx-123", + }); + }); + + test("updateBuiltSiteRenewalState updates selected fields", async () => { + const row = await insertBuiltSite("Renewal Update", "ru.b-cdn.net"); + await updateBuiltSiteRenewalState(row.id, { + readOnlyFrom: "2027-01-01T00:00:00Z", + }); + await updateBuiltSiteRenewalState(row.id, { + renewalToken: "tok-456", + renewalTokenIndex: "idx-456", + }); + expect(await getBuiltSiteByRenewalTokenIndex("idx-456")).toMatchObject({ + readOnlyFrom: "2027-01-01T00:00:00Z", + renewalToken: "tok-456", + renewalTokenIndex: "idx-456", + }); + }); + + test("an empty renewal index remains an empty string", async () => { + const row = await insertBuiltSite("Empty Index", "empty-index.b-cdn.net"); + await updateBuiltSiteRenewalState(row.id, { renewalTokenIndex: "" }); + expect( + (await builtSitesCrudTable.findById(row.id))?.renewalTokenIndex, + ).toBe(""); + }); + + test("an empty renewal token clears the stored token", async () => { + const row = await insertBuiltSite("Empty Token", "empty-token.b-cdn.net"); + await updateBuiltSiteRenewalState(row.id, { + renewalToken: "stored-token", + }); + + await updateBuiltSiteRenewalState(row.id, { renewalToken: "" }); + + expect( + (await builtSitesCrudTable.findById(row.id))?.renewalToken, + ).toBeNull(); + }); +}); diff --git a/test/shared/db/built-sites/blob.test.ts b/test/shared/db/built-sites/blob.test.ts new file mode 100644 index 000000000..7e3ab1aa2 --- /dev/null +++ b/test/shared/db/built-sites/blob.test.ts @@ -0,0 +1,104 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { + blobToSiteFields, + buildSiteDataBlobFromInput, + parseSiteDataBlob, +} from "#shared/db/built-sites/blob.ts"; +import { TEST_SCHEDULED_KEY } from "#test-utils/scheduled.ts"; + +test("builds and parses every current site-data field", () => { + expect( + parseSiteDataBlob( + buildSiteDataBlobFromInput({ + dbProvider: "turso", + dbToken: "database-token", + dbUrl: "libsql://database.example", + hostingId: "app-42", + hostingProvider: "deno", + name: "Child site", + renewalToken: "renewal-token", + scheduledTaskKey: TEST_SCHEDULED_KEY, + siteUrl: "child.example", + }), + ), + ).toEqual({ + d: "libsql://database.example", + dp: "turso", + hp: "deno", + n: "Child site", + rt: "renewal-token", + s: "app-42", + sk: TEST_SCHEDULED_KEY, + t: "database-token", + u: "child.example", + v: 2, + }); +}); + +test("omits empty optional fields and supplies required defaults", () => { + expect(parseSiteDataBlob(buildSiteDataBlobFromInput({}))).toEqual({ + dp: "bunny", + hp: "bunny", + n: "", + u: "", + v: 2, + }); +}); + +test("parses legacy blobs without current optional fields", () => { + expect( + parseSiteDataBlob( + JSON.stringify({ n: "Legacy", u: "legacy.example", v: 1 }), + ), + ).toEqual({ n: "Legacy", u: "legacy.example", v: 1 }); +}); + +test("turns a legacy blob into every site field default", () => { + expect( + blobToSiteFields({ + n: "Legacy", + rt: "", + u: "legacy.example", + v: 1, + }), + ).toEqual({ + dbProvider: "bunny", + dbToken: "", + dbUrl: "", + hostingId: "", + hostingProvider: "bunny", + name: "Legacy", + renewalToken: "", + scheduledTaskKey: null, + siteUrl: "legacy.example", + }); +}); + +test("rejects invalid scheduled keys while building the blob", () => { + expect(() => + buildSiteDataBlobFromInput({ scheduledTaskKey: "invalid" }), + ).toThrow("Invalid value for stored JSON in built_sites.site_data"); +}); + +test("rejects malformed and unknown stored fields", () => { + expect(() => parseSiteDataBlob("{")).toThrow( + "Invalid stored JSON in built_sites.site_data", + ); + expect(() => + parseSiteDataBlob( + JSON.stringify({ hp: "unknown", n: "Site", u: "site.test", v: 1 }), + ), + ).toThrow("Invalid stored JSON in built_sites.site_data"); + expect(() => + parseSiteDataBlob( + JSON.stringify({ extra: true, n: "Site", u: "site.test", v: 1 }), + ), + ).toThrow("Invalid stored JSON in built_sites.site_data"); +}); + +test("rejects invalid site data before serialization", () => { + expect(() => + buildSiteDataBlobFromInput({ hostingProvider: "invalid" as never }), + ).toThrow("Invalid value for stored JSON in built_sites.site_data"); +}); diff --git a/test/shared/db/built-sites/core.test.ts b/test/shared/db/built-sites/core.test.ts new file mode 100644 index 000000000..ec5bf19d2 --- /dev/null +++ b/test/shared/db/built-sites/core.test.ts @@ -0,0 +1,220 @@ +import { expect } from "@std/expect"; +import { describe, it as test } from "@std/testing/bdd"; +import { getAllCacheStats } from "#shared/cache-registry.ts"; +import { parseSiteDataBlob } from "#shared/db/built-sites/blob.ts"; +import { + builtSites, + builtSitesCrudTable, + insertBuiltSite, + siteBaseUrl, +} from "#shared/db/built-sites.ts"; +import { execute } from "#shared/db/client.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; +import { TEST_SCHEDULED_KEY } from "#test-utils/scheduled.ts"; + +const formBlob = async ( + input: Parameters[0], +) => { + const values = await builtSitesCrudTable.toDbValues(input); + return parseSiteDataBlob(values.site_data as string); +}; + +describe("siteBaseUrl", () => { + test("prepends https:// to a bare hostname", () => { + expect(siteBaseUrl("site.b-cdn.net")).toBe("https://site.b-cdn.net"); + }); + + test("keeps an existing scheme", () => { + expect(siteBaseUrl("http://example.com")).toBe("http://example.com"); + }); + + test("strips a trailing slash so a path can be appended", () => { + expect(siteBaseUrl("https://example.com/")).toBe("https://example.com"); + }); + + test("collapses a path, query, and hash to the origin", () => { + expect(siteBaseUrl("https://example.com/admin?x=1#y")).toBe( + "https://example.com", + ); + }); + + test("normalizes an uppercase scheme to a lowercase origin", () => { + expect(siteBaseUrl("HTTPS://example.com")).toBe("https://example.com"); + }); +}); + +describeWithEnv("built-site storage", { db: true }, () => { + test("toDbValues creates valid site-data JSON", async () => { + const parsed = await formBlob({ + name: "Test Site", + siteUrl: "test.b-cdn.net", + }); + expect(parsed).toMatchObject({ + dp: "bunny", + hp: "bunny", + n: "Test Site", + u: "test.b-cdn.net", + v: 2, + }); + }); + + test("toDbValues includes selected providers", async () => { + const parsed = await formBlob({ + dbProvider: "turso", + hostingProvider: "deno", + name: "Test Site", + siteUrl: "test.b-cdn.net", + }); + expect(parsed.dp).toBe("turso"); + expect(parsed.hp).toBe("deno"); + }); + + test("toDbValues includes db credentials when provided", async () => { + const parsed = await formBlob({ + dbToken: "secret-token", + dbUrl: "libsql://db.turso.io", + name: "Test Site", + siteUrl: "test.b-cdn.net", + }); + expect(parsed.d).toBe("libsql://db.turso.io"); + expect(parsed.t).toBe("secret-token"); + }); + + test("toDbValues omits db keys when empty", async () => { + const parsed = await formBlob({ + name: "Test Site", + siteUrl: "test.b-cdn.net", + }); + expect(parsed.d).toBeUndefined(); + expect(parsed.t).toBeUndefined(); + }); + + test("toDbValues includes hosting id when provided", async () => { + const parsed = await formBlob({ + hostingId: "98765", + name: "Test Site", + siteUrl: "test.b-cdn.net", + }); + expect(parsed.s).toBe("98765"); + }); + + test("toDbValues omits hosting id when empty", async () => { + const parsed = await formBlob({ + name: "Test Site", + siteUrl: "test.b-cdn.net", + }); + expect(parsed.s).toBeUndefined(); + }); + + test("parseSiteDataBlob handles legacy blobs without optional keys", () => { + const parsed = parseSiteDataBlob( + JSON.stringify({ n: "Old Site", u: "old.b-cdn.net", v: 1 }), + ); + expect(parsed.d).toBeUndefined(); + expect(parsed.t).toBeUndefined(); + expect(parsed.s).toBeUndefined(); + }); + + test("insertBuiltSite creates a row with encrypted site data", async () => { + const row = await insertBuiltSite("Alpha Site", "alpha.b-cdn.net"); + expect(row.id).toBe(1); + expect(row.created).toBeTruthy(); + }); + + test("insertBuiltSite stores credentials and hosting id", async () => { + await insertBuiltSite( + "Configured Site", + "configured.b-cdn.net", + "libsql://db.turso.io", + "secret-token", + false, + "12345", + ); + const site = (await builtSites.getAll())[0]!; + expect(site.dbUrl).toBe("libsql://db.turso.io"); + expect(site.dbToken).toBe("secret-token"); + expect(site.hostingId).toBe("12345"); + }); + + test("insertBuiltSite defaults optional strings", async () => { + await insertBuiltSite("Default Site", "default.b-cdn.net"); + const site = (await builtSites.getAll())[0]!; + expect(site.dbUrl).toBe(""); + expect(site.dbToken).toBe(""); + expect(site.hostingId).toBe(""); + }); + + test("getAll returns decrypted sites sorted by name", async () => { + const alpha = await insertBuiltSite("Alpha", "alpha.b-cdn.net"); + const zulu = await insertBuiltSite("Zulu", "zulu.b-cdn.net"); + await execute("UPDATE built_sites SET created = ? WHERE id = ?", [ + "2025-01-01T00:00:00Z", + alpha.id, + ]); + await execute("UPDATE built_sites SET created = ? WHERE id = ?", [ + "2026-01-01T00:00:00Z", + zulu.id, + ]); + expect((await builtSites.getAll()).map(({ name }) => name)).toEqual([ + "Alpha", + "Zulu", + ]); + }); + + test("getAll returns an empty array when no sites exist", async () => { + expect(await builtSites.getAll()).toEqual([]); + }); + + test("cache stats identify the built-site cache", () => { + expect( + getAllCacheStats().filter(({ name }) => name === "built_sites"), + ).toHaveLength(1); + }); + + test("the scheduler key round-trips through encrypted site data", async () => { + const row = await insertBuiltSite( + "Scheduled Site", + "scheduled.example.test", + "", + "", + false, + "", + undefined, + "bunny", + "bunny", + TEST_SCHEDULED_KEY, + ); + const site = await builtSitesCrudTable.findById(row.id); + expect(site?.scheduledTaskKey).toBe(TEST_SCHEDULED_KEY); + }); + + test("rejects an invalid scheduler key before inserting the site", async () => { + await expect( + insertBuiltSite( + "Invalid scheduled key", + "invalid-key.example.test", + "", + "", + false, + "", + undefined, + "bunny", + "bunny", + "not-a-scheduled-key", + ), + ).rejects.toThrow(); + expect(await builtSites.getAll()).toEqual([]); + }); + + test("legacy empty renewal tokens remain empty strings", async () => { + const row = await builtSites.table.insert({ + siteData: JSON.stringify({ + n: "Legacy renewal", + rt: "", + u: "legacy.example.test", + v: 1, + }), + }); + expect((await builtSitesCrudTable.findById(row.id))?.renewalToken).toBe(""); + }); +}); diff --git a/test/shared/db/built-sites/crud.test.ts b/test/shared/db/built-sites/crud.test.ts new file mode 100644 index 000000000..2306ff04d --- /dev/null +++ b/test/shared/db/built-sites/crud.test.ts @@ -0,0 +1,246 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { parseSiteDataBlob } from "#shared/db/built-sites/blob.ts"; +import type { BuiltSite } from "#shared/db/built-sites/types.ts"; +import { + assignBuiltSite, + builtSitesCrudTable, + insertBuiltSite, + updateBuiltSiteRenewalState, +} from "#shared/db/built-sites.ts"; +import { mustReadFromPrimary } from "#shared/db/primary-reads.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; +import { statementSql, wrapDbClient } from "#test-utils/record-queries.ts"; +import { builtSiteFormInput } from "./fixtures.ts"; + +const siteFixture = (overrides: Partial = {}): BuiltSite => ({ + assignable: false, + assignedAttendeeId: null, + assignedListingId: null, + created: "2026-01-01", + dbProvider: "bunny", + dbToken: "", + dbUrl: "", + hostingId: "", + hostingProvider: "bunny", + id: 1, + name: "Test", + readOnlyFrom: "", + renewalToken: null, + renewalTokenIndex: null, + scheduledTaskKey: null, + siteDataRevision: 0, + siteUrl: "test.bunny.run", + updates: "release", + ...overrides, +}); + +describeWithEnv("built-sites CRUD table", { db: true }, () => { + test("findAll returns all built sites", async () => { + await insertBuiltSite("Site A", "a.bunny.run"); + await insertBuiltSite("Site B", "b.bunny.run"); + expect(await builtSitesCrudTable.findAll()).toHaveLength(2); + }); + + test("fromDb returns the row unchanged", async () => { + const site = siteFixture(); + expect(await builtSitesCrudTable.fromDb(site)).toEqual(site); + }); + + test("readColumn returns the stored value unchanged", async () => { + expect(await builtSitesCrudTable.readColumn("name", "Test")).toBe("Test"); + }); + + test("uses the physical built-sites table name", () => { + expect(builtSitesCrudTable.name).toBe("built_sites"); + }); + + test("inputKeyMap exposes form-facing fields", () => { + expect(builtSitesCrudTable.inputKeyMap).toEqual({ + assignable: "assignable", + db_provider: "dbProvider", + db_token: "dbToken", + db_url: "dbUrl", + hosting_id: "hostingId", + hosting_provider: "hostingProvider", + name: "name", + site_url: "siteUrl", + updates: "updates", + }); + }); + + test("rowToInput exposes form-input fields for reuse", () => { + const site = siteFixture({ + assignable: true, + dbToken: "token", + dbUrl: "libsql://db", + hostingId: "script-123", + id: 42, + name: "Mirror", + siteUrl: "example.bunny.run", + updates: "beta", + }); + expect(builtSitesCrudTable.rowToInput(site)).toEqual({ + assignable: true, + dbProvider: "bunny", + dbToken: "token", + dbUrl: "libsql://db", + hostingId: "script-123", + hostingProvider: "bunny", + name: "Mirror", + siteUrl: "example.bunny.run", + updates: "beta", + }); + }); + + test("toDbValues builds site data from input", async () => { + const values = await builtSitesCrudTable.toDbValues({ + assignable: false, + dbToken: "tok123", + dbUrl: "libsql://test.turso.io", + hostingId: "777", + name: "Test", + siteUrl: "test.bunny.run", + }); + expect(parseSiteDataBlob(values.site_data as string)).toMatchObject({ + d: "libsql://test.turso.io", + n: "Test", + s: "777", + t: "tok123", + u: "test.bunny.run", + }); + }); + + test("toDbValues supplies every empty-form default", async () => { + const values = await builtSitesCrudTable.toDbValues({}); + expect(values.assignable).toBe(0); + expect(parseSiteDataBlob(values.site_data as string)).toMatchObject({ + n: "", + u: "", + }); + }); + + test("toDbValues stores true assignable as one", async () => { + const values = await builtSitesCrudTable.toDbValues( + builtSiteFormInput({ assignable: true }), + ); + expect(values.assignable).toBe(1); + }); + + test("update preserves the existing name", async () => { + const site = await builtSitesCrudTable.insert(builtSiteFormInput()); + const updated = await builtSitesCrudTable.update(site.id, { + siteUrl: "new.bunny.run", + }); + expect(updated?.name).toBe("Original"); + expect(updated?.siteUrl).toBe("new.bunny.run"); + }); + + test("update preserves the existing site URL", async () => { + const site = await builtSitesCrudTable.insert(builtSiteFormInput()); + const updated = await builtSitesCrudTable.update(site.id, { + name: "Updated", + }); + expect(updated?.name).toBe("Updated"); + expect(updated?.siteUrl).toBe("original.bunny.run"); + }); + + test("update preserves credentials not included in the edit", async () => { + const site = await builtSitesCrudTable.insert( + builtSiteFormInput({ + dbToken: "tok123", + dbUrl: "libsql://db.turso.io", + hostingId: "98765", + }), + ); + const updated = await builtSitesCrudTable.update(site.id, { + name: "Updated", + }); + expect(updated).toMatchObject({ + dbToken: "tok123", + dbUrl: "libsql://db.turso.io", + hostingId: "98765", + }); + }); + + test("update changes hosting id when provided", async () => { + const site = await builtSitesCrudTable.insert( + builtSiteFormInput({ hostingId: "111" }), + ); + expect( + await builtSitesCrudTable.update(site.id, { hostingId: "222" }), + ).toMatchObject({ hostingId: "222" }); + }); + + test("update returns null for a missing id", async () => { + expect(await builtSitesCrudTable.update(999, { name: "Test" })).toBeNull(); + }); + + test("reads from the primary before updating a newly written site", async () => { + const site = await builtSitesCrudTable.insert(builtSiteFormInput()); + const primaryReads: boolean[] = []; + const restore = wrapDbClient({ + batch: (statements, mode) => { + if ( + statements.some((statement) => + statementSql(statement).includes("FROM built_sites"), + ) + ) { + primaryReads.push(mode === "write"); + } + }, + execute: (statement) => { + if (statementSql(statement).includes("FROM built_sites")) { + primaryReads.push(mustReadFromPrimary()); + } + return null; + }, + }); + try { + await builtSitesCrudTable.update(site.id, { assignable: true }); + } finally { + restore(); + } + + expect(primaryReads).toEqual([true]); + }); + + test("update preserves stored state and advances its revision", async () => { + const row = await insertBuiltSite( + "Stateful", + "stateful.example.test", + "", + "", + true, + ); + await assignBuiltSite(row.id, 42, 7); + await updateBuiltSiteRenewalState(row.id, { + readOnlyFrom: "2027-01-01T00:00:00Z", + renewalToken: "renewal-token", + renewalTokenIndex: "renewal-index", + }); + const updated = await builtSitesCrudTable.update(row.id, { + name: "Edited stateful", + }); + expect(updated).toMatchObject({ + assignedAttendeeId: 42, + assignedListingId: 7, + readOnlyFrom: "2027-01-01T00:00:00Z", + renewalTokenIndex: "renewal-index", + siteDataRevision: 3, + }); + }); + + test("concurrent edits preserve both changes", async () => { + const site = await builtSitesCrudTable.insert(builtSiteFormInput()); + await Promise.all([ + builtSitesCrudTable.update(site.id, { name: "Renamed" }), + builtSitesCrudTable.update(site.id, { siteUrl: "moved.bunny.run" }), + ]); + expect(await builtSitesCrudTable.findById(site.id)).toMatchObject({ + name: "Renamed", + siteDataRevision: 2, + siteUrl: "moved.bunny.run", + }); + }); +}); diff --git a/test/shared/db/built-sites/fields.test.ts b/test/shared/db/built-sites/fields.test.ts new file mode 100644 index 000000000..e5d9cd164 --- /dev/null +++ b/test/shared/db/built-sites/fields.test.ts @@ -0,0 +1,205 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { + builtSiteBlobColumns, + builtSiteFormMappings, + builtSiteInputKeyMap, + builtSitePlainColumns, + builtSitePlainSchema, + emptyBuiltSiteFormInput, + mapPlainFields, + plainSiteInput, +} from "#shared/db/built-sites/fields.ts"; + +const plainFields = { + assignable: true, + assignedAttendeeId: 42, + assignedListingId: 7, + readOnlyFrom: "2027-01-01T00:00:00Z", + renewalTokenIndex: "renewal-index", + siteDataRevision: 3, + updates: "beta", +} as const; + +test("maps every plain site field to table input", () => { + expect(plainSiteInput(plainFields)).toEqual({ + assignable: 1, + assignedAttendeeId: 42, + assignedListingId: 7, + readOnlyFrom: "2027-01-01T00:00:00Z", + renewalTokenIndex: "renewal-index", + siteDataRevision: 3, + updates: "beta", + }); + expect(mapPlainFields(plainFields, "dbKey")).toEqual({ + assignable: 1, + assigned_attendee_id: 42, + assigned_listing_id: 7, + read_only_from: "2027-01-01T00:00:00Z", + renewal_token_index: "renewal-index", + site_data_revision: 3, + updates: "beta", + }); +}); + +test("maps false and null plain values without dropping them", () => { + expect( + plainSiteInput({ + assignable: false, + assignedAttendeeId: null, + assignedListingId: null, + renewalTokenIndex: null, + }), + ).toEqual({ + assignable: 0, + assignedAttendeeId: null, + assignedListingId: null, + renewalTokenIndex: null, + }); + expect( + plainSiteInput({ + assignedAttendeeId: 0, + readOnlyFrom: "", + renewalTokenIndex: "", + }), + ).toEqual({ + assignedAttendeeId: 0, + readOnlyFrom: "", + renewalTokenIndex: "", + }); +}); + +test("defines exact form names and defaults", () => { + expect(builtSiteInputKeyMap).toEqual({ + assignable: "assignable", + db_provider: "dbProvider", + db_token: "dbToken", + db_url: "dbUrl", + hosting_id: "hostingId", + hosting_provider: "hostingProvider", + name: "name", + site_url: "siteUrl", + updates: "updates", + }); + expect(emptyBuiltSiteFormInput()).toEqual({ + assignable: false, + dbProvider: "bunny", + dbToken: "", + dbUrl: "", + hostingId: "", + hostingProvider: "bunny", + name: "", + siteUrl: "", + updates: "release", + }); + expect(builtSiteFormMappings).toHaveLength(9); +}); + +test("converts plain database values back to site fields", () => { + const row = { + assignable: 1, + assigned_attendee_id: 42, + assigned_listing_id: 7, + read_only_from: "date", + renewal_token_index: "index", + site_data_revision: 3, + updates: "alpha", + } as const; + const fields = Object.fromEntries( + builtSitePlainColumns.map((column) => [ + column.siteKey, + column.fromRow(row[column.dbKey] as never), + ]), + ); + expect(fields).toEqual({ + assignable: true, + assignedAttendeeId: 42, + assignedListingId: 7, + readOnlyFrom: "date", + renewalTokenIndex: "index", + siteDataRevision: 3, + updates: "alpha", + }); + expect(builtSitePlainColumns[0].fromRow(0)).toBe(false); +}); + +test("defines exact plain defaults and blob mappings", () => { + expect({ + assignedAttendeeId: builtSitePlainSchema.assigned_attendee_id.default?.(), + assignedListingId: builtSitePlainSchema.assigned_listing_id.default?.(), + readOnlyFrom: builtSitePlainSchema.read_only_from.default?.(), + renewalTokenIndex: builtSitePlainSchema.renewal_token_index.default?.(), + siteDataRevision: builtSitePlainSchema.site_data_revision.default?.(), + updates: builtSitePlainSchema.updates.default?.(), + }).toEqual({ + assignedAttendeeId: null, + assignedListingId: null, + readOnlyFrom: "", + renewalTokenIndex: null, + siteDataRevision: 0, + updates: "release", + }); + expect(builtSiteBlobColumns).toEqual([ + { + blobKey: "n", + defaultValue: "", + formDbKey: "name", + required: true, + siteKey: "name", + }, + { + blobKey: "u", + defaultValue: "", + formDbKey: "site_url", + required: true, + siteKey: "siteUrl", + }, + { + blobKey: "d", + defaultValue: "", + formDbKey: "db_url", + required: false, + siteKey: "dbUrl", + }, + { + blobKey: "t", + defaultValue: "", + formDbKey: "db_token", + required: false, + siteKey: "dbToken", + }, + { + blobKey: "s", + defaultValue: "", + formDbKey: "hosting_id", + required: false, + siteKey: "hostingId", + }, + { + blobKey: "hp", + defaultValue: "bunny", + formDbKey: "hosting_provider", + required: false, + siteKey: "hostingProvider", + }, + { + blobKey: "dp", + defaultValue: "bunny", + formDbKey: "db_provider", + required: false, + siteKey: "dbProvider", + }, + { + blobKey: "rt", + defaultValue: null, + required: false, + siteKey: "renewalToken", + }, + { + blobKey: "sk", + defaultValue: null, + required: false, + siteKey: "scheduledTaskKey", + }, + ]); +}); diff --git a/test/shared/db/built-sites/fixtures.ts b/test/shared/db/built-sites/fixtures.ts new file mode 100644 index 000000000..7f80fbfcd --- /dev/null +++ b/test/shared/db/built-sites/fixtures.ts @@ -0,0 +1,15 @@ +import type { builtSitesCrudTable } from "#shared/db/built-sites.ts"; + +type BuiltSiteFormInput = Parameters[0]; + +export const builtSiteFormInput = ( + overrides: Partial = {}, +): BuiltSiteFormInput => ({ + assignable: false, + dbToken: "", + dbUrl: "", + hostingId: "", + name: "Original", + siteUrl: "original.bunny.run", + ...overrides, +}); diff --git a/test/shared/db/built-sites/types.test.ts b/test/shared/db/built-sites/types.test.ts new file mode 100644 index 000000000..6fb17503a --- /dev/null +++ b/test/shared/db/built-sites/types.test.ts @@ -0,0 +1,45 @@ +import { expect } from "@std/expect"; +import { describe, it as test } from "@std/testing/bdd"; +import { + DEFAULT_UPDATE_TIER, + isUpdateTier, + providerOrBunny, + siteAcceptsDeployTier, + UPDATE_TIERS, +} from "#shared/db/built-sites/types.ts"; + +describe("built-site update tiers", () => { + test("orders channels from most to least eager", () => { + expect(UPDATE_TIERS).toEqual(["alpha", "beta", "release"]); + expect(DEFAULT_UPDATE_TIER).toBe("release"); + }); + + test("accepts every channel and rejects unknown values", () => { + for (const tier of UPDATE_TIERS) expect(isUpdateTier(tier)).toBe(true); + for (const value of ["", "ALPHA", "stable", "rel", "release "]) { + expect(isUpdateTier(value)).toBe(false); + } + }); + + test("matches each site channel to the deployments it accepts", () => { + expect( + UPDATE_TIERS.flatMap((siteTier) => + UPDATE_TIERS.map((deployTier) => + siteAcceptsDeployTier(siteTier, deployTier), + ), + ), + ).toEqual([true, true, true, false, true, true, false, false, true]); + }); +}); + +describe("built-site providers", () => { + test("keeps the selected non-Bunny provider", () => { + expect(providerOrBunny("deno", "deno")).toBe("deno"); + expect(providerOrBunny("turso", "turso")).toBe("turso"); + }); + + test("uses Bunny when the named provider is not selected", () => { + expect(providerOrBunny(null, "deno")).toBe("bunny"); + expect(providerOrBunny("bunny", "turso")).toBe("bunny"); + }); +}); diff --git a/test/shared/db/built-sites/update-tier.test.ts b/test/shared/db/built-sites/update-tier.test.ts new file mode 100644 index 000000000..3d711eeca --- /dev/null +++ b/test/shared/db/built-sites/update-tier.test.ts @@ -0,0 +1,94 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { + assignBuiltSite, + builtSites, + builtSitesCrudTable, + insertBuiltSite, + updateBuiltSiteRenewalState, +} from "#shared/db/built-sites.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; +import { builtSiteFormInput } from "./fixtures.ts"; + +describeWithEnv("built-site update channel", { db: true }, () => { + test("insertBuiltSite defaults the channel to release", async () => { + await insertBuiltSite("Defaulted", "defaulted.b-cdn.net"); + expect((await builtSites.getAll())[0]?.updates).toBe("release"); + }); + + test("insertBuiltSite stores an explicit channel", async () => { + await insertBuiltSite( + "Alpha Chan", + "ac.b-cdn.net", + "", + "", + false, + "", + "alpha", + ); + expect((await builtSites.getAll())[0]?.updates).toBe("alpha"); + }); + + test("CRUD insert defaults the channel to release", async () => { + expect( + await builtSitesCrudTable.insert( + builtSiteFormInput({ name: "Crud Default" }), + ), + ).toMatchObject({ updates: "release" }); + }); + + test("CRUD insert persists an explicit channel", async () => { + const site = await builtSitesCrudTable.insert( + builtSiteFormInput({ name: "Crud Beta", updates: "beta" }), + ); + expect(site.updates).toBe("beta"); + expect((await builtSites.getAll())[0]?.updates).toBe("beta"); + }); + + test("CRUD update changes the channel", async () => { + const site = await builtSitesCrudTable.insert(builtSiteFormInput()); + expect( + await builtSitesCrudTable.update(site.id, { updates: "alpha" }), + ).toMatchObject({ updates: "alpha" }); + }); + + test("CRUD update preserves the channel", async () => { + const site = await builtSitesCrudTable.insert( + builtSiteFormInput({ updates: "beta" }), + ); + expect( + await builtSitesCrudTable.update(site.id, { name: "Renamed" }), + ).toMatchObject({ name: "Renamed", updates: "beta" }); + }); + + test("assigning a site preserves its channel", async () => { + const row = await insertBuiltSite( + "Assign Chan", + "ach.b-cdn.net", + "", + "", + true, + "", + "beta", + ); + expect(await assignBuiltSite(row.id, 1, 2)).toMatchObject({ + updates: "beta", + }); + }); + + test("updating renewal state preserves the channel", async () => { + const row = await insertBuiltSite( + "Renew Chan", + "rch.b-cdn.net", + "", + "", + false, + "", + "alpha", + ); + await updateBuiltSiteRenewalState(row.id, { + readOnlyFrom: "2027-01-01T00:00:00Z", + }); + expect((await builtSites.getAll())[0]?.updates).toBe("alpha"); + }); +}); diff --git a/test/shared/db/migrations/2026-06-19_built_sites_last_pruned.test.ts b/test/shared/db/migrations/2026-06-19_built_sites_last_pruned.test.ts new file mode 100644 index 000000000..37dba4764 --- /dev/null +++ b/test/shared/db/migrations/2026-06-19_built_sites_last_pruned.test.ts @@ -0,0 +1,13 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import builtSitesLastPrunedMigration from "#shared/db/migrations/2026-06-19_built_sites_last_pruned.ts"; +import { buildMigrationContext } from "#test-utils/migrations.ts"; + +test("keeps the built-site prune marker as an inert historical migration", async () => { + const migration = builtSitesLastPrunedMigration(buildMigrationContext()); + + expect(migration.id).toBe("2026-06-19_built_sites_last_pruned"); + expect(migration.description).toBe("Historical built-site prune marker."); + expect(migration.requires).toEqual({}); + await expect(migration.up()).resolves.toBeUndefined(); +}); diff --git a/test/shared/db/migrations/2026-07-18_drop_built_sites_last_pruned.test.ts b/test/shared/db/migrations/2026-07-18_drop_built_sites_last_pruned.test.ts new file mode 100644 index 000000000..954cffb33 --- /dev/null +++ b/test/shared/db/migrations/2026-07-18_drop_built_sites_last_pruned.test.ts @@ -0,0 +1,36 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { getDb } from "#shared/db/client.ts"; +import dropLastPrunedMigration from "#shared/db/migrations/2026-07-18_drop_built_sites_last_pruned.ts"; +import { recreateTable } from "#shared/db/migrations/schema-sync.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; +import { buildMigrationContext } from "#test-utils/migrations.ts"; + +const context = buildMigrationContext({ recreateTable }); + +describeWithEnv( + "db > migrations > drop built-site prune marker", + { db: true }, + () => { + test("declares its id and description", () => { + const migration = dropLastPrunedMigration(context); + expect(migration.id).toBe("2026-07-18_drop_built_sites_last_pruned"); + expect(migration.description).toBe( + "Remove the unused built-site prune marker.", + ); + }); + + test("uses the current schema to remove only last_pruned", async () => { + await getDb().execute( + "ALTER TABLE built_sites ADD COLUMN last_pruned TEXT NOT NULL DEFAULT ''", + ); + + await dropLastPrunedMigration(context).up(); + + const columns = await getDb().execute("PRAGMA table_info(built_sites)"); + const names = columns.rows.map((row) => String(row.name)); + expect(names).not.toContain("last_pruned"); + expect(names).toContain("site_data_revision"); + }); + }, +); diff --git a/test/shared/db/migrations/2026-07-18_maintenance_tasks.test.ts b/test/shared/db/migrations/2026-07-18_maintenance_tasks.test.ts new file mode 100644 index 000000000..e4fc5c9bc --- /dev/null +++ b/test/shared/db/migrations/2026-07-18_maintenance_tasks.test.ts @@ -0,0 +1,137 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { FakeTime } from "@std/testing/time"; +import { getDb } from "#shared/db/client.ts"; +import maintenanceMigration from "#shared/db/migrations/2026-07-18_maintenance_tasks.ts"; +import { + applySchemaChanges, + syncIndexes, +} from "#shared/db/migrations/schema-sync.ts"; +import { + ACTIVITY_LOG_BACKFILL_INTERVAL_MS, + PRUNE_INTERVAL_MS, +} from "#shared/limits.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; +import { buildMigrationContext } from "#test-utils/migrations.ts"; + +const context = buildMigrationContext({ applySchemaChanges, syncIndexes }); +const runMigration = () => maintenanceMigration(context).up(); +const LEGACY_MARKERS = [ + "last_pruned_addresses", + "last_pruned_contacts", + "last_pruned_invites", + "last_pruned_logins", + "last_pruned_orphans", + "last_pruned_payments", + "last_pruned_sessions", + "last_pruned_strings", + "last_pruned_sumup", + "last_pruned_tokens", + "activity_log_backfill_done", + "last_activity_log_backfill", +] as const; + +describeWithEnv( + "db > migrations > scheduled maintenance tasks", + { db: true }, + () => { + test("declares its id, description, and schema requirements", () => { + const migration = maintenanceMigration(context); + expect(migration.id).toBe("2026-07-18_maintenance_tasks"); + expect(migration.description).toBe( + "Add durable scheduled maintenance task claims.", + ); + expect(migration.requires).toEqual({ + columns: { built_sites: ["site_data_revision"] }, + indexes: ["idx_maintenance_tasks_due"], + newTables: ["maintenance_tasks"], + }); + }); + + test("moves old due times and removes every old marker", async () => { + await getDb().batch( + [ + "DROP TABLE maintenance_tasks", + ...LEGACY_MARKERS.map((key, index) => ({ + args: [ + key, + key === "activity_log_backfill_done" + ? "false" + : String(index + 1), + ], + sql: "INSERT OR REPLACE INTO settings (key, value) VALUES (?, ?)", + })), + ], + "write", + ); + + await runMigration(); + + const tasks = await getDb().execute( + "SELECT name, next_run_at FROM maintenance_tasks ORDER BY name", + ); + expect(tasks.rows.map((row) => String(row.name))).toEqual([ + "activity_log_backfill", + "database_pruning", + ]); + expect(Number(tasks.rows[0]?.next_run_at)).toBe( + 12 + ACTIVITY_LOG_BACKFILL_INTERVAL_MS, + ); + expect(Number(tasks.rows[1]?.next_run_at)).toBe(1 + PRUNE_INTERVAL_MS); + const markers = await getDb().execute({ + args: [...LEGACY_MARKERS], + sql: `SELECT key FROM settings WHERE key IN (${LEGACY_MARKERS.map(() => "?").join(", ")})`, + }); + expect(markers.rows).toEqual([]); + }); + + test("uses database migration time when old markers are invalid", async () => { + using _time = new FakeTime(1_800_000_000_000); + await getDb().batch( + [ + "DROP TABLE maintenance_tasks", + "INSERT OR REPLACE INTO settings (key, value) VALUES ('last_pruned_sessions', '0')", + "INSERT OR REPLACE INTO settings (key, value) VALUES ('last_activity_log_backfill', 'not-a-time')", + ], + "write", + ); + + await runMigration(); + + const tasks = await getDb().execute( + "SELECT name, next_run_at FROM maintenance_tasks ORDER BY name", + ); + expect(tasks.rows).toEqual([ + { name: "activity_log_backfill", next_run_at: 1_800_000_000_000 }, + { name: "database_pruning", next_run_at: 1_800_000_000_000 }, + ]); + }); + + test("adds a revision fence to existing built sites", async () => { + await getDb().execute("DROP TABLE maintenance_tasks"); + await runMigration(); + + const columns = await getDb().execute("PRAGMA table_info(built_sites)"); + expect(columns.rows.map((row) => String(row.name))).toContain( + "site_data_revision", + ); + }); + + test("does not reschedule a backfill that was already complete", async () => { + await getDb().batch( + [ + "DROP TABLE maintenance_tasks", + "INSERT OR REPLACE INTO settings (key, value) VALUES ('activity_log_backfill_done', 'true')", + ], + "write", + ); + + await runMigration(); + + const task = await getDb().execute( + "SELECT name FROM maintenance_tasks WHERE name = 'activity_log_backfill'", + ); + expect(task.rows).toEqual([]); + }); + }, +); diff --git a/test/shared/db/migrations/2026-07-19_maintenance_checkpoint.test.ts b/test/shared/db/migrations/2026-07-19_maintenance_checkpoint.test.ts new file mode 100644 index 000000000..c5e5dc8e5 --- /dev/null +++ b/test/shared/db/migrations/2026-07-19_maintenance_checkpoint.test.ts @@ -0,0 +1,42 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { getDb } from "#shared/db/client.ts"; +import maintenanceCheckpointMigration from "#shared/db/migrations/2026-07-19_maintenance_checkpoint.ts"; +import { applySchemaChanges } from "#shared/db/migrations/schema-sync.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; +import { buildMigrationContext } from "#test-utils/migrations.ts"; + +const context = buildMigrationContext({ applySchemaChanges }); + +describeWithEnv( + "db > migrations > maintenance checkpoint", + { db: true }, + () => { + test("declares its identity and checkpoint column", () => { + const migration = maintenanceCheckpointMigration(context); + expect({ + description: migration.description, + id: migration.id, + requires: migration.requires, + }).toEqual({ + description: + "Remember where bounded maintenance scans should continue.", + id: "2026-07-19_maintenance_checkpoint", + requires: { columns: { maintenance_tasks: ["checkpoint"] } }, + }); + }); + + test("adds the checkpoint column to an existing maintenance table", async () => { + await getDb().execute( + "ALTER TABLE maintenance_tasks DROP COLUMN checkpoint", + ); + + await maintenanceCheckpointMigration(context).up(); + + const columns = await getDb().execute( + "PRAGMA table_info(maintenance_tasks)", + ); + expect(columns.rows.map((row) => row.name)).toContain("checkpoint"); + }); + }, +); diff --git a/test/shared/db/migrations/registry.test.ts b/test/shared/db/migrations/registry.test.ts index 173897052..36e8d6f47 100644 --- a/test/shared/db/migrations/registry.test.ts +++ b/test/shared/db/migrations/registry.test.ts @@ -29,4 +29,12 @@ describe("db > migration registry", () => { test("MIGRATION_IDS lists the registry ids in run order", () => { expect(MIGRATION_IDS).toEqual(MIGRATION_REGISTRY.map((entry) => entry.id)); }); + + test("orders every scheduled-maintenance schema change", () => { + expect(MIGRATION_IDS.slice(-3)).toEqual([ + "2026-07-18_maintenance_tasks", + "2026-07-18_drop_built_sites_last_pruned", + "2026-07-19_maintenance_checkpoint", + ]); + }); }); diff --git a/test/shared/db/migrations/schema/tables-core.test.ts b/test/shared/db/migrations/schema/tables-core.test.ts new file mode 100644 index 000000000..75bd4b7b6 --- /dev/null +++ b/test/shared/db/migrations/schema/tables-core.test.ts @@ -0,0 +1,36 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { coreTables } from "#shared/db/migrations/schema/tables-core.ts"; +import { jsonHash } from "#test-utils/hash.ts"; + +test("keeps the complete core schema declaration exact", async () => { + expect(await jsonHash(coreTables)).toBe( + "3d4047b1e4c102fc2e01a0d25ac5fe7a5c28707d5130f86dd2e2b3b59ddb51b0", + ); +}); + +test("defines fenced maintenance task state and its due-work index", () => { + expect(coreTables).toContainEqual([ + "maintenance_tasks", + { + columns: [ + ["name", "TEXT PRIMARY KEY"], + ["checkpoint", "TEXT"], + ["next_run_at", "INTEGER NOT NULL"], + ["lease_token", "TEXT"], + ["lease_expires_at", "INTEGER"], + ["last_started_at", "INTEGER"], + [ + "last_finished_at", + "INTEGER CHECK ((lease_token IS NULL) = (lease_expires_at IS NULL))", + ], + ], + indexes: [ + { + columns: ["next_run_at", "lease_expires_at", "name"], + name: "idx_maintenance_tasks_due", + }, + ], + }, + ]); +}); diff --git a/test/shared/db/migrations/schema/tables-questions.test.ts b/test/shared/db/migrations/schema/tables-questions.test.ts new file mode 100644 index 000000000..fab30dcd4 --- /dev/null +++ b/test/shared/db/migrations/schema/tables-questions.test.ts @@ -0,0 +1,22 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { questionTables } from "#shared/db/migrations/schema/tables-questions.ts"; +import { jsonHash } from "#test-utils/hash.ts"; + +test("keeps the complete question and built-site schema exact", async () => { + expect(await jsonHash(questionTables)).toBe( + "b2cb65571a95e6cad5cf42d2de7f88818719da14151f8eabbff2e761f8f52c85", + ); +}); + +test("versions built-site data without the retired prune marker", () => { + const builtSites = questionTables.find(([name]) => name === "built_sites"); + if (!builtSites) throw new Error("built_sites schema is missing"); + + const columns = builtSites[1].columns; + expect(columns).toContainEqual([ + "site_data_revision", + "INTEGER NOT NULL DEFAULT 0", + ]); + expect(columns.map(([name]) => name)).not.toContain("last_pruned"); +}); diff --git a/test/lib/db/migration-schema-guard.test.ts b/test/shared/db/migrations/schema/version/guard.test.ts similarity index 81% rename from test/lib/db/migration-schema-guard.test.ts rename to test/shared/db/migrations/schema/version/guard.test.ts index ea7daa915..4d6e9ed87 100644 --- a/test/lib/db/migration-schema-guard.test.ts +++ b/test/shared/db/migrations/schema/version/guard.test.ts @@ -1,6 +1,13 @@ import { expect } from "@std/expect"; import { describe, it as test } from "@std/testing/bdd"; import { MIGRATION_IDS } from "#shared/db/migrations/registry.ts"; +import { + DB_SCHEMA_HASH_KEY, + LATEST_DB_UPDATE_KEY, + LATEST_UPDATE, + MIGRATION_LOCK_KEY, + SCHEMA_MIGRATIONS_TABLE, +} from "#shared/db/migrations/schema/version.ts"; import { SCHEMA_HASH } from "#shared/db/migrations.ts"; describe("db > migrations > schema change guard", () => { @@ -88,8 +95,28 @@ describe("db > migrations > schema change guard", () => { "2026-07-15_attendee_status_integrity", "2026-07-15_checkout_stages", "2026-07-16_drop_checkout_stage_revisions", + "2026-07-18_maintenance_tasks", + "2026-07-18_drop_built_sites_last_pruned", + "2026-07-19_maintenance_checkpoint", ], - schemaHash: "19cwfaa", + schemaHash: "dhpq62", + }); + }); + + test("names the current update and schema metadata exactly", () => { + expect({ + dbSchemaHash: DB_SCHEMA_HASH_KEY, + latestDbUpdate: LATEST_DB_UPDATE_KEY, + latestUpdate: LATEST_UPDATE, + migrationLock: MIGRATION_LOCK_KEY, + schemaMigrations: SCHEMA_MIGRATIONS_TABLE, + }).toEqual({ + dbSchemaHash: "db_schema_hash", + latestDbUpdate: "latest_db_update", + latestUpdate: + "Add secure local scheduled maintenance and durable task claims.", + migrationLock: "migration_lock", + schemaMigrations: "schema_migrations", }); }); }); diff --git a/test/shared/db/prune/helpers.ts b/test/shared/db/prune/helpers.ts index e189d16b4..138f165a8 100644 --- a/test/shared/db/prune/helpers.ts +++ b/test/shared/db/prune/helpers.ts @@ -1,11 +1,8 @@ -import { expect } from "@std/expect"; import { attendeeAccount, WORLD } from "#shared/accounting/accounts.ts"; import { KIND } from "#shared/accounting/kinds.ts"; import { postTransfers } from "#shared/accounting/store.ts"; import { hmacHash } from "#shared/crypto/hashing.ts"; -import { getDb, insert } from "#shared/db/client.ts"; -import { maybeRunPrunes } from "#shared/db/prune.ts"; -import { settings } from "#shared/db/settings.ts"; +import { executeBatch, getDb, insert } from "#shared/db/client.ts"; import { nowMs } from "#shared/now.ts"; export const insertFinalizedPayment = async ( @@ -112,6 +109,22 @@ export const insertString = async ( ); }; +export const insertStrings = ( + prefix: string, + created: string, + count: number, +): Promise => + executeBatch( + Array.from({ length: count }, (_, index) => + insert("strings", { + created, + encrypted_text: "ciphertext", + text_index: `${prefix}-${index}`, + used_count: 0, + }), + ), + ); + export const stringExists = async (textIndex: string): Promise => { const { rows } = await getDb().execute({ args: [textIndex], @@ -206,37 +219,3 @@ export const attendeeExists = async (id: number): Promise => { export const oldOrphanIso = (): string => new Date(nowMs() - 365 * 24 * 60 * 60 * 1000).toISOString(); - -/** Every last-pruned stamp the scheduler tracks, one setter per table. */ -const LAST_PRUNED_SETTERS = [ - settings.update.lastPrunedPayments, - settings.update.lastPrunedSessions, - settings.update.lastPrunedLogins, - settings.update.lastPrunedTokens, - settings.update.lastPrunedSumup, - settings.update.lastPrunedStrings, - settings.update.lastPrunedContacts, - settings.update.lastPrunedAddresses, - settings.update.lastPrunedInvites, - settings.update.lastPrunedOrphans, -]; - -export const setAllLastPruned = async (value: string): Promise => { - for (const setStamp of LAST_PRUNED_SETTERS) { - await setStamp(value); - } -}; - -export const clearAllLastPruned = (): Promise => setAllLastPruned(""); - -export const expectFreshPrunedTimestampAfterRun = async ( - getLastPruned: () => string, -): Promise => { - const before = nowMs(); - - await maybeRunPrunes(); - - const ts = Number.parseInt(getLastPruned(), 10); - expect(ts).toBeGreaterThanOrEqual(before); - expect(ts).toBeLessThanOrEqual(nowMs()); -}; diff --git a/test/shared/db/prune/invites.test.ts b/test/shared/db/prune/invites.test.ts new file mode 100644 index 000000000..187c8877e --- /dev/null +++ b/test/shared/db/prune/invites.test.ts @@ -0,0 +1,157 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { getAllCacheStats } from "#shared/cache-registry.ts"; +import { executeBatch, getDb, setDb } from "#shared/db/client.ts"; +import { runDatabasePruning } from "#shared/db/prune.ts"; +import { createInvitedUser, getAllUsers } from "#shared/db/users.ts"; +import { MAINTENANCE_PRUNE_BATCH } from "#shared/limits.ts"; +import { proxyMembers } from "#shared/proxy-members.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; +import { + addUserOwnedAccessRecords, + getUserOwnedRowSources, +} from "#test-utils/user-owned-records.ts"; + +const createInvite = (username: string, expiry: string) => + createInvitedUser(username, "agent", `${username}-invite`, expiry); + +const cloneInvites = ( + sourceId: number, + prefix: string, + count: number, +): Promise => + executeBatch( + Array.from({ length: count }, (_, index) => ({ + args: [`${prefix}-${index}`, sourceId], + sql: `INSERT INTO users ( + username_hash, username_index, password_hash, wrapped_data_key, + admin_level, invite_code_hash, invite_expiry, kek_version, + invite_wrapped_data_key + ) + SELECT username_hash, ?, password_hash, wrapped_data_key, + admin_level, invite_code_hash, invite_expiry, kek_version, + invite_wrapped_data_key + FROM users WHERE id = ?`, + })), + ); + +describeWithEnv("db > expired invite pruning", { db: true }, () => { + test("rejects an invalid saved scan position", async () => { + for (const checkpoint of ["not-an-id", "0"]) { + await expect(runDatabasePruning(checkpoint)).rejects.toThrow( + `Invalid invite pruning checkpoint: ${checkpoint}`, + ); + } + }); + + test("removes every expired invite and its owned access records", async () => { + const first = await createInvite( + "expired-first", + "2000-01-01T00:00:00.000Z", + ); + const second = await createInvite( + "expired-second", + "2000-01-02T00:00:00.000Z", + ); + const future = await createInvite( + "future-invite", + "2099-01-01T00:00:00.000Z", + ); + await addUserOwnedAccessRecords(first.id, "expired-first"); + await addUserOwnedAccessRecords(future.id, "future-invite"); + expect(await getUserOwnedRowSources(first.id)).toEqual([ + "api_keys", + "sessions", + "user_logistics_agents", + "users", + ]); + expect(await getUserOwnedRowSources(second.id)).toEqual(["users"]); + + await runDatabasePruning(); + + expect(await getUserOwnedRowSources(first.id)).toEqual([]); + expect(await getUserOwnedRowSources(second.id)).toEqual([]); + expect(await getUserOwnedRowSources(future.id)).toEqual([ + "api_keys", + "sessions", + "user_logistics_agents", + "users", + ]); + }); + + test("keeps the users cache warm when no invite has expired", async () => { + await createInvite("current-invite", "2099-01-01T00:00:00.000Z"); + await getAllUsers(); + const usersStat = () => + getAllCacheStats().find(({ name }) => name === "users")?.entries; + expect(usersStat()).toBe(2); + + await runDatabasePruning(); + + expect(usersStat()).toBe(2); + }); + + test("removes an expired invite after a full batch of current invites", async () => { + const current = await createInvite( + "current-first", + "2099-01-01T00:00:00.000Z", + ); + await cloneInvites( + current.id, + "current-clone", + MAINTENANCE_PRUNE_BATCH - 1, + ); + const expired = await createInvite( + "expired-after-current", + "2000-01-01T00:00:00.000Z", + ); + + const first = await runDatabasePruning(); + + expect(first.fullBatch).toBe(true); + expect(first.checkpoint).not.toBeNull(); + expect(await getUserOwnedRowSources(expired.id)).toEqual(["users"]); + await runDatabasePruning(first.checkpoint); + expect(await getUserOwnedRowSources(expired.id)).toEqual([]); + expect(await getUserOwnedRowSources(current.id)).toEqual(["users"]); + }); + + test("keeps an invite accepted after expiry scanning", async () => { + const invite = await createInvite( + "accepted-during-prune", + "2000-01-01T00:00:00.000Z", + ); + await addUserOwnedAccessRecords(invite.id, "accepted-during-prune"); + const real = getDb(); + let accepted = false; + setDb( + proxyMembers(real, { + batch: async (...args: Parameters) => { + if (!accepted) { + accepted = true; + await real.execute({ + args: ["password-hash", "wrapped-key", invite.id], + sql: `UPDATE users + SET password_hash = ?, wrapped_data_key = ? + WHERE id = ?`, + }); + } + return real.batch(...args); + }, + }), + ); + try { + await runDatabasePruning(); + } finally { + setDb(real); + } + + expect(accepted).toBe(true); + expect(await getUserOwnedRowSources(invite.id)).toEqual([ + "api_keys", + "sessions", + "user_logistics_agents", + "users", + ]); + }); +}); diff --git a/test/shared/db/prune/payments.test.ts b/test/shared/db/prune/payments.test.ts index 51baac820..74a830a22 100644 --- a/test/shared/db/prune/payments.test.ts +++ b/test/shared/db/prune/payments.test.ts @@ -1,6 +1,6 @@ import { expect } from "@std/expect"; import { it as test } from "@std/testing/bdd"; -import { prunePayments } from "#shared/db/prune.ts"; +import { runDatabasePruning } from "#shared/db/prune.ts"; import { PRUNE_PAYMENTS_RETENTION_MS } from "#shared/limits.ts"; import { nowMs } from "#shared/now.ts"; import { describeWithEnv } from "#test-utils/db.ts"; @@ -31,7 +31,7 @@ describeWithEnv("db > prunePayments", { db: true }, () => { test("deletes old finalized payments with no useful refund reference", async () => { await insertFinalizedPayment("sess_old", oldEnoughToPrune()); - await prunePayments(); + await runDatabasePruning(); expect(await paymentExists("sess_old")).toBe(false); }); @@ -39,7 +39,7 @@ describeWithEnv("db > prunePayments", { db: true }, () => { test("keeps old finalized payments while their refund reference is useful", async () => { await insertOldReferencedPayment("sess_refund_useful"); - await prunePayments(); + await runDatabasePruning(); expect(await paymentExists("sess_refund_useful")).toBe(true); }); @@ -48,7 +48,7 @@ describeWithEnv("db > prunePayments", { db: true }, () => { const attendeeId = await insertOldReferencedPayment("sess_refund_done"); await postRefundCash(attendeeId); - await prunePayments(); + await runDatabasePruning(); expect(await paymentExists("sess_refund_done")).toBe(false); }); @@ -57,7 +57,7 @@ describeWithEnv("db > prunePayments", { db: true }, () => { const recent = new Date(nowMs() - 1000).toISOString(); await insertFinalizedPayment("sess_recent", recent); - await prunePayments(); + await runDatabasePruning(); expect(await paymentExists("sess_recent")).toBe(true); }); @@ -65,7 +65,7 @@ describeWithEnv("db > prunePayments", { db: true }, () => { test("leaves unfinalized reservations alone regardless of age", async () => { await insertUnfinalizedPayment("sess_unfinalized", oldEnoughToPrune()); - await prunePayments(); + await runDatabasePruning(); expect(await paymentExists("sess_unfinalized")).toBe(true); }); @@ -73,7 +73,7 @@ describeWithEnv("db > prunePayments", { db: true }, () => { test("deletes recorded terminal failures older than retention window", async () => { await insertFailedPayment("sess_failed_old", oldEnoughToPrune()); - await prunePayments(); + await runDatabasePruning(); expect(await paymentExists("sess_failed_old")).toBe(false); }); @@ -82,7 +82,7 @@ describeWithEnv("db > prunePayments", { db: true }, () => { const recent = new Date(nowMs() - 1000).toISOString(); await insertFailedPayment("sess_failed_recent", recent); - await prunePayments(); + await runDatabasePruning(); expect(await paymentExists("sess_failed_recent")).toBe(true); }); diff --git a/test/shared/db/prune/scheduler.test.ts b/test/shared/db/prune/scheduler.test.ts deleted file mode 100644 index 10766568d..000000000 --- a/test/shared/db/prune/scheduler.test.ts +++ /dev/null @@ -1,200 +0,0 @@ -import { expect } from "@std/expect"; -import { describe, it as test } from "@std/testing/bdd"; -import { stub } from "@std/testing/mock"; -import { FakeTime } from "@std/testing/time"; -import { getDb } from "#shared/db/client.ts"; -import { maybeRunPrunes } from "#shared/db/prune.ts"; -import { settings } from "#shared/db/settings.ts"; -import { - PRUNE_INTERVAL_MS, - PRUNE_PAYMENTS_RETENTION_MS, -} from "#shared/limits.ts"; -import { nowMs } from "#shared/now.ts"; -import { describeWithEnv } from "#test-utils/db.ts"; -import { - attendeeExists, - clearAllLastPruned, - expectFreshPrunedTimestampAfterRun, - insertFinalizedPayment, - insertOrphanAttendee, - oldOrphanIso, - paymentExists, - setAllLastPruned, -} from "./helpers.ts"; - -describeWithEnv("db > prune scheduler", { db: true }, () => { - describe("orphan auto-purge scheduling", () => { - test("maybeRunPrunes purges orphans when auto-purge is on", async () => { - await settings.update.autoPurgeOrphans(true); - await settings.update.orphanPurgeRetention("182"); - await clearAllLastPruned(); - const id = await insertOrphanAttendee(oldOrphanIso()); - - await maybeRunPrunes(); - - expect(await attendeeExists(id)).toBe(false); - expect(settings.lastPrunedOrphans).not.toBe(""); - }); - - test("maybeRunPrunes leaves orphans alone when auto-purge is off", async () => { - await settings.update.autoPurgeOrphans(false); - await clearAllLastPruned(); - const id = await insertOrphanAttendee(oldOrphanIso()); - - await maybeRunPrunes(); - - expect(await attendeeExists(id)).toBe(true); - expect(settings.lastPrunedOrphans).toBe(""); - }); - }); - - describe("maybeRunPrunes scheduler", () => { - test("records fresh payments timestamp after running", async () => { - await clearAllLastPruned(); - await expectFreshPrunedTimestampAfterRun( - () => settings.lastPrunedPayments, - ); - }); - - test("records fresh sessions timestamp after running", async () => { - await clearAllLastPruned(); - await expectFreshPrunedTimestampAfterRun( - () => settings.lastPrunedSessions, - ); - }); - - test("records fresh sumup timestamp after running", async () => { - await clearAllLastPruned(); - await expectFreshPrunedTimestampAfterRun(() => settings.lastPrunedSumup); - }); - - test("records fresh strings timestamp after running", async () => { - await clearAllLastPruned(); - await expectFreshPrunedTimestampAfterRun( - () => settings.lastPrunedStrings, - ); - }); - - test("records fresh logins timestamp after running", async () => { - await clearAllLastPruned(); - await expectFreshPrunedTimestampAfterRun(() => settings.lastPrunedLogins); - }); - - test("records fresh tokens timestamp after running", async () => { - await clearAllLastPruned(); - await expectFreshPrunedTimestampAfterRun(() => settings.lastPrunedTokens); - }); - - test("records fresh contacts timestamp after running", async () => { - await clearAllLastPruned(); - await expectFreshPrunedTimestampAfterRun( - () => settings.lastPrunedContacts, - ); - }); - - test("records fresh address-cache timestamp after running", async () => { - await clearAllLastPruned(); - await expectFreshPrunedTimestampAfterRun( - () => settings.lastPrunedAddresses, - ); - }); - - test("records fresh invites timestamp after running", async () => { - await clearAllLastPruned(); - await expectFreshPrunedTimestampAfterRun( - () => settings.lastPrunedInvites, - ); - }); - - test("skips tasks not yet due since last run", async () => { - await setAllLastPruned(String(nowMs())); - const old = new Date( - nowMs() - PRUNE_PAYMENTS_RETENTION_MS - 60_000, - ).toISOString(); - await insertFinalizedPayment("sess_skip", old); - - await maybeRunPrunes(); - - expect(await paymentExists("sess_skip")).toBe(true); - }); - - test("runs tasks when last-run is older than the interval", async () => { - const old = new Date( - nowMs() - PRUNE_PAYMENTS_RETENTION_MS - 60_000, - ).toISOString(); - await insertFinalizedPayment("sess_due", old); - await setAllLastPruned(String(nowMs() - PRUNE_INTERVAL_MS - 60_000)); - - await maybeRunPrunes(); - - expect(await paymentExists("sess_due")).toBe(false); - }); - - test("one task's failure does not block the others", async () => { - const old = new Date( - nowMs() - PRUNE_PAYMENTS_RETENTION_MS - 60_000, - ).toISOString(); - await insertFinalizedPayment("sess_isolation", old); - await clearAllLastPruned(); - await getDb().execute("DROP TABLE sessions"); - - await maybeRunPrunes(); - - expect(await paymentExists("sess_isolation")).toBe(false); - }); - - test("does not prune when the marker batch cannot be written", async () => { - const old = new Date( - nowMs() - PRUNE_PAYMENTS_RETENTION_MS - 60_000, - ).toISOString(); - await insertFinalizedPayment("sess_marker_failure", old); - await clearAllLastPruned(); - using batchStub = stub(getDb(), "batch", () => - Promise.reject(new Error("marker write failed")), - ); - - await maybeRunPrunes(); - - expect(batchStub.calls).toHaveLength(1); - expect(await paymentExists("sess_marker_failure")).toBe(true); - expect(settings.lastPrunedPayments).toBe(""); - }); - - test("treats an invalid last-pruned value as never-run", async () => { - await settings.update.lastPrunedPayments("not-a-number"); - await expectFreshPrunedTimestampAfterRun( - () => settings.lastPrunedPayments, - ); - }); - - test("concurrent calls leave the DB in a consistent state", async () => { - const old = new Date( - nowMs() - PRUNE_PAYMENTS_RETENTION_MS - 60_000, - ).toISOString(); - await insertFinalizedPayment("sess_concurrent", old); - await clearAllLastPruned(); - - await Promise.all([maybeRunPrunes(), maybeRunPrunes(), maybeRunPrunes()]); - - expect(await paymentExists("sess_concurrent")).toBe(false); - }); - }); - - test("a task becomes due exactly PRUNE_INTERVAL_MS after its last run", async () => { - const start = 1_700_000_000_000; - using time = new FakeTime(start); - await setAllLastPruned(String(start)); - const old = new Date( - start - PRUNE_PAYMENTS_RETENTION_MS - 60_000, - ).toISOString(); - await insertFinalizedPayment("sess_interval", old); - - time.tick(PRUNE_INTERVAL_MS - 1); - await maybeRunPrunes(); - expect(await paymentExists("sess_interval")).toBe(true); - - time.tick(1); - await maybeRunPrunes(); - expect(await paymentExists("sess_interval")).toBe(false); - }); -}); diff --git a/test/shared/db/prune/tables.test.ts b/test/shared/db/prune/tables.test.ts index 80eb396bc..48512582b 100644 --- a/test/shared/db/prune/tables.test.ts +++ b/test/shared/db/prune/tables.test.ts @@ -1,17 +1,11 @@ import { expect } from "@std/expect"; import { describe, it as test } from "@std/testing/bdd"; -import { - pruneContacts, - pruneLoginAttempts, - pruneOrphanAttendees, - pruneSessions, - pruneSumupCheckouts, - pruneTokenAttempts, - pruneUnusedStrings, -} from "#shared/db/prune.ts"; +import { stub } from "@std/testing/mock"; +import { runDatabasePruning } from "#shared/db/prune.ts"; import { createSession, getAllSessions } from "#shared/db/sessions.ts"; import { settings } from "#shared/db/settings.ts"; import { + MAINTENANCE_PRUNE_BATCH, PRUNE_CONTACTS_RETENTION_MS, PRUNE_LOGINS_RETENTION_MS, PRUNE_SESSIONS_RETENTION_MS, @@ -19,6 +13,7 @@ import { PRUNE_TOKENS_RETENTION_MS, PRUNE_UNUSED_STRINGS_RETENTION_MS, } from "#shared/limits.ts"; +import { setSuppressDebugLogs } from "#shared/log-settings.ts"; import { nowMs } from "#shared/now.ts"; import { describeWithEnv } from "#test-utils/db.ts"; import { @@ -28,6 +23,7 @@ import { insertLoginAttempt, insertOrphanAttendee, insertString, + insertStrings, insertSumupCheckout, insertTokenAttempt, loginAttemptExists, @@ -38,6 +34,49 @@ import { } from "./helpers.ts"; describeWithEnv("db > table pruning", { db: true }, () => { + test("reports a drained pruning run", async () => { + expect(await runDatabasePruning()).toEqual({ + checkpoint: null, + fullBatch: false, + }); + }); + + test("reports a full bounded batch when stale rows remain", async () => { + const old = new Date( + nowMs() - PRUNE_UNUSED_STRINGS_RETENTION_MS - 60_000, + ).toISOString(); + await insertStrings("prune-backlog", old, MAINTENANCE_PRUNE_BATCH + 1); + + const result = await runDatabasePruning(); + + expect(result).toEqual({ checkpoint: null, fullBatch: true }); + expect(await stringExists(`prune-backlog-${MAINTENANCE_PRUNE_BATCH}`)).toBe( + true, + ); + }); + + test("logs the exact total number of deleted rows", async () => { + setSuppressDebugLogs(false); + const debugStub = stub(console, "debug"); + try { + const old = new Date( + nowMs() - PRUNE_SUMUP_RETENTION_MS - 60_000, + ).toISOString(); + await insertSumupCheckout("idx_only", old); + + await runDatabasePruning(); + + const pruneLogs = debugStub.calls + .map((call) => String(call.args[0])) + .filter((line) => line.includes("[Prune]")); + expect(pruneLogs).toHaveLength(1); + expect(pruneLogs[0]).toMatch(/\[Prune\] deleted 1 expired rows$/); + } finally { + debugStub.restore(); + setSuppressDebugLogs(null); + } + }); + describe("pruneSumupCheckouts", () => { test("deletes checkout metadata older than retention window", async () => { const old = new Date( @@ -45,7 +84,7 @@ describeWithEnv("db > table pruning", { db: true }, () => { ).toISOString(); await insertSumupCheckout("idx_old", old); - await pruneSumupCheckouts(); + await runDatabasePruning(); expect(await sumupCheckoutExists("idx_old")).toBe(false); }); @@ -54,7 +93,7 @@ describeWithEnv("db > table pruning", { db: true }, () => { const recent = new Date(nowMs() - 1000).toISOString(); await insertSumupCheckout("idx_recent", recent); - await pruneSumupCheckouts(); + await runDatabasePruning(); expect(await sumupCheckoutExists("idx_recent")).toBe(true); }); @@ -67,7 +106,7 @@ describeWithEnv("db > table pruning", { db: true }, () => { ).toISOString(); await insertString("string_old_unused", old, 0); - await pruneUnusedStrings(); + await runDatabasePruning(); expect(await stringExists("string_old_unused")).toBe(false); }); @@ -76,7 +115,7 @@ describeWithEnv("db > table pruning", { db: true }, () => { const recent = new Date(nowMs() - 1000).toISOString(); await insertString("string_recent_unused", recent, 0); - await pruneUnusedStrings(); + await runDatabasePruning(); expect(await stringExists("string_recent_unused")).toBe(true); }); @@ -87,7 +126,7 @@ describeWithEnv("db > table pruning", { db: true }, () => { ).toISOString(); await insertString("string_old_used", old, 1); - await pruneUnusedStrings(); + await runDatabasePruning(); expect(await stringExists("string_old_used")).toBe(true); }); @@ -98,7 +137,7 @@ describeWithEnv("db > table pruning", { db: true }, () => { const expiredMs = nowMs() - PRUNE_SESSIONS_RETENTION_MS - 60_000; await createSession("stale-tok", "csrf-stale", expiredMs, null, 1); - await pruneSessions(); + await runDatabasePruning(); const remaining = await getAllSessions(); expect(remaining.map((session) => session.csrf_token)).not.toContain( @@ -115,7 +154,7 @@ describeWithEnv("db > table pruning", { db: true }, () => { 1, ); - await pruneSessions(); + await runDatabasePruning(); const remaining = await getAllSessions(); expect(remaining.map((session) => session.csrf_token)).toContain( @@ -132,7 +171,7 @@ describeWithEnv("db > table pruning", { db: true }, () => { 1, ); - await pruneSessions(); + await runDatabasePruning(); const remaining = await getAllSessions(); expect(remaining.map((session) => session.csrf_token)).toContain( @@ -149,7 +188,7 @@ describeWithEnv("db > table pruning", { db: true }, () => { nowMs() - PRUNE_LOGINS_RETENTION_MS - 60_000, ); - await pruneLoginAttempts(); + await runDatabasePruning(); expect(await loginAttemptExists(ipHash)).toBe(false); }); @@ -157,7 +196,7 @@ describeWithEnv("db > table pruning", { db: true }, () => { test("keeps counter-only rows (locked_until IS NULL)", async () => { const ipHash = await insertLoginAttempt("5.6.7.8", 2, null); - await pruneLoginAttempts(); + await runDatabasePruning(); expect(await loginAttemptExists(ipHash)).toBe(true); }); @@ -169,7 +208,7 @@ describeWithEnv("db > table pruning", { db: true }, () => { nowMs() + 60_000, ); - await pruneLoginAttempts(); + await runDatabasePruning(); expect(await loginAttemptExists(ipHash)).toBe(true); }); @@ -180,7 +219,7 @@ describeWithEnv("db > table pruning", { db: true }, () => { const stale = nowMs() - PRUNE_TOKENS_RETENTION_MS - 60_000; const ipHash = await insertTokenAttempt("13.14.15.16", null, stale); - await pruneTokenAttempts(); + await runDatabasePruning(); expect(await tokenAttemptExists(ipHash)).toBe(false); }); @@ -188,7 +227,7 @@ describeWithEnv("db > table pruning", { db: true }, () => { test("keeps rows with a recent last_attempt", async () => { const ipHash = await insertTokenAttempt("17.18.19.20", null, nowMs()); - await pruneTokenAttempts(); + await runDatabasePruning(); expect(await tokenAttemptExists(ipHash)).toBe(true); }); @@ -201,7 +240,7 @@ describeWithEnv("db > table pruning", { db: true }, () => { stale, ); - await pruneTokenAttempts(); + await runDatabasePruning(); expect(await tokenAttemptExists(ipHash)).toBe(false); }); @@ -212,7 +251,7 @@ describeWithEnv("db > table pruning", { db: true }, () => { const stale = nowMs() - PRUNE_CONTACTS_RETENTION_MS - 60_000; await insertContactPreference("contact_old", 0, stale); - await pruneContacts(); + await runDatabasePruning(); expect(await contactPreferenceExists("contact_old")).toBe(false); }); @@ -220,7 +259,7 @@ describeWithEnv("db > table pruning", { db: true }, () => { test("keeps subscribed rows within the retention window", async () => { await insertContactPreference("contact_recent", 0, nowMs()); - await pruneContacts(); + await runDatabasePruning(); expect(await contactPreferenceExists("contact_recent")).toBe(true); }); @@ -229,18 +268,27 @@ describeWithEnv("db > table pruning", { db: true }, () => { const stale = nowMs() - PRUNE_CONTACTS_RETENTION_MS - 60_000; await insertContactPreference("contact_opt_out", 1, stale); - await pruneContacts(); + await runDatabasePruning(); expect(await contactPreferenceExists("contact_opt_out")).toBe(true); }); }); describe("pruneOrphanAttendees", () => { + test("keeps old orphans while automatic purging is off", async () => { + await settings.update.autoPurgeOrphans(false); + const id = await insertOrphanAttendee(oldOrphanIso()); + + await runDatabasePruning(); + + expect(await attendeeExists(id)).toBe(true); + }); + test("deletes orphans older than the configured retention", async () => { await settings.update.orphanPurgeRetention("182"); const id = await insertOrphanAttendee(oldOrphanIso()); - await pruneOrphanAttendees(); + await runDatabasePruning(); expect(await attendeeExists(id)).toBe(false); }); @@ -249,7 +297,7 @@ describeWithEnv("db > table pruning", { db: true }, () => { await settings.update.orphanPurgeRetention("1825"); const id = await insertOrphanAttendee(oldOrphanIso()); - await pruneOrphanAttendees(); + await runDatabasePruning(); expect(await attendeeExists(id)).toBe(true); }); diff --git a/test/shared/db/query-log.test.ts b/test/shared/db/query-log.test.ts index f9c041e08..5fa2e3777 100644 --- a/test/shared/db/query-log.test.ts +++ b/test/shared/db/query-log.test.ts @@ -21,10 +21,21 @@ import { // mirror is a cache hit — keeping their fire-and-forget flush deterministic // rather than time-dependent. import { setSuppressDebugLogs } from "#shared/log-settings.ts"; -import { BUNNY_SUBREQUEST_LIMIT } from "#shared/subrequest-budget.ts"; +import { + BUNNY_SUBREQUEST_LIMIT, + getSubrequestUsage, + runWithSubrequestBudget, +} from "#shared/subrequest-budget.ts"; describe("query-log", () => { describe("enableQueryLog resets previous entries", () => { + test("does not record before logging is enabled", async () => { + await runWithQueryLogContext(async () => { + await trackSql("SELECT hidden", () => Promise.resolve()); + expect(getQueryLog()).toEqual([]); + }); + }); + test("clears log on enable", async () => { await runWithQueryLogContext(async () => { enableQueryLog(); @@ -84,6 +95,10 @@ describe("query-log", () => { expect(sqlWallClockMs([entry(100, 20), entry(105, 5)])).toBe(20); }); + test("sorts by start time when a contained query is listed first", () => { + expect(sqlWallClockMs([entry(105, 5), entry(100, 20)])).toBe(20); + }); + test("counts a shared batch round-trip window once", () => { // Batch statements share one [start, start+elapsed] window. const batch = [entry(100, 10), entry(100, 10), entry(100, 10)]; @@ -361,6 +376,17 @@ describe("query-log", () => { countRoundTrips(BUNNY_SUBREQUEST_LIMIT + 1, "startup"), ).not.toThrow(); }); + + test("counts database calls in the combined subrequest budget", () => { + runWithSubrequestBudget(() => { + countDatabaseRoundTrip("scheduled claim"); + expect(getSubrequestUsage()).toEqual({ + database: 1, + external: 0, + total: 1, + }); + }); + }); }); describe("transaction round-trip guard", () => { @@ -381,7 +407,9 @@ describe("query-log", () => { TRANSACTION_ROUNDTRIP_THRESHOLD + 1, "INSERT INTO t VALUES (1)", ), - ).toThrow(/Interactive transaction too chatty/); + ).toThrow( + "Interactive transaction too chatty: 31 statements (limit 30) held the write lock open — prepare reads outside the transaction and apply the writes as one batch. Last statement: INSERT INTO t VALUES (1)", + ); }); }); diff --git a/test/shared/db/retry-write.test.ts b/test/shared/db/retry-write.test.ts new file mode 100644 index 000000000..f171756b3 --- /dev/null +++ b/test/shared/db/retry-write.test.ts @@ -0,0 +1,25 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { retryWrite } from "#shared/db/retry-write.ts"; + +test("retryWrite returns the first revision-fenced write that succeeds", async () => { + let attempts = 0; + const result = await retryWrite("failed", () => { + attempts += 1; + return Promise.resolve(attempts === 3 ? { value: "saved" } : null); + }); + + expect(result).toBe("saved"); + expect(attempts).toBe(3); +}); + +test("retryWrite fails after four lost races", async () => { + let attempts = 0; + await expect( + retryWrite("Could not save", () => { + attempts += 1; + return Promise.resolve(null); + }), + ).rejects.toThrow("Could not save"); + expect(attempts).toBe(4); +}); diff --git a/test/shared/db/settings.test.ts b/test/shared/db/settings.test.ts index 33d558e5c..66c6ce7e7 100644 --- a/test/shared/db/settings.test.ts +++ b/test/shared/db/settings.test.ts @@ -26,6 +26,17 @@ import { seedCountry, testWithSetting } from "#test-utils/settings.ts"; describeWithEnv("db > settings", { db: true }, () => { describe("basic CRUD", () => { + test("excludes retired maintenance timestamps from settings snapshots", () => { + expect( + ALL_SETTINGS_KEYS.filter( + (key) => + key.startsWith("last_pruned_") || + key === "last_activity_log_backfill" || + key === "activity_log_backfill_done", + ), + ).toEqual([]); + }); + test("getSetting returns null for missing key", () => { const value = settings.getCachedRaw("missing"); expect(value).toBeNull(); diff --git a/test/shared/db/settings/public-api.test.ts b/test/shared/db/settings/public-api.test.ts new file mode 100644 index 000000000..88d8c63d9 --- /dev/null +++ b/test/shared/db/settings/public-api.test.ts @@ -0,0 +1,148 @@ +import { expect } from "@std/expect"; +import { describe, it as test } from "@std/testing/bdd"; +import { TEMPLATE_KEYS } from "#shared/db/settings/apply.ts"; +import { CONFIG_KEYS, settings } from "#shared/db/settings.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; + +describeWithEnv("db > settings public API", { db: true }, () => { + test("reports absent provider credentials as not configured", () => { + settings.setForTest({ + address_lookup_api_key: "", + email_api_key: "", + sms_gateway_passphrase: "", + sms_gateway_password: "", + sms_gateway_webhook_secret: "", + square_access_token: "", + }); + + expect({ + addressLookup: settings.addressLookup.hasKey, + email: settings.email.hasApiKey, + smsPassphrase: settings.smsGateway.hasPassphrase, + smsPassword: settings.smsGateway.hasPassword, + smsWebhook: settings.smsGateway.hasWebhookSecret, + square: settings.square.hasToken, + }).toEqual({ + addressLookup: false, + email: false, + smsPassphrase: false, + smsPassword: false, + smsWebhook: false, + square: false, + }); + }); + + test("returns the stored listings calendar grouping", () => { + settings.setForTest({ calendar_feeds_group_by: "listings" }); + + expect(settings.calendarFeedsGroupBy).toBe("listings"); + }); + + describe("email templates", () => { + const templateKey = TEMPLATE_KEYS["confirmation:subject"]; + const subject = "Your updated ticket"; + + test("updates the current snapshot", async () => { + await settings.update.email.template("confirmation", "subject", subject); + + expect(settings.email.template("confirmation", "subject")).toBe(subject); + }); + + test("persists an encrypted template", async () => { + await settings.update.email.template("confirmation", "subject", subject); + settings.invalidateCache(); + await settings.loadKeys([templateKey]); + + expect(settings.email.template("confirmation", "subject")).toBe(subject); + expect(settings.getCachedRaw(templateKey)).toMatch(/^enc:1:/); + }); + }); + + describe("listing defaults", () => { + const defaults = { hidden: true, minimumDaysBefore: 4 } as const; + + test("updates the current snapshot", async () => { + await settings.update.listingDefaults(defaults); + + expect(settings.listingDefaults).toEqual(defaults); + }); + + test("persists encrypted defaults", async () => { + await settings.update.listingDefaults(defaults); + settings.invalidateCache(); + await settings.loadKeys([CONFIG_KEYS.LISTING_DEFAULTS]); + + expect(settings.listingDefaults).toEqual(defaults); + expect(settings.getCachedRaw(CONFIG_KEYS.LISTING_DEFAULTS)).toMatch( + /^enc:1:/, + ); + }); + }); + + describe("payment provider", () => { + test("replaces the current provider", async () => { + await settings.update.paymentProvider("square"); + await settings.update.paymentProvider("stripe"); + + expect(settings.paymentProvider).toBe("stripe"); + expect(settings.paymentProviderSetting).toBe("stripe"); + }); + + test("sets the current provider to none", async () => { + await settings.update.paymentProvider("stripe"); + await settings.update.setPaymentProviderNone(); + + expect(settings.paymentProvider).toBeNull(); + expect(settings.paymentProviderSetting).toBe("none"); + }); + + test("persists the explicit none setting", async () => { + await settings.update.paymentProvider("stripe"); + await settings.update.setPaymentProviderNone(); + settings.invalidateCache(); + await settings.loadKeys([CONFIG_KEYS.PAYMENT_PROVIDER]); + + expect(settings.paymentProvider).toBeNull(); + expect(settings.paymentProviderSetting).toBe("none"); + }); + }); + + describe("Stripe activation", () => { + const current = { + secretKey: "sk_test_current", + webhookEndpointId: "we_current", + webhookSecret: "whsec_current", + }; + const replacement = { + secretKey: "sk_test_replacement", + webhookEndpointId: "we_replacement", + webhookSecret: "whsec_replacement", + }; + + test("replaces the current credentials and provider", async () => { + await settings.update.stripe.activate(current); + await settings.update.stripe.activate(replacement); + + expect({ + paymentProvider: settings.paymentProvider, + paymentProviderSetting: settings.paymentProviderSetting, + secretKey: settings.stripe.secretKey, + webhookEndpointId: settings.stripe.webhookEndpointId, + webhookSecret: settings.stripe.webhookSecret, + }).toEqual({ + paymentProvider: "stripe", + paymentProviderSetting: "stripe", + ...replacement, + }); + }); + + test("persists the Stripe provider selection", async () => { + await settings.update.stripe.activate(replacement); + settings.invalidateCache(); + await settings.loadKeys([CONFIG_KEYS.PAYMENT_PROVIDER]); + + expect(settings.paymentProvider).toBe("stripe"); + expect(settings.paymentProviderSetting).toBe("stripe"); + }); + }); +}); diff --git a/test/shared/db/users/contracts.test.ts b/test/shared/db/users/contracts.test.ts index aff7f5a9a..afc0cb581 100644 --- a/test/shared/db/users/contracts.test.ts +++ b/test/shared/db/users/contracts.test.ts @@ -12,6 +12,7 @@ import { createUser, decryptAdminLevel, decryptUsername, + deleteUser, getAllUsers, getUserAuthFieldsById, getUserById, @@ -28,6 +29,10 @@ import { import { describeWithEnv } from "#test-utils/db.ts"; import { TEST_ADMIN_USERNAME } from "#test-utils/internal.ts"; import { recordQueries } from "#test-utils/record-queries.ts"; +import { + addUserOwnedAccessRecords, + getUserOwnedRowSources, +} from "#test-utils/user-owned-records.ts"; describeWithEnv("db > users contracts", { db: true }, () => { test("returns every display field in user id order", async () => { @@ -232,4 +237,24 @@ describeWithEnv("db > users contracts", { db: true }, () => { expect(migrated.kek_version).toBe(2); expect(migrated.wrapped_data_key).not.toBeNull(); }); + + test("deletes the user and every owned access record", async () => { + const user = await createInvitedUser( + "deleted-agent", + "agent", + "delete-invite", + "2099-01-01T00:00:00.000Z", + ); + await addUserOwnedAccessRecords(user.id, "delete"); + expect(await getUserOwnedRowSources(user.id)).toEqual([ + "api_keys", + "sessions", + "user_logistics_agents", + "users", + ]); + + await deleteUser(user.id); + + expect(await getUserOwnedRowSources(user.id)).toEqual([]); + }); }); diff --git a/test/shared/deno-deploy-api.test.ts b/test/shared/deno-deploy-api.test.ts index 3e8547c92..0d58bdfb0 100644 --- a/test/shared/deno-deploy-api.test.ts +++ b/test/shared/deno-deploy-api.test.ts @@ -1,6 +1,10 @@ import { expect } from "@std/expect"; import { it as test } from "@std/testing/bdd"; -import { denoDeployApi, slugifyForDeno } from "#shared/deno-deploy-api.ts"; +import { + denoDeployApi, + denoHostingProvider, + slugifyForDeno, +} from "#shared/deno-deploy-api.ts"; import { describeWithEnv } from "#test-utils/db.ts"; import { stubFetch } from "#test-utils/fetch-stub.ts"; @@ -11,6 +15,8 @@ const DENO_ENV = { interface CapturedRequest { body: unknown; + contentType?: string | null; + method?: string; url: string | undefined; } @@ -20,6 +26,8 @@ const captureRequest = (url: string, init?: RequestInit): Response => { captured.url = url; captured.body = JSON.parse(init?.body as string); + captured.contentType = new Headers(init?.headers).get("content-type"); + if (init?.method !== undefined) captured.method = init.method; return new Response(JSON.stringify(responseBody)); }; @@ -52,13 +60,11 @@ describeWithEnv("deno-deploy-api", { env: DENO_ENV }, () => { }); test("slugifyForDeno pads short slugs to at least 3 chars", () => { - const result = slugifyForDeno("ab"); - expect(result.length).toBeGreaterThanOrEqual(3); + expect(slugifyForDeno("ab")).toBe("abapp"); }); test("slugifyForDeno handles single-char input", () => { - const result = slugifyForDeno("a"); - expect(result.length).toBeGreaterThanOrEqual(3); + expect(slugifyForDeno("a")).toBe("aapp"); }); // ── createApp ────────────────────────────────────────────────────────────── @@ -76,6 +82,8 @@ describeWithEnv("deno-deploy-api", { env: DENO_ENV }, () => { } expect(captured.url).toContain("/v2/apps"); expect(captured.body).toEqual({ orgId: "test-org-id", slug: "my-app" }); + expect(captured.contentType).toBe("application/json"); + expect(captured.method).toBe("POST"); }); test("createApp uses Bearer auth header", async () => { @@ -106,24 +114,33 @@ describeWithEnv("deno-deploy-api", { env: DENO_ENV }, () => { // ── setEnvVars ───────────────────────────────────────────────────────────── test("setEnvVars PATCHes only the supplied vars (no GET)", async () => { - let patchUrl: string | undefined; - let patchBody: unknown; + const captured: CapturedRequest = { body: undefined, url: undefined }; let callCount = 0; using _fetch = stubFetch((url, init) => { callCount++; - patchUrl = url; - patchBody = JSON.parse(init?.body as string); - return new Response(JSON.stringify({ id: "app_ev", slug: "env-app" })); + return captureRequest({ id: "app_ev", slug: "env-app" }, captured)( + url, + init, + ); }); const result = await denoDeployApi.setEnvVars("app_ev", [ ["NEW_VAR", "new-value"], ]); expect(result.ok).toBe(true); expect(callCount).toBe(1); - expect(patchUrl).toContain("/apps/app_ev"); - const envVars = (patchBody as { env_vars: { key: string }[] }).env_vars; - expect(envVars.map((e) => e.key)).toContain("NEW_VAR"); + expect(captured.url).toContain("/apps/app_ev"); + expect(captured.method).toBe("PATCH"); + expect(captured.body).toEqual({ + env_vars: [ + { + contexts: ["production"], + key: "NEW_VAR", + secret: true, + value: "new-value", + }, + ], + }); }); test("setEnvVars returns error when PATCH fails", async () => { @@ -142,7 +159,7 @@ describeWithEnv("deno-deploy-api", { env: DENO_ENV }, () => { // ── deployCode ───────────────────────────────────────────────────────────── - test("deployCode POSTs assets and config to /deployments endpoint", async () => { + test("deployCode POSTs assets and config to the revision endpoint", async () => { const captured: CapturedRequest = { body: undefined, url: undefined }; using _fetch = stubFetch( captureRequest({ domains: ["my-app.deno.dev"], id: "dep_123" }, captured), @@ -151,38 +168,20 @@ describeWithEnv("deno-deploy-api", { env: DENO_ENV }, () => { "app_dc", "console.log('hello')", ); - expect(result.ok).toBe(true); - if (result.ok) { - expect(result.value).toBe("https://my-app.deno.dev"); - } - expect(captured.url).toContain("/apps/app_dc/deployments"); - expect( - (captured.body as { assets: { "main.ts": { content: string } } }).assets[ - "main.ts" - ].content, - ).toBe("console.log('hello')"); - }); - - test("deployCode falls back to hostnames when domains is empty", async () => { - using _fetch = stubFetch( - new Response( - JSON.stringify({ hostnames: ["fallback.deno.dev"], id: "dep_fb" }), - ), - ); - const result = await denoDeployApi.deployCode("app_fb", "code"); - expect(result.ok).toBe(true); - if (result.ok) { - expect(result.value).toBe("https://fallback.deno.dev"); - } - }); - - test("deployCode returns error when response has no hostname", async () => { - using _fetch = stubFetch(new Response(JSON.stringify({ id: "dep_nh" }))); - const result = await denoDeployApi.deployCode("app_nh", "code"); - expect(result.ok).toBe(false); - if (!result.ok) { - expect(result.error).toContain("no hostname"); - } + expect(result).toEqual({ ok: true, value: undefined }); + expect(captured.url).toBe("https://api.deno.com/v2/apps/app_dc/deploy"); + expect(captured.method).toBe("POST"); + expect(captured.body).toEqual({ + assets: { + "main.ts": { + content: "console.log('hello')", + encoding: "utf-8", + kind: "file", + }, + }, + config: { runtime: { entrypoint: "main.ts", type: "dynamic" } }, + production: true, + }); }); test("deployCode returns error when API responds with failure", async () => { @@ -258,4 +257,27 @@ describeWithEnv("deno-deploy-api", { env: DENO_ENV }, () => { expect(result.error).toContain("app not found"); } }); + + test("hosting provider names its required credential", () => { + expect(denoHostingProvider.configEnvVar).toBe("DENO_DEPLOY_TOKEN"); + }); + + test("hosting publish returns only provider success", async () => { + using _fetch = stubFetch( + new Response(JSON.stringify({ domains: ["child.deno.dev"], id: "dep" })), + ); + + expect(await denoHostingProvider.publishSite("app_1", "code")).toEqual({ + ok: true, + }); + }); + + test("hosting publish preserves provider failure", async () => { + using _fetch = stubFetch(new Response("failed", { status: 500 })); + + expect(await denoHostingProvider.publishSite("app_1", "code")).toEqual({ + error: "Deploy code failed (500): failed", + ok: false, + }); + }); }); diff --git a/test/shared/fetch.test.ts b/test/shared/fetch.test.ts index d9907b934..9c1e1662b 100644 --- a/test/shared/fetch.test.ts +++ b/test/shared/fetch.test.ts @@ -1,8 +1,19 @@ import { expect } from "@std/expect"; import { describe, it as test } from "@std/testing/bdd"; -import { fetchText, parseApiError } from "#shared/fetch.ts"; +import { fetchText, jsonHeaders, parseApiError } from "#shared/fetch.ts"; +import { + getSubrequestUsage, + runWithSubrequestBudget, +} from "#shared/subrequest-budget.ts"; import { stubFetch } from "#test-utils/fetch-stub.ts"; +test("adds the JSON content type to authentication headers", () => { + expect(jsonHeaders({ Authorization: "Bearer token" })).toEqual({ + Authorization: "Bearer token", + "Content-Type": "application/json", + }); +}); + describe("fetchText", () => { test("returns status, ok, text, and headers from a successful response", async () => { using _fetch = stubFetch( @@ -66,6 +77,19 @@ describe("fetchText", () => { "Network error", ); }); + + test("counts one external subrequest", async () => { + using _fetch = stubFetch(new Response("ok")); + + await runWithSubrequestBudget(async () => { + await fetchText("https://example.com/api"); + expect(getSubrequestUsage()).toEqual({ + database: 0, + external: 1, + total: 1, + }); + }); + }); }); describe("parseApiError", () => { diff --git a/test/shared/interval-gate.test.ts b/test/shared/interval-gate.test.ts deleted file mode 100644 index 5ffb7b8e3..000000000 --- a/test/shared/interval-gate.test.ts +++ /dev/null @@ -1,28 +0,0 @@ -import { expect } from "@std/expect"; -import { describe, test } from "@std/testing/bdd"; -import { taskIsDue } from "#shared/interval-gate.ts"; - -describe("taskIsDue", () => { - test("is due when the interval has fully elapsed since the last run", () => { - expect(taskIsDue("1000", 500, 1500)).toBe(true); - }); - - test("is due exactly at the boundary", () => { - expect(taskIsDue("1000", 500, 1500)).toBe(true); - expect(taskIsDue("1000", 501, 1500)).toBe(false); - }); - - test("is not due when less than the interval has passed", () => { - expect(taskIsDue("1000", 500, 1400)).toBe(false); - }); - - test("treats an empty last-run value as never run, so it is due now", () => { - // Empty parses to 0, so any now at least one interval past the epoch is due. - expect(taskIsDue("", 500, 500)).toBe(true); - expect(taskIsDue("", 500, 499)).toBe(false); - }); - - test("treats an unreadable last-run value as never run", () => { - expect(taskIsDue("not-a-number", 500, 500)).toBe(true); - }); -}); diff --git a/test/shared/limits.test.ts b/test/shared/limits.test.ts index 1b23db50b..1316b42d3 100644 --- a/test/shared/limits.test.ts +++ b/test/shared/limits.test.ts @@ -15,6 +15,7 @@ import { formatSeconds, LIMIT_ENTRIES, LOGIN_LOCKOUT_MS, + MAINTENANCE_PRUNE_BATCH, MAX_ADDRESS_LOOKUPS, MAX_ATTACHMENT_SIZE, MAX_BACKUPS, @@ -92,6 +93,10 @@ describe("limits", () => { }); describe("assertPaymentsRetentionSafe", () => { + test("uses the providers' three-day webhook retry window", () => { + expect(WEBHOOK_RETRY_WINDOW_DAYS).toBe(3); + }); + test("returns the value when it meets the webhook-retry floor", () => { expect(assertPaymentsRetentionSafe(WEBHOOK_RETRY_WINDOW_DAYS)).toBe( WEBHOOK_RETRY_WINDOW_DAYS, @@ -105,7 +110,13 @@ describe("limits", () => { // the paid session and risking a duplicate refund — so it must fail loudly. expect(() => assertPaymentsRetentionSafe(WEBHOOK_RETRY_WINDOW_DAYS - 1), - ).toThrow("webhook-retry window"); + ).toThrow( + "PRUNE_PAYMENTS_RETENTION_DAYS=2 is below the 3-day provider " + + "webhook-retry window. A shorter retention can prune a payment's " + + "idempotency row while the provider is still retrying its webhook, " + + "which would re-process the session and risk a duplicate refund. " + + "Set it to at least 3 (the default is 90).", + ); }); test("the live retention constant satisfies its own floor", () => { @@ -142,6 +153,7 @@ describe("limits", () => { "FORM_STASH_MAX_ENTRIES", "FORM_STASH_TTL_MS", "LOGIN_LOCKOUT_MS", + "MAINTENANCE_PRUNE_BATCH", "MAX_ADDRESS_LOOKUPS", "MAX_APIKEY_ATTEMPTS", "MAX_ATTACHMENT_SIZE", @@ -192,6 +204,9 @@ describe("limits", () => { ); expect(currentByKey.get("MAX_LOGIN_ATTEMPTS")).toBe(MAX_LOGIN_ATTEMPTS); expect(currentByKey.get("LOGIN_LOCKOUT_MS")).toBe(LOGIN_LOCKOUT_MS); + expect(currentByKey.get("MAINTENANCE_PRUNE_BATCH")).toBe( + MAINTENANCE_PRUNE_BATCH, + ); expect(currentByKey.get("PRUNE_PAYMENTS_RETENTION_DAYS")).toBe( PRUNE_PAYMENTS_RETENTION_DAYS, ); diff --git a/test/shared/limits/registry.test.ts b/test/shared/limits/registry.test.ts new file mode 100644 index 000000000..576c2bed1 --- /dev/null +++ b/test/shared/limits/registry.test.ts @@ -0,0 +1,159 @@ +import { expect } from "@std/expect"; +import { describe, it as test } from "@std/testing/bdd"; +import { + ACTIVITY_LOG_BACKFILL_INTERVAL_MS, + ADDRESS_CACHE_MS, + LIMIT_ENTRIES, + PRUNE_CONTACTS_RETENTION_MS, + PRUNE_LOGINS_RETENTION_MS, + PRUNE_PAYMENTS_RETENTION_MS, + PRUNE_SESSIONS_RETENTION_MS, + PRUNE_SUMUP_RETENTION_MS, + PRUNE_TOKENS_RETENTION_MS, + PRUNE_UNUSED_STRINGS_RETENTION_MS, + parsePositiveInt, +} from "#shared/limits.ts"; + +const metadata = LIMIT_ENTRIES.map(({ defaultValue, envKey, label, unit }) => [ + envKey, + defaultValue, + label, + unit, +]); + +describe("limit registry contract", () => { + test("keeps every public default, label, and unit exact", () => { + expect(metadata).toEqual([ + ["MAX_IMAGE_SIZE", 33_554_432, "Max image size", "bytes"], + ["MAX_ATTACHMENT_SIZE", 26_214_400, "Max attachment size", "bytes"], + ["MAX_BACKUPS", 30, "Max retained backups", "backups"], + ["MAX_TEXTAREA_LENGTH", 10_240, "Max textarea length", "chars"], + ["MAX_FORM_LINES", 1000, "Max attendee-form line items", "lines"], + ["ATTACHMENT_URL_MAX_AGE_S", 3600, "Attachment URL max age", "seconds"], + ["SESSION_MAX_AGE_S", 86_400, "Session max age", "seconds"], + ["SCANNER_CSRF_MAX_AGE_S", 86_400, "Scanner CSRF max age", "seconds"], + ["STALE_RESERVATION_MS", 300_000, "Stale reservation threshold", "ms"], + ["MAX_LOGIN_ATTEMPTS", 5, "Max login attempts", "attempts"], + ["LOGIN_LOCKOUT_MS", 900_000, "Login lockout duration", "ms"], + ["MAX_TOKEN_404S", 5, "Max token 404s before lockout", "attempts"], + ["TOKEN_WINDOW_MS", 60_000, "Token 404 window", "ms"], + ["TOKEN_LOCKOUT_MS", 300_000, "Token lockout duration", "ms"], + [ + "MAX_BOOKING_ATTEMPTS", + 10, + "Max booking attempts before lockout", + "attempts", + ], + ["BOOKING_LOCKOUT_MS", 600_000, "Booking lockout duration", "ms"], + [ + "MAX_ADDRESS_LOOKUPS", + 30, + "Max address lookups before lockout", + "attempts", + ], + [ + "ADDRESS_LOOKUP_LOCKOUT_MS", + 600_000, + "Address lookup lockout duration", + "ms", + ], + [ + "MAX_APIKEY_ATTEMPTS", + 20, + "Max failed API-key attempts before lockout", + "attempts", + ], + ["APIKEY_LOCKOUT_MS", 900_000, "API-key lockout duration", "ms"], + [ + "PRUNE_PAYMENTS_RETENTION_DAYS", + 90, + "Prune: payments retention", + "days", + ], + [ + "PRUNE_SESSIONS_RETENTION_DAYS", + 90, + "Prune: sessions retention", + "days", + ], + [ + "PRUNE_LOGINS_RETENTION_DAYS", + 90, + "Prune: login-attempts retention", + "days", + ], + [ + "PRUNE_TOKENS_RETENTION_DAYS", + 7, + "Prune: token-attempts retention", + "days", + ], + [ + "PRUNE_SUMUP_RETENTION_HOURS", + 24, + "Prune: SumUp checkout staging retention", + "hours", + ], + [ + "PRUNE_UNUSED_STRINGS_RETENTION_DAYS", + 7, + "Prune: unused encrypted strings retention", + "days", + ], + [ + "PRUNE_CONTACTS_RETENTION_DAYS", + 1825, + "Prune: contact-preferences retention", + "days", + ], + ["ADDRESS_CACHE_DAYS", 90, "Address lookup cache retention", "days"], + ["PRUNE_INTERVAL_HOURS", 24, "Prune: run interval", "hours"], + ["MAINTENANCE_PRUNE_BATCH", 500, "Maintenance prune batch size", "rows"], + [ + "ACTIVITY_LOG_BACKFILL_BATCH", + 200, + "Activity-log backfill batch size", + "rows", + ], + ["MAX_EMAIL_TEMPLATES", 1000, "Max saved email templates", "templates"], + ["SUPPORT_FORM_NAG_DAYS", 7, "Support form repeat-submit notice", "days"], + ["FORM_STASH_TTL_MS", 15_000, "Form re-fill stash TTL", "ms"], + ["FORM_STASH_MAX_BYTES", 32_768, "Form re-fill stash max size", "bytes"], + [ + "FORM_STASH_MAX_ENTRIES", + 100, + "Form re-fill stash max entries", + "entries", + ], + ]); + }); + + test("keeps derived maintenance windows exact", () => { + expect({ + activityLogBackfill: ACTIVITY_LOG_BACKFILL_INTERVAL_MS, + addressCache: ADDRESS_CACHE_MS, + contacts: PRUNE_CONTACTS_RETENTION_MS, + logins: PRUNE_LOGINS_RETENTION_MS, + payments: PRUNE_PAYMENTS_RETENTION_MS, + sessions: PRUNE_SESSIONS_RETENTION_MS, + strings: PRUNE_UNUSED_STRINGS_RETENTION_MS, + sumup: PRUNE_SUMUP_RETENTION_MS, + tokens: PRUNE_TOKENS_RETENTION_MS, + }).toEqual({ + activityLogBackfill: 60_000, + addressCache: 7_776_000_000, + contacts: 157_680_000_000, + logins: 7_776_000_000, + payments: 7_776_000_000, + sessions: 7_776_000_000, + strings: 604_800_000, + sumup: 86_400_000, + tokens: 604_800_000, + }); + }); + + test("parses one but rejects hexadecimal notation", () => { + expect(parsePositiveInt("1", 99)).toBe(1); + expect(parsePositiveInt("0x10", 99)).toBe(99); + }); +}); diff --git a/test/shared/maintenance/claims.test.ts b/test/shared/maintenance/claims.test.ts new file mode 100644 index 000000000..cbe1fd2cb --- /dev/null +++ b/test/shared/maintenance/claims.test.ts @@ -0,0 +1,147 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { execute, queryOne } from "#shared/db/client.ts"; +import { + claimNextMaintenanceTask, + finishMaintenanceTask, + syncMaintenanceTaskRows, +} from "#shared/maintenance/claims.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; + +const TASK = "claim_test"; +const OTHER_TASK = "other_test"; + +const taskRow = (name = TASK) => + queryOne<{ + last_finished_at: number | null; + last_started_at: number | null; + lease_token: string | null; + checkpoint: string | null; + next_run_at: number; + }>( + `SELECT checkpoint, next_run_at, lease_token, last_started_at, last_finished_at + FROM maintenance_tasks WHERE name = ?`, + [name], + ); + +const replaceExpiredClaim = async () => { + await syncMaintenanceTaskRows([TASK], []); + const first = await claimNextMaintenanceTask([TASK], 60_000); + await execute( + "UPDATE maintenance_tasks SET lease_expires_at = 0 WHERE name = ?", + [TASK], + ); + const second = await claimNextMaintenanceTask([TASK], 60_000); + return { first, second }; +}; + +describeWithEnv("maintenance task claims", { db: true }, () => { + test("does not claim when no task is allowed", async () => { + expect(await claimNextMaintenanceTask([], 60_000)).toBeNull(); + }); + + test("creates a missing task once and gives one concurrent caller the lease", async () => { + await syncMaintenanceTaskRows([TASK], []); + + const claims = await Promise.all([ + claimNextMaintenanceTask([TASK], 60_000), + claimNextMaintenanceTask([TASK], 60_000), + ]); + + expect(claims.filter(Boolean).length).toBe(1); + expect(claims.filter((claim) => claim === null).length).toBe(1); + }); + + test("leaves the due time unchanged while work is leased", async () => { + await syncMaintenanceTaskRows([TASK], []); + const before = await taskRow(); + const claim = await claimNextMaintenanceTask([TASK], 60_000); + const after = await taskRow(); + + expect(claim?.name).toBe(TASK); + expect(after?.next_run_at).toBe(before?.next_run_at); + expect(after?.last_started_at).not.toBeNull(); + }); + + test("reclaims crashed work immediately after its lease expires", async () => { + const { first, second } = await replaceExpiredClaim(); + + expect(second?.name).toBe(TASK); + expect(second?.leaseToken).not.toBe(first?.leaseToken); + }); + + test("prevents a stale worker from finishing its successor's claim", async () => { + const { first, second } = await replaceExpiredClaim(); + + await expect( + finishMaintenanceTask(first!, { checkpoint: null, intervalMs: 60_000 }), + ).rejects.toThrow("Lost maintenance lease for claim_test"); + expect((await taskRow())?.lease_token).toBe(second?.leaseToken); + }); + + test("success schedules from database completion time without catch-up", async () => { + await syncMaintenanceTaskRows([TASK], []); + await execute( + "UPDATE maintenance_tasks SET next_run_at = 1 WHERE name = ?", + [TASK], + ); + const claim = await claimNextMaintenanceTask([TASK], 60_000); + await finishMaintenanceTask(claim!, { + checkpoint: "next-page", + intervalMs: 60_000, + }); + + const row = await taskRow(); + expect(row?.next_run_at).toBe((row?.last_finished_at ?? 0) + 60_000); + expect(row?.checkpoint).toBe("next-page"); + expect(row?.last_finished_at).not.toBeNull(); + expect(row?.lease_token).toBeNull(); + }); + + test("failure uses the bounded retry interval", async () => { + await syncMaintenanceTaskRows([TASK], []); + const claim = await claimNextMaintenanceTask([TASK], 60_000); + await finishMaintenanceTask(claim!, { + checkpoint: null, + intervalMs: 120_000, + }); + + const row = await taskRow(); + expect(row?.next_run_at).toBe((row?.last_finished_at ?? 0) + 120_000); + expect(row?.last_finished_at).not.toBeNull(); + }); + + test("claims separate tasks by oldest due time", async () => { + await syncMaintenanceTaskRows([TASK, OTHER_TASK], []); + await execute( + `UPDATE maintenance_tasks + SET next_run_at = CASE name WHEN ? THEN 2 ELSE 1 END`, + [TASK], + ); + + const claim = await claimNextMaintenanceTask([TASK, OTHER_TASK], 60_000); + + expect(claim?.name).toBe(OTHER_TASK); + }); + + test("removes rows for declarations that are no longer enabled", async () => { + await syncMaintenanceTaskRows([TASK, OTHER_TASK], []); + await syncMaintenanceTaskRows([TASK], [OTHER_TASK]); + expect(await taskRow(OTHER_TASK)).toBeNull(); + }); + + test("keeps a disabled row until its active lease finishes", async () => { + await syncMaintenanceTaskRows([TASK], []); + const claim = await claimNextMaintenanceTask([TASK], 60_000); + + await syncMaintenanceTaskRows([], [TASK]); + + expect((await taskRow())?.lease_token).toBe(claim?.leaseToken); + await finishMaintenanceTask(claim!, { + checkpoint: claim!.checkpoint, + intervalMs: 60_000, + }); + await syncMaintenanceTaskRows([], [TASK]); + expect(await taskRow()).toBeNull(); + }); +}); diff --git a/test/shared/maintenance/definition.test.ts b/test/shared/maintenance/definition.test.ts new file mode 100644 index 000000000..26e2633c9 --- /dev/null +++ b/test/shared/maintenance/definition.test.ts @@ -0,0 +1,108 @@ +import { expect } from "@std/expect"; +import { describe, it as test } from "@std/testing/bdd"; +import { + defineMaintenanceTasks, + MAINTENANCE_RELEASE_HEADROOM_MS, + MAINTENANCE_REQUEST_CALL_LIMIT, + MAINTENANCE_TASK_CALL_LIMIT, + type MaintenanceTaskDeclaration, + maintenanceTaskByName, +} from "#shared/maintenance/definition.ts"; + +const task = ( + overrides: Partial = {}, +): MaintenanceTaskDeclaration => ({ + deadlineMs: 10_000, + enabled: () => true, + failureRetryIntervalMs: 60_000, + intervalMs: 60_000, + maxDatabaseCalls: 1, + maxExternalCalls: 0, + name: "test_task", + run: () => Promise.resolve(), + settingsKeys: [], + wakePolicy: "organic_safe", + ...overrides, +}); + +describe("maintenance task declarations", () => { + test("reserves seven calls and one second for scheduler bookkeeping", () => { + expect(MAINTENANCE_REQUEST_CALL_LIMIT - MAINTENANCE_TASK_CALL_LIMIT).toBe( + 7, + ); + expect(MAINTENANCE_RELEASE_HEADROOM_MS).toBe(1_000); + }); + + test("rejects a name that cannot be stored as a task key", () => { + expect(() => defineMaintenanceTasks([task({ name: "Not valid" })])).toThrow( + "Invalid maintenance task name", + ); + }); + + test("fails loudly when a claimed task was not declared", () => { + expect(() => maintenanceTaskByName([task()], "missing")).toThrow( + "Claimed undeclared maintenance task: missing", + ); + }); + + test("keeps one validated static declaration", () => { + const declaration = task(); + expect(defineMaintenanceTasks([declaration])[0]).toBe(declaration); + }); + + test("rejects duplicate stable names", () => { + expect(() => defineMaintenanceTasks([task(), task()])).toThrow( + "Duplicate maintenance task: test_task", + ); + }); + + test("rejects intervals shorter than one minute", () => { + expect(() => + defineMaintenanceTasks([task({ intervalMs: 59_999 })]), + ).toThrow("test_task interval must be at least 60000ms"); + }); + + test("rejects retries longer than the normal interval", () => { + expect(() => + defineMaintenanceTasks([ + task({ failureRetryIntervalMs: 60_001, intervalMs: 60_000 }), + ]), + ).toThrow("test_task failure retry must be between 60000ms and 60000ms"); + }); + + test("rejects a task that cannot fit the whole request budget", () => { + expect(() => + defineMaintenanceTasks([ + task({ + maxDatabaseCalls: MAINTENANCE_TASK_CALL_LIMIT, + maxExternalCalls: 1, + }), + ]), + ).toThrow( + `test_task declares ${MAINTENANCE_TASK_CALL_LIMIT + 1} calls; maximum is ${MAINTENANCE_TASK_CALL_LIMIT}`, + ); + }); + + test("accepts the exact whole-request task allowance", () => { + expect(() => + defineMaintenanceTasks([ + task({ maxDatabaseCalls: MAINTENANCE_TASK_CALL_LIMIT }), + ]), + ).not.toThrow(); + }); + + test("rejects invalid call counts and deadlines", () => { + expect(() => + defineMaintenanceTasks([task({ maxDatabaseCalls: -1 })]), + ).toThrow("test_task database calls must be a non-negative integer"); + expect(() => + defineMaintenanceTasks([task({ maxExternalCalls: -1 })]), + ).toThrow("test_task external calls must be a non-negative integer"); + expect(() => defineMaintenanceTasks([task({ deadlineMs: 0 })])).toThrow( + "test_task deadline must be greater than 0ms", + ); + expect(() => + defineMaintenanceTasks([task({ deadlineMs: 1 })]), + ).not.toThrow(); + }); +}); diff --git a/test/shared/maintenance/registry.test.ts b/test/shared/maintenance/registry.test.ts new file mode 100644 index 000000000..a3ec3339c --- /dev/null +++ b/test/shared/maintenance/registry.test.ts @@ -0,0 +1,115 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { hasLegacyActivityLog } from "#shared/db/activity-log-backfill.ts"; +import { settings } from "#shared/db/settings.ts"; +import { + MAINTENANCE_PRUNE_BATCH, + PRUNE_UNUSED_STRINGS_RETENTION_MS, +} from "#shared/limits.ts"; +import type { MaintenanceTaskDeclaration } from "#shared/maintenance/definition.ts"; +import { MAINTENANCE_TASKS } from "#shared/maintenance/registry.ts"; +import { nowMs } from "#shared/now.ts"; +import { insertLegacyActivity } from "#test-utils/activity-log.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; +import { + insertLoginAttempt, + insertStrings, + loginAttemptExists, +} from "../db/prune/helpers.ts"; + +const taskNamed = (name: string): MaintenanceTaskDeclaration => { + const task = MAINTENANCE_TASKS.find((candidate) => candidate.name === name); + if (!task) throw new Error(`Maintenance task not found: ${name}`); + return task; +}; + +const runTask = ( + task: MaintenanceTaskDeclaration, + requestFollowUp: () => void = () => {}, +): void | Promise => + task.run({ + budget: { + remaining: () => ({ database: 2, external: 0, total: 2 }), + }, + checkpoint: null, + deadline: Date.now() + 10_000, + requestFollowUp, + setCheckpoint: () => {}, + }); + +describeWithEnv("maintenance registry", { db: true }, () => { + test("declares only bounded local pruning and activity backfill", () => { + expect( + MAINTENANCE_TASKS.map( + ({ enabled: _enabled, run: _run, ...task }) => task, + ), + ).toEqual([ + { + deadlineMs: 15_000, + failureRetryIntervalMs: 300_000, + intervalMs: 86_400_000, + maxDatabaseCalls: 2, + maxExternalCalls: 0, + name: "database_pruning", + settingsKeys: ["auto_purge_orphans", "orphan_purge_retention"], + wakePolicy: "organic_safe", + }, + { + deadlineMs: 10_000, + failureRetryIntervalMs: 60_000, + intervalMs: 60_000, + maxDatabaseCalls: 2, + maxExternalCalls: 0, + name: "activity_log_backfill", + settingsKeys: ["public_key"], + wakePolicy: "organic_safe", + }, + ]); + }); + + test("the pruning task runs one bounded database batch", async () => { + expect(await taskNamed("database_pruning").enabled()).toBe(true); + const ipHash = await insertLoginAttempt("192.0.2.10", 1, 0); + expect(await loginAttemptExists(ipHash)).toBe(true); + + expect(await runTask(taskNamed("database_pruning"))).toBeUndefined(); + + expect(await loginAttemptExists(ipHash)).toBe(false); + }); + + test("the pruning task requests a follow-up for a full batch", async () => { + const old = new Date( + nowMs() - PRUNE_UNUSED_STRINGS_RETENTION_MS - 60_000, + ).toISOString(); + await insertStrings("registry-backlog", old, MAINTENANCE_PRUNE_BATCH + 1); + + let followUps = 0; + + await runTask(taskNamed("database_pruning"), () => { + followUps += 1; + }); + + expect(followUps).toBe(1); + }); + + test("the activity task enables and drains legacy rows", async () => { + await insertLegacyActivity("registry legacy"); + const task = taskNamed("activity_log_backfill"); + + expect(await task.enabled()).toBe(true); + await runTask(task); + + expect(await hasLegacyActivityLog()).toBe(false); + }); + + test("the activity task stays disabled when no legacy rows remain", async () => { + expect(await taskNamed("activity_log_backfill").enabled()).toBe(false); + }); + + test("the activity task stays disabled without an owner public key", async () => { + await insertLegacyActivity("legacy without owner key"); + settings.setForTest({ public_key: "" }); + + expect(await taskNamed("activity_log_backfill").enabled()).toBe(false); + }); +}); diff --git a/test/shared/maintenance/report.test.ts b/test/shared/maintenance/report.test.ts new file mode 100644 index 000000000..1298eec0d --- /dev/null +++ b/test/shared/maintenance/report.test.ts @@ -0,0 +1,15 @@ +import { expect } from "@std/expect"; +import { describe, it as test } from "@std/testing/bdd"; +import { reportMaintenanceFailure } from "#shared/maintenance/report.ts"; +import { setupErrorSpy } from "#test-utils/error-spy.ts"; + +describe("maintenance failure reporting", () => { + const errors = setupErrorSpy(); + + test("logs the maintenance failure detail", () => { + reportMaintenanceFailure("scheduled maintenance failed", new Error("boom")); + + expect(errors.calls.length).toBe(1); + expect(errors.contains("scheduled maintenance failed")).toBe(true); + }); +}); diff --git a/test/shared/maintenance/runner.test.ts b/test/shared/maintenance/runner.test.ts new file mode 100644 index 000000000..5324e1fd3 --- /dev/null +++ b/test/shared/maintenance/runner.test.ts @@ -0,0 +1,419 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { execute, queryOne } from "#shared/db/client.ts"; +import { + defineMaintenanceTasks, + MAINTENANCE_MIN_INTERVAL_MS, + type MaintenanceTaskDeclaration, +} from "#shared/maintenance/definition.ts"; +import { maintenance } from "#shared/maintenance/runner.ts"; +import { + countSubrequest, + getSubrequestUsage, + runWithSubrequestBudget, +} from "#shared/subrequest-budget.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; + +const declaration = ( + name: string, + run: MaintenanceTaskDeclaration["run"], + overrides: Partial = {}, +): MaintenanceTaskDeclaration => ({ + deadlineMs: 10_000, + enabled: () => true, + failureRetryIntervalMs: 60_000, + intervalMs: 60_000, + maxDatabaseCalls: 0, + maxExternalCalls: 0, + name, + run, + settingsKeys: [], + wakePolicy: "organic_safe", + ...overrides, +}); + +const forceDue = (names: string[]): Promise => + execute( + `UPDATE maintenance_tasks SET next_run_at = 0 + WHERE name IN (${names.map(() => "?").join(", ")})`, + names, + ); + +const nextRunAt = async (name: string): Promise => { + const row = await queryOne<{ next_run_at: number }>( + "SELECT next_run_at FROM maintenance_tasks WHERE name = ?", + [name], + ); + if (!row) throw new Error(`Maintenance task not found: ${name}`); + return row.next_run_at; +}; + +const taskTiming = (name: string) => + queryOne<{ last_finished_at: number; next_run_at: number }>( + `SELECT last_finished_at, next_run_at + FROM maintenance_tasks WHERE name = ?`, + [name], + ); + +describeWithEnv("maintenance runner", { db: true }, () => { + test("isolates a failed task and reports it after another task runs", async () => { + const ran: string[] = []; + const tasks = defineMaintenanceTasks([ + declaration( + "fails", + () => { + ran.push("fails"); + throw new Error("broken task"); + }, + { intervalMs: 120_000 }, + ), + declaration( + "works", + () => { + ran.push("works"); + }, + { intervalMs: 120_000 }, + ), + ]); + + await expect( + runWithSubrequestBudget(() => maintenance.run(tasks)), + ).rejects.toThrow("Maintenance failed: fails"); + expect(ran.sort()).toEqual(["fails", "works"]); + expect( + (await nextRunAt("works")) - (await nextRunAt("fails")), + ).toBeGreaterThan(50_000); + }); + + test("reports every failed task by name", async () => { + const fail = (name: string): MaintenanceTaskDeclaration => + declaration(name, () => { + throw new Error(name); + }); + + await expect( + runWithSubrequestBudget(() => + maintenance.run( + defineMaintenanceTasks([fail("first"), fail("second")]), + ), + ), + ).rejects.toThrow("Maintenance failed: first, second"); + }); + + test("reports a lost release without hiding the completed task", async () => { + const tasks = defineMaintenanceTasks([ + declaration( + "release_fails", + async () => { + await execute(`CREATE TRIGGER fail_maintenance_release + BEFORE UPDATE ON maintenance_tasks + WHEN OLD.name = 'release_fails' AND NEW.lease_token IS NULL + BEGIN SELECT RAISE(ABORT, 'release failed'); END`); + }, + { maxDatabaseCalls: 1 }, + ), + ]); + + await expect( + runWithSubrequestBudget(() => maintenance.run(tasks)), + ).rejects.toThrow("Maintenance failed: release_fails"); + }); + + test("an immediate second run does not catch up missed intervals", async () => { + const calls: number[] = []; + const tasks = defineMaintenanceTasks([ + declaration("cadence", () => { + calls.push(Date.now()); + }), + ]); + + await runWithSubrequestBudget(() => maintenance.run(tasks)); + await runWithSubrequestBudget(() => maintenance.run(tasks)); + + expect(calls.length).toBe(1); + }); + + test("a successful task can request an early follow-up", async () => { + const tasks = defineMaintenanceTasks([ + declaration("follow_up", ({ requestFollowUp }) => requestFollowUp(), { + intervalMs: 300_000, + }), + ]); + await runWithSubrequestBudget(() => maintenance.run(tasks)); + + const timing = await taskTiming("follow_up"); + expect(timing?.next_run_at).toBe( + (timing?.last_finished_at ?? 0) + MAINTENANCE_MIN_INTERVAL_MS, + ); + }); + + test("passes and saves a successful task checkpoint", async () => { + await execute( + `INSERT INTO maintenance_tasks (name, checkpoint, next_run_at) + VALUES ('checkpoint_task', 'page-1', 0)`, + ); + const tasks = defineMaintenanceTasks([ + declaration("checkpoint_task", ({ checkpoint, setCheckpoint }) => { + expect(checkpoint).toBe("page-1"); + setCheckpoint("page-2"); + }), + ]); + + await runWithSubrequestBudget(() => maintenance.run(tasks)); + + expect( + await queryOne<{ checkpoint: string }>( + "SELECT checkpoint FROM maintenance_tasks WHERE name = 'checkpoint_task'", + ), + ).toEqual({ checkpoint: "page-2" }); + }); + + test("scheduled-only work is excluded from organic runs", async () => { + const calls: string[] = []; + const tasks = defineMaintenanceTasks([ + declaration( + "scheduled_only", + () => { + calls.push("ran"); + }, + { wakePolicy: "scheduled_only" }, + ), + ]); + + await runWithSubrequestBudget(() => maintenance.runOrganic(tasks)); + + expect(calls).toEqual([]); + }); + + test("the default scheduled wake includes scheduled-only work", async () => { + const calls: string[] = []; + const tasks = defineMaintenanceTasks([ + declaration( + "scheduled_default", + () => { + calls.push("ran"); + }, + { wakePolicy: "scheduled_only" }, + ), + ]); + + await runWithSubrequestBudget(() => maintenance.run(tasks)); + + expect(calls).toEqual(["ran"]); + }); + + test("a task that cannot fit keeps its due time", async () => { + const tasks = defineMaintenanceTasks([ + declaration("too_large", () => {}, { maxDatabaseCalls: 1 }), + ]); + await runWithSubrequestBudget(() => maintenance.run(tasks)); + await forceDue(["too_large"]); + const before = await queryOne<{ next_run_at: number }>( + "SELECT next_run_at FROM maintenance_tasks WHERE name = 'too_large'", + ); + + await runWithSubrequestBudget(() => + maintenance.run(tasks, { combinedAllowance: 4 }), + ); + + const after = await queryOne<{ next_run_at: number }>( + "SELECT next_run_at FROM maintenance_tasks WHERE name = 'too_large'", + ); + expect(after?.next_run_at).toBe(before?.next_run_at); + }); + + test("passes a cooperative deadline and exact task allowances", async () => { + const seen: { + database: number; + deadline: number; + external: number; + total: number; + }[] = []; + const tasks = defineMaintenanceTasks([ + declaration( + "budgeted", + ({ budget, deadline }) => { + const remaining = budget.remaining(); + seen.push({ + database: remaining.database, + deadline, + external: remaining.external, + total: remaining.total, + }); + }, + { deadlineMs: 5_000, maxDatabaseCalls: 3, maxExternalCalls: 2 }, + ), + ]); + const before = Date.now(); + + await runWithSubrequestBudget(() => maintenance.run(tasks)); + + expect(seen).toEqual([ + { + database: 3, + deadline: seen[0]!.deadline, + external: 2, + total: 5, + }, + ]); + expect(seen[0]!.deadline).toBeGreaterThanOrEqual(before + 4_000); + expect(seen[0]!.deadline).toBeLessThanOrEqual(before + 5_100); + }); + + test("keeps the lease past the request deadline", async () => { + let leaseExpiresAt = 0; + const tasks = defineMaintenanceTasks([ + declaration( + "lease_lifetime", + async () => { + const row = await queryOne<{ lease_expires_at: number }>( + "SELECT lease_expires_at FROM maintenance_tasks WHERE name = 'lease_lifetime'", + ); + if (!row) throw new Error("Maintenance lease not found"); + leaseExpiresAt = row.lease_expires_at; + }, + { maxDatabaseCalls: 1 }, + ), + ]); + const requestDeadline = Date.now() + 5_000; + + await runWithSubrequestBudget(() => + maintenance.run(tasks, { requestDeadline }), + ); + + expect(leaseExpiresAt - requestDeadline).toBeGreaterThan(500); + expect(leaseExpiresAt - requestDeadline).toBeLessThan(1_500); + }); + + test("the default request deadline preserves release headroom", async () => { + let deadline = 0; + const tasks = defineMaintenanceTasks([ + declaration( + "release_headroom", + (context) => { + deadline = context.deadline; + }, + { deadlineMs: 25_000 }, + ), + ]); + const before = Date.now(); + + await runWithSubrequestBudget(() => maintenance.run(tasks)); + + expect(deadline).toBeGreaterThan(before + 23_000); + expect(deadline).toBeLessThan(before + 24_500); + }); + + test("an expired request deadline starts no task", async () => { + const calls: string[] = []; + const tasks = defineMaintenanceTasks([ + declaration("expired_request", () => { + calls.push("ran"); + }), + ]); + + await runWithSubrequestBudget(() => + maintenance.run(tasks, { requestDeadline: 0 }), + ); + + expect(calls).toEqual([]); + }); + + test("zero combined allowance skips scheduler bookkeeping", async () => { + const calls: string[] = []; + const tasks = defineMaintenanceTasks([ + declaration("no_allowance", () => { + calls.push("ran"); + }), + ]); + + await runWithSubrequestBudget(async () => { + await maintenance.run(tasks, { combinedAllowance: 0 }); + expect(getSubrequestUsage()).toEqual({ + database: 0, + external: 0, + total: 0, + }); + }); + expect(calls).toEqual([]); + }); + + test("subtracts request calls already used from the maintenance envelope", async () => { + const calls: string[] = []; + const tasks = defineMaintenanceTasks([ + declaration( + "prior_usage", + () => { + calls.push("ran"); + }, + { maxDatabaseCalls: 30 }, + ), + ]); + + await runWithSubrequestBudget(async () => { + countSubrequest("database", "earlier request work"); + countSubrequest("database", "more earlier request work"); + await maintenance.run(tasks); + }); + + expect(calls).toEqual(["ran"]); + }); + + test("zero external allowance leaves external tasks due", async () => { + const calls: string[] = []; + const tasks = defineMaintenanceTasks([ + declaration( + "external", + () => { + calls.push("ran"); + }, + { maxExternalCalls: 1 }, + ), + ]); + + await runWithSubrequestBudget(() => maintenance.runOrganic(tasks)); + + expect(calls).toEqual([]); + }); + + test("organic and authenticated wakes race through one durable claim", async () => { + const calls: string[] = []; + const tasks = defineMaintenanceTasks([ + declaration("wake_race", () => { + calls.push("ran"); + }), + ]); + + await Promise.all([ + runWithSubrequestBudget(() => + maintenance.run(tasks, { wakePolicy: "organic_safe" }), + ), + runWithSubrequestBudget(() => maintenance.run(tasks)), + ]); + + expect(calls).toEqual(["ran"]); + }); + + for (const [minutes, expectedBatches] of [ + [1, 15], + [5, 3], + [15, 1], + ] as const) { + test(`${minutes}-minute pings drain ${expectedBatches} bounded batch(es) in fifteen minutes`, async () => { + const name = `cadence_${minutes}`; + let batches = 0; + const tasks = defineMaintenanceTasks([ + declaration(name, () => { + batches += 1; + }), + ]); + + for (let minute = minutes; minute <= 15; minute += minutes) { + await forceDue([name]); + await runWithSubrequestBudget(() => maintenance.run(tasks)); + } + + expect(batches).toBe(expectedBatches); + }); + } +}); diff --git a/test/shared/path.test.ts b/test/shared/path.test.ts new file mode 100644 index 000000000..615c342df --- /dev/null +++ b/test/shared/path.test.ts @@ -0,0 +1,12 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { normalizePath } from "#shared/path.ts"; + +test("normalizes non-root paths without changing root", () => { + expect([ + normalizePath("/"), + normalizePath("/scheduled"), + normalizePath("/scheduled/"), + normalizePath("/scheduled///"), + ]).toEqual(["/", "/scheduled", "/scheduled", "/scheduled"]); +}); diff --git a/test/shared/renewal-helpers.test.ts b/test/shared/renewal-helpers.test.ts index 0daead1b0..f3fb94aa5 100644 --- a/test/shared/renewal-helpers.test.ts +++ b/test/shared/renewal-helpers.test.ts @@ -13,6 +13,10 @@ describe("isProvisioned", () => { const site = testBuiltSite({ renewalTokenIndex: null }); expect(isProvisioned(site)).toBe(false); }); + + test("returns false when a legacy renewal index is empty", () => { + expect(isProvisioned(testBuiltSite({ renewalTokenIndex: "" }))).toBe(false); + }); }); describe("formatDeadlineLabel", () => { diff --git a/test/shared/scheduled-access.test.ts b/test/shared/scheduled-access.test.ts new file mode 100644 index 000000000..e6ebb6aa1 --- /dev/null +++ b/test/shared/scheduled-access.test.ts @@ -0,0 +1,103 @@ +import { expect } from "@std/expect"; +import { describe, it as test } from "@std/testing/bdd"; +import { + checkScheduledAccess, + scheduledResponse, +} from "#shared/scheduled-access.ts"; +import { + scheduledAuthorization, + TEST_SCHEDULED_KEY, +} from "#test-utils/scheduled.ts"; + +const request = ( + method: string, + authorization?: string, + path = "/scheduled", +): Request => + new Request(`https://example.test${path}`, { + ...(authorization ? { headers: { authorization } } : {}), + method, + }); + +describe("scheduled access", () => { + test("leaves every other path to the normal app", () => { + expect( + checkScheduledAccess(request("GET", undefined, "/health"), undefined), + ).toEqual({ kind: "not_scheduled" }); + }); + + test("hides every non-POST method", () => { + for (const method of ["GET", "HEAD", "PUT", "PATCH", "DELETE", "OPTIONS"]) { + expect( + checkScheduledAccess( + request(method, scheduledAuthorization().authorization), + TEST_SCHEDULED_KEY, + ), + ).toEqual({ kind: "rejected", status: 404 }); + } + }); + + test("hides POST when the active key is unset", () => { + expect( + checkScheduledAccess( + request("POST", scheduledAuthorization().authorization), + undefined, + ), + ).toEqual({ kind: "rejected", status: 404 }); + }); + + test("rejects missing, malformed, and wrong bearer values", () => { + for (const authorization of [ + undefined, + `Basic ${TEST_SCHEDULED_KEY}`, + "Bearer", + "Bearer mutated", + "Bearer wrong", + ]) { + expect( + checkScheduledAccess( + request("POST", authorization), + TEST_SCHEDULED_KEY, + ), + ).toEqual({ kind: "rejected", status: 401 }); + } + }); + + test("accepts only the configured key", () => { + expect( + checkScheduledAccess( + request("POST", scheduledAuthorization().authorization), + TEST_SCHEDULED_KEY, + ), + ).toEqual({ kind: "authorized" }); + }); + + test("accepts the configured key with a trailing path slash", () => { + expect( + checkScheduledAccess( + request("POST", scheduledAuthorization().authorization, "/scheduled/"), + TEST_SCHEDULED_KEY, + ), + ).toEqual({ kind: "authorized" }); + }); + + test("returns an empty no-store response for every outcome", async () => { + for (const status of [204, 401, 404, 503] as const) { + const response = scheduledResponse(status); + expect(response.status).toBe(status); + expect(response.headers.get("cache-control")).toBe("no-store"); + expect(await response.text()).toBe(""); + } + }); + + test("adds a bearer challenge only to an unauthorized response", () => { + expect(scheduledResponse(401).headers.get("www-authenticate")).toBe( + "Bearer", + ); + for (const status of [204, 404, 503] as const) { + expect( + scheduledResponse(status).headers.get("www-authenticate"), + ).toBeNull(); + } + }); +}); diff --git a/test/shared/scheduled-keys.test.ts b/test/shared/scheduled-keys.test.ts new file mode 100644 index 000000000..951a9e223 --- /dev/null +++ b/test/shared/scheduled-keys.test.ts @@ -0,0 +1,49 @@ +import { expect } from "@std/expect"; +import { describe, it as test } from "@std/testing/bdd"; +import { + generateScheduledTaskKey, + isScheduledTaskKey, + SCHEDULED_KEY_BYTES, + SCHEDULED_TASK_KEY_ENV, + validateScheduledTaskKey, +} from "#shared/scheduled-keys.ts"; +import { withEnv } from "#test-utils/env.ts"; +import { TEST_SCHEDULED_KEY } from "#test-utils/scheduled.ts"; + +describe("scheduled task keys", () => { + test("generates distinct canonical 256-bit keys", () => { + const first = generateScheduledTaskKey(); + const second = generateScheduledTaskKey(); + + expect(first).not.toBe(second); + expect(isScheduledTaskKey(first)).toBe(true); + expect(isScheduledTaskKey(second)).toBe(true); + expect(SCHEDULED_KEY_BYTES).toBe(32); + }); + + test("recognizes only canonical unpadded keys", () => { + expect(isScheduledTaskKey(TEST_SCHEDULED_KEY)).toBe(true); + for (const value of [ + "", + `${TEST_SCHEDULED_KEY}=`, + "A".repeat(42), + "A".repeat(44), + `${TEST_SCHEDULED_KEY.slice(0, 42)}B`, + `${TEST_SCHEDULED_KEY.slice(0, 42)}+`, + ]) { + expect(isScheduledTaskKey(value)).toBe(false); + } + }); + + test("accepts one canonical configured key", () => { + using _env = withEnv({ SCHEDULED_TASK_KEY: TEST_SCHEDULED_KEY }); + expect(() => validateScheduledTaskKey()).not.toThrow(); + }); + + test("rejects a malformed configured key", () => { + using _env = withEnv({ SCHEDULED_TASK_KEY: "invalid" }); + expect(() => validateScheduledTaskKey()).toThrow( + `${SCHEDULED_TASK_KEY_ENV} must be canonical unpadded base64url for exactly ${SCHEDULED_KEY_BYTES} bytes`, + ); + }); +}); diff --git a/test/shared/sentry-sdk.test.ts b/test/shared/sentry-sdk.test.ts new file mode 100644 index 000000000..1500e78a0 --- /dev/null +++ b/test/shared/sentry-sdk.test.ts @@ -0,0 +1,127 @@ +import * as Sentry from "@sentry/deno"; +import { expect } from "@std/expect"; +import { afterEach, describe, it as test } from "@std/testing/bdd"; +import { spy } from "@std/testing/mock"; +import { ErrorCode } from "#shared/logger.ts"; +import "#shared/sentry-sdk.ts"; +import { captureServerError, initSentry } from "#shared/sentry.ts"; +import { + getSubrequestUsage, + runWithSubrequestBudget, + withSubrequestAllowance, +} from "#shared/subrequest-budget.ts"; +import { withEnv } from "#test-utils/env.ts"; +import { stubFetch } from "#test-utils/fetch-stub.ts"; +import { resetSentry } from "#test-utils/sentry.ts"; + +const DSN = "https://key@bugs.example.test/1"; + +const requireClient = () => { + const client = Sentry.getClient(); + if (!client) throw new Error("Sentry client was not created"); + return client; +}; + +describe("Sentry SDK transport", () => { + afterEach(resetSentry); + + test("starts the manual SDK client", async () => { + using _env = withEnv({ SENTRY_URL: DSN }); + const initSpy = spy(Sentry.DenoClient.prototype, "init"); + try { + await initSentry(); + + expect(initSpy.calls).toHaveLength(1); + } finally { + initSpy.restore(); + } + }); + + test("does not sample traces", async () => { + using _env = withEnv({ SENTRY_URL: DSN }); + await initSentry(); + + expect(requireClient().getOptions().tracesSampleRate).toBe(0); + }); + + test("counts one external subrequest per envelope", async () => { + using _env = withEnv({ + SENTRY_URL: DSN, + }); + using fetchStub = stubFetch(new Response(null, { status: 200 })); + + await runWithSubrequestBudget(async () => { + expect(await initSentry()).toBe(true); + await captureServerError({ code: ErrorCode.DB_QUERY }); + expect(getSubrequestUsage()).toEqual({ + database: 0, + external: 1, + total: 1, + }); + }); + expect(fetchStub.calls).toHaveLength(1); + }); + + test("uses the exact HTTP transport and forwards rate-limit headers", async () => { + using _env = withEnv({ SENTRY_URL: DSN }); + using fetchStub = stubFetch( + new Response(null, { + headers: { + "retry-after": "17", + "x-sentry-rate-limits": "60:error", + }, + status: 202, + }), + ); + await initSentry(); + const client = requireClient(); + const responseReady = Promise.withResolvers<{ + headers?: Record; + statusCode?: number; + }>(); + const stopListening = client.on("afterSendEvent", (_event, response) => { + responseReady.resolve(response); + }); + try { + await captureServerError({ code: ErrorCode.DB_QUERY }); + + const options = fetchStub.calls[0]!.args[1] as RequestInit; + expect(options.method).toBe("POST"); + expect(options.referrerPolicy).toBe("strict-origin"); + expect(await responseReady.promise).toEqual({ + headers: { + "retry-after": "17", + "x-sentry-rate-limits": "60:error", + }, + statusCode: 202, + }); + } finally { + stopListening(); + } + }); + + test("names a blocked Sentry transport request", async () => { + using _env = withEnv({ SENTRY_URL: DSN }); + await initSentry(); + const client = requireClient(); + const transport = client.getTransport(); + if (!transport) throw new Error("Sentry transport was not created"); + const envelope: Parameters[0] = [ + { event_id: "budget-event", sent_at: "2026-07-19T00:00:00.000Z" }, + [ + [ + { type: "event" }, + { event_id: "budget-event", message: "Budget event", timestamp: 0 }, + ], + ], + ]; + + await expect( + runWithSubrequestBudget(() => + withSubrequestAllowance({ database: 50, external: 0, total: 0 }, () => + transport.send(envelope), + ), + ), + ).rejects.toThrow("Blocked external operation: Sentry transport"); + }); +}); diff --git a/test/shared/settings/keys.test.ts b/test/shared/settings/keys.test.ts new file mode 100644 index 000000000..b9f078e52 --- /dev/null +++ b/test/shared/settings/keys.test.ts @@ -0,0 +1,29 @@ +import { expect } from "@std/expect"; +import { describe, it as test } from "@std/testing/bdd"; +import { CONFIG_KEY_NAMES, CONFIG_KEYS } from "#shared/settings/keys.ts"; +import { jsonHash } from "#test-utils/hash.ts"; + +describe("settings keys", () => { + test("maps every key name to its lowercase stored value", () => { + expect(Object.values(CONFIG_KEYS)).toEqual( + CONFIG_KEY_NAMES.map((name) => name.toLowerCase()), + ); + }); + + test("keeps the complete public key catalog exact", async () => { + expect(await jsonHash(CONFIG_KEY_NAMES)).toBe( + "b5747ab73abc3050541eb99eb6b73528089d47d69d5801d76bce5fb1e32dc0f1", + ); + }); + + test("does not register retired maintenance timestamps", () => { + expect( + Object.keys(CONFIG_KEYS).filter( + (name) => + name.startsWith("LAST_PRUNED_") || + name === "LAST_ACTIVITY_LOG_BACKFILL" || + name === "ACTIVITY_LOG_BACKFILL_DONE", + ), + ).toEqual([]); + }); +}); diff --git a/test/shared/settings/registry.test.ts b/test/shared/settings/registry.test.ts index 3d0ce092c..8bbc2a2c9 100644 --- a/test/shared/settings/registry.test.ts +++ b/test/shared/settings/registry.test.ts @@ -6,12 +6,11 @@ import { EMAIL_BODY_KEYS, ENCRYPTED_KEYS, PLAINTEXT_KEYS, - PRUNE_KEYS, STRING_ACCESSORS, STRING_SETTING_DEFINITIONS, } from "#shared/settings/registry.ts"; -type ExpectedSettingFlag = "emailBody" | "prune" | "readOnly"; +type ExpectedSettingFlag = "emailBody" | "readOnly"; type ExpectedSettingRow = readonly [ keyof typeof CONFIG_KEYS, "plaintext" | "encrypted", @@ -26,7 +25,6 @@ const lines = (text: string): string[] => .map((line) => line.trim()); const EXPECTED_CONFIG_KEY_NAMES = lines(` - ACTIVITY_LOG_BACKFILL_DONE ADDRESS_LOOKUP_API_KEY ADDRESS_LOOKUP_PROVIDER APPLE_WALLET_PASS_TYPE_ID @@ -66,17 +64,6 @@ const EXPECTED_CONFIG_KEY_NAMES = lines(` GOOGLE_WALLET_SERVICE_ACCOUNT_KEY HEADER_IMAGE_URL HOMEPAGE_TEXT - LAST_ACTIVITY_LOG_BACKFILL - LAST_PRUNED_ADDRESSES - LAST_PRUNED_CONTACTS - LAST_PRUNED_INVITES - LAST_PRUNED_LOGINS - LAST_PRUNED_ORPHANS - LAST_PRUNED_PAYMENTS - LAST_PRUNED_SESSIONS - LAST_PRUNED_STRINGS - LAST_PRUNED_SUMUP - LAST_PRUNED_TOKENS LATEST_SCRIPT_VERSION LATEST_SCRIPT_VERSION_NAME LISTING_COLUMN_ORDER @@ -145,19 +132,7 @@ const EXPECTED_SETTING_ROWS = [ ], ["LISTING_COLUMN_ORDER", "plaintext", "listingColumnOrder"], ["ATTENDEE_COLUMN_ORDER", "plaintext", "attendeeColumnOrder"], - ["LAST_PRUNED_PAYMENTS", "plaintext", "lastPrunedPayments"], - ["LAST_PRUNED_SESSIONS", "plaintext", "lastPrunedSessions"], - ["LAST_PRUNED_SUMUP", "plaintext", "lastPrunedSumup"], - ["LAST_PRUNED_STRINGS", "plaintext", "lastPrunedStrings", "prune"], - ["LAST_PRUNED_LOGINS", "plaintext", "lastPrunedLogins", "prune"], - ["LAST_PRUNED_TOKENS", "plaintext", "lastPrunedTokens", "prune"], - ["LAST_PRUNED_CONTACTS", "plaintext", "lastPrunedContacts", "prune"], - ["LAST_PRUNED_ADDRESSES", "plaintext", "lastPrunedAddresses", "prune"], - ["LAST_PRUNED_INVITES", "plaintext", "lastPrunedInvites", "prune"], - ["LAST_PRUNED_ORPHANS", "plaintext", "lastPrunedOrphans", "prune"], ["SMS_GATEWAY_BASE_URL", "plaintext", "smsGatewayBaseUrl"], - ["ACTIVITY_LOG_BACKFILL_DONE", "plaintext", "activityLogBackfillDone"], - ["LAST_ACTIVITY_LOG_BACKFILL", "plaintext", "lastActivityLogBackfill"], ["BUSINESS_EMAIL", "encrypted", "businessEmail"], ["HEADER_IMAGE_URL", "encrypted", "headerImageUrl"], ["WEBSITE_TITLE", "encrypted", "websiteTitle"], @@ -195,7 +170,7 @@ const EXPECTED_SETTING_ROWS = [ const sorted = (values: readonly string[]): string[] => [...values].sort(); -const taggedKeys = (tag: "emailBody" | "prune"): string[] => +const taggedKeys = (tag: "emailBody"): string[] => STRING_SETTING_DEFINITIONS.filter( (definition) => "tags" in definition && @@ -208,7 +183,7 @@ const accessorKeys = (): string[] => ).map((definition) => definition.key); const settingTags = (flags: readonly ExpectedSettingFlag[]) => - flags.filter((flag): flag is "emailBody" | "prune" => flag !== "readOnly"); + flags.filter((flag): flag is "emailBody" => flag !== "readOnly"); const settingRow = ([ keyName, @@ -289,7 +264,6 @@ describe("settings registry", () => { }); test("derives tagged bundles from the same setting entries", () => { - expect(PRUNE_KEYS).toEqual(taggedKeys("prune")); expect(EMAIL_BODY_KEYS).toEqual(taggedKeys("emailBody")); }); diff --git a/test/shared/site-build.test.ts b/test/shared/site-build.test.ts new file mode 100644 index 000000000..4cca9e9e5 --- /dev/null +++ b/test/shared/site-build.test.ts @@ -0,0 +1,122 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { stub } from "@std/testing/mock"; +import { + type BuildSiteResult, + builderApi, + type PreparedBuildSite, +} from "#shared/builder.ts"; +import { setEncryptionKeyForTest } from "#shared/crypto/encryption.ts"; +import { builtSites } from "#shared/db/built-sites.ts"; +import { buildAssignableSite, buildRetainedSite } from "#shared/site-build.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; +import { setupTestEncryptionKey, withEnv } from "#test-utils/env.ts"; +import { TEST_SCHEDULED_KEY } from "#test-utils/scheduled.ts"; + +const BUILD_RESULT = { + dbProvider: "bunny", + dbToken: "database-token", + dbUrl: "libsql://built-site.test", + defaultHostname: "00001.example.test", + hostingId: "123", + hostingProvider: "bunny", + ok: true, +} satisfies BuildSiteResult; + +const PREPARED_SITE = { + ...BUILD_RESULT, + scheduledTaskKey: TEST_SCHEDULED_KEY, +} satisfies PreparedBuildSite; + +describeWithEnv("site build", { db: true }, () => { + test("checks builder storage before provider provisioning", async () => { + const buildStub = stub(builderApi, "buildSite", () => + Promise.resolve({ error: "provider should not start", ok: false }), + ); + setEncryptionKeyForTest(null); + using _env = withEnv({ DB_ENCRYPTION_KEY: undefined }); + try { + await expect( + buildRetainedSite("Unchecked Site", { siteName: "Unchecked Site" }), + ).rejects.toThrow("DB_ENCRYPTION_KEY environment variable is required"); + expect(buildStub.calls).toHaveLength(0); + } finally { + setupTestEncryptionKey(); + buildStub.restore(); + } + }); + + test("retains a local bundle before reporting success", async () => { + const buildStub = stub(builderApi, "buildSite", async (input, retain) => { + expect(input).toEqual({ code: "local bundle", siteName: "Local Site" }); + await retain(PREPARED_SITE); + return BUILD_RESULT; + }); + try { + expect( + await buildRetainedSite("Local Site", { + code: "local bundle", + siteName: "Local Site", + }), + ).toMatchObject({ result: BUILD_RESULT }); + expect(await builtSites.getAll()).toMatchObject([ + { + name: "Local Site", + scheduledTaskKey: TEST_SCHEDULED_KEY, + }, + ]); + } finally { + buildStub.restore(); + } + }); + + test("retains the scheduled key before making the site assignable", async () => { + let requestedName = ""; + let retainedAssignable: boolean | undefined; + const buildStub = stub(builderApi, "buildSite", async (input, retain) => { + requestedName = input.siteName; + await retain(PREPARED_SITE); + retainedAssignable = (await builtSites.getAll())[0]!.assignable; + return BUILD_RESULT; + }); + try { + const site = await buildAssignableSite(); + + expect(requestedName).toBe("00001"); + expect(retainedAssignable).toBe(false); + expect(site).toMatchObject({ + assignable: true, + name: "00001", + scheduledTaskKey: TEST_SCHEDULED_KEY, + }); + } finally { + buildStub.restore(); + } + }); + + test("returns null when the builder fails", async () => { + const buildStub = stub(builderApi, "buildSite", () => + Promise.resolve({ error: "provider failed", ok: false }), + ); + const errorStub = stub(console, "error"); + try { + expect(await buildAssignableSite()).toBeNull(); + } finally { + errorStub.restore(); + buildStub.restore(); + } + }); + + test("rejects a builder success that was not retained", async () => { + const buildStub = stub(builderApi, "buildSite", () => + Promise.resolve(BUILD_RESULT), + ); + try { + await expect(buildAssignableSite()).rejects.toThrow( + "Built site was not retained", + ); + } finally { + buildStub.restore(); + } + }); +}); diff --git a/test/shared/site-db.test.ts b/test/shared/site-db.test.ts index 7908724f6..15c2fd6c0 100644 --- a/test/shared/site-db.test.ts +++ b/test/shared/site-db.test.ts @@ -33,6 +33,7 @@ describe("hasSiteDbCredentials", () => { false, ); expect(hasSiteDbCredentials({ dbToken: "t", dbUrl: "" })).toBe(false); + expect(hasSiteDbCredentials({ dbToken: "", dbUrl: "" })).toBe(false); }); }); @@ -67,6 +68,28 @@ describe("withSiteDb", () => { createStub.restore(); } }); + + test("closes the client after a successful read", async () => { + const client = await seededClient([]); + let closes = 0; + const close = client.close.bind(client); + const closeStub = stub(client, "close", () => { + closes++; + }); + const createStub = stub(siteDbApi, "createClient", () => client); + try { + expect( + await withSiteDb({ dbToken: "token", dbUrl: "libsql://site" }, () => + Promise.resolve("read"), + ), + ).toEqual({ ok: true, value: "read" }); + expect(closes).toBe(1); + } finally { + createStub.restore(); + closeStub.restore(); + close(); + } + }); }); describe("readSiteSetting", () => { diff --git a/test/shared/site-scheduler.test.ts b/test/shared/site-scheduler.test.ts new file mode 100644 index 000000000..44b5e1350 --- /dev/null +++ b/test/shared/site-scheduler.test.ts @@ -0,0 +1,164 @@ +import { expect } from "@std/expect"; +import { it as test } from "@std/testing/bdd"; +import { stub } from "@std/testing/mock"; +import { bunnyHostingProvider } from "#shared/bunny-cdn.ts"; +import { ensureBuiltSiteSchedulerKey } from "#shared/db/built-site-scheduler.ts"; +import { + builtSitesCrudTable, + insertBuiltSite, +} from "#shared/db/built-sites.ts"; +import { provisionSiteScheduler } from "#shared/site-scheduler.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; +import { stubFetch } from "#test-utils/fetch-stub.ts"; +import { restoreStubsAfterEach } from "#test-utils/mocks.ts"; +import { + insertScheduledTestSite, + stubBunnySchedulerSecrets, +} from "#test-utils/scheduled.ts"; + +describeWithEnv( + "built-site scheduler provisioning", + { db: true, env: { BUNNY_API_KEY: "test-key" } }, + () => { + const stubs: { restore(): void }[] = []; + restoreStubsAfterEach(stubs); + + const site = () => insertScheduledTestSite(null); + test("refuses to replace a live primary key with no encrypted parent copy", async () => { + const child = await site(); + stubs.push( + stub(bunnyHostingProvider, "getSecretNames", () => + Promise.resolve({ ok: true, value: ["SCHEDULED_TASK_KEY"] }), + ), + ); + + const result = await provisionSiteScheduler(child.id); + + expect(result).toEqual({ + error: + "The child already has a scheduled task key that this site cannot read.", + ok: false, + }); + expect( + (await builtSitesCrudTable.findById(child.id))?.scheduledTaskKey, + ).toBeNull(); + }); + + test("fails loudly when the child record does not exist", async () => { + await expect(provisionSiteScheduler(999_999)).rejects.toThrow( + "Built site not found", + ); + }); + + test("reuses the primary key when a replica read is stale", async () => { + const child = await site(); + const stale = await builtSitesCrudTable.findById(child.id); + const key = await ensureBuiltSiteSchedulerKey(child.id); + const pushed: string[] = []; + stubs.push( + stub(builtSitesCrudTable, "findById", () => Promise.resolve(stale)), + stub(bunnyHostingProvider, "getSecretNames", () => + Promise.resolve({ ok: true, value: ["SCHEDULED_TASK_KEY"] }), + ), + stub(bunnyHostingProvider, "setSecrets", (_hostingId, secrets) => { + pushed.push(secrets[0]![1]); + return Promise.resolve({ ok: true, value: undefined }); + }), + ); + using _fetch = stubFetch(new Response(null, { status: 204 })); + + expect(await provisionSiteScheduler(child.id)).toEqual({ + ok: true, + value: undefined, + }); + expect(pushed).toEqual([key]); + }); + + test("requires provider access before provisioning", async () => { + const child = await insertScheduledTestSite(null); + await builtSitesCrudTable.update(child.id, { hostingId: "" }); + + expect(await provisionSiteScheduler(child.id)).toEqual({ + error: + "This site has no hosting ID, so its scheduled task key cannot be set.", + ok: false, + }); + }); + + test("surfaces a provider secret-list failure", async () => { + const child = await site(); + stubs.push( + stub(bunnyHostingProvider, "getSecretNames", () => + Promise.resolve({ error: "cannot list", ok: false }), + ), + ); + + expect(await provisionSiteScheduler(child.id)).toEqual({ + error: "cannot list", + ok: false, + }); + }); + + test("retains one active key when provider setup fails", async () => { + const child = await site(); + stubBunnySchedulerSecrets(stubs, [], { error: "host down", ok: false }); + + expect((await provisionSiteScheduler(child.id)).ok).toBe(false); + const first = (await builtSitesCrudTable.findById(child.id)) + ?.scheduledTaskKey; + expect((await provisionSiteScheduler(child.id)).ok).toBe(false); + expect( + (await builtSitesCrudTable.findById(child.id))?.scheduledTaskKey, + ).toBe(first); + }); + + test("requires an empty 204 from the live child", async () => { + const child = await site(); + stubBunnySchedulerSecrets(stubs, [], { ok: true }); + using fetchStub = stubFetch(new Response(null, { status: 200 })); + + const result = await provisionSiteScheduler(child.id); + + expect(result).toEqual({ + error: "The child did not accept the scheduled task key.", + ok: false, + }); + const [url, init] = fetchStub.calls[0]!.args as [string, RequestInit]; + expect(url).toBe("https://child.example.test/scheduled"); + expect(init.method).toBe("POST"); + expect(init.redirect).toBe("manual"); + expect(new Headers(init.headers).get("authorization")).toMatch( + /^Bearer [A-Za-z0-9_-]{43}$/, + ); + }); + + test("reports a failed live verification request", async () => { + const child = await site(); + stubBunnySchedulerSecrets(stubs, [], { ok: true }); + using _fetch = stubFetch(new Error("network down")); + + expect(await provisionSiteScheduler(child.id)).toEqual({ + error: "The child could not verify the scheduled task key.", + ok: false, + }); + }); + + test("verifies only the child origin when its URL includes a path", async () => { + const child = await insertBuiltSite( + "Child", + "HTTPS://child.example.test/unsafe/path", + "", + "", + false, + "42", + ); + stubBunnySchedulerSecrets(stubs, [], { ok: true }); + using fetchStub = stubFetch(new Response(null, { status: 204 })); + + expect((await provisionSiteScheduler(child.id)).ok).toBe(true); + expect(String(fetchStub.calls[0]!.args[0])).toBe( + "https://child.example.test/scheduled", + ); + }); + }, +); diff --git a/test/shared/site-secrets.test.ts b/test/shared/site-secrets.test.ts index a3dc92331..61d52b733 100644 --- a/test/shared/site-secrets.test.ts +++ b/test/shared/site-secrets.test.ts @@ -3,7 +3,7 @@ import { it as test } from "@std/testing/bdd"; import { stub } from "@std/testing/mock"; import { collectHostSecrets } from "#shared/builder.ts"; import { bunnyCdnApi, type EdgeScriptSecret } from "#shared/bunny-cdn.ts"; -import type { BuiltSite } from "#shared/db/built-sites.ts"; +import type { BuiltSite } from "#shared/db/built-sites/types.ts"; import { denoDeployApi } from "#shared/deno-deploy-api.ts"; import { okResult } from "#shared/result.ts"; import { @@ -246,16 +246,20 @@ describeWithEnv( ), async () => { const view = await loadSiteSecretsStatus(buildSite()); - expect(view.ok).toBe(false); - if (!view.ok) expect(view.error).toContain("network down"); + expect(view).toEqual({ + error: "Failed to list secrets: network down", + ok: false, + }); }, ); }); test("refuses a site with no script id", async () => { const view = await loadSiteSecretsStatus(buildSite({ hostingId: "" })); - expect(view.ok).toBe(false); - if (!view.ok) expect(view.error).toContain("no hosting ID"); + expect(view).toEqual({ + error: "This site has no hosting ID, so its secrets can't be read.", + ok: false, + }); }); }, ); diff --git a/test/shared/site-update.test.ts b/test/shared/site-update.test.ts index 051851c15..e4b9204cb 100644 --- a/test/shared/site-update.test.ts +++ b/test/shared/site-update.test.ts @@ -5,7 +5,10 @@ import { type Stub, stub } from "@std/testing/mock"; import { queryOne } from "#shared/db/client.ts"; import { ALL_SETTINGS_KEYS, settings } from "#shared/db/settings.ts"; import { siteDbApi } from "#shared/site-db.ts"; -import { loadBuiltSiteUpdateState } from "#shared/site-update.ts"; +import { + deployAndReport, + loadBuiltSiteUpdateState, +} from "#shared/site-update.ts"; import { CURRENT_SCRIPT_VERSION_KEY, readRecordedScriptCommit, @@ -18,6 +21,31 @@ import { createTestBuiltSite } from "#test-utils/db-helpers/built-sites.ts"; const LATEST_TAG = "v2099-01-01-120000"; +test("deployAndReport keeps the update task name and full failure context", async () => { + const taskStub = stub( + settings, + "withCurrentTask", + async (_task: string, run: () => Promise) => ({ + ok: true as const, + value: await run(), + }), + ); + try { + const response = await deployAndReport({ + deploy: () => Promise.reject(new Error("provider down")), + logPrefix: "Updated", + onError: (message) => new Response(message, { status: 500 }), + onSuccess: (message) => new Response(message), + successPrefix: "Updated", + }); + expect(taskStub.calls[0]?.args[0]).toBe("update"); + expect(response.status).toBe(500); + expect(await response.text()).toBe("Update failed: provider down"); + } finally { + taskStub.restore(); + } +}); + /** Seed an in-memory libsql client standing in for the remote site's DB. */ const seedSiteDb = async (version: string | null): Promise => { const client = createClient({ url: ":memory:" }); @@ -82,6 +110,8 @@ describeWithEnv( const state = await loadBuiltSiteUpdateState(site); + expect(state.latestVersion).toBe(LATEST_TAG); + expect(state.latestVersionName).toBe("2099-01-01 - Big Update"); expect(state.siteVersionLabel).toContain("2026"); expect(state.updateAvailable).toBe(true); expect(state.upToDate).toBe(false); diff --git a/test/shared/storage/bunny.test.ts b/test/shared/storage/bunny.test.ts index 17a64398d..651f7c87a 100644 --- a/test/shared/storage/bunny.test.ts +++ b/test/shared/storage/bunny.test.ts @@ -1,16 +1,38 @@ import { expect } from "@std/expect"; import { it as test } from "@std/testing/bdd"; import { + deleteFile, downloadRaw, listFiles, listFilesWithMeta, runWithStorageConfig, uploadRaw, } from "#shared/storage.ts"; +import { + runWithSubrequestBudget, + withSubrequestAllowance, +} from "#shared/subrequest-budget.ts"; import { describeWithEnv } from "#test-utils/db.ts"; import { withBunnyStorageStub } from "#test-utils/mocks.ts"; import { STORAGE_TEST_ENV } from "./fixtures.ts"; +const noExternalRequests = { + database: 50, + external: 0, + total: 50, +}; + +const expectBlockedOperation = async ( + label: string, + action: () => Promise, +): Promise => { + await expect( + runWithSubrequestBudget(() => + withSubrequestAllowance(noExternalRequests, action), + ), + ).rejects.toThrow(`Blocked external operation: ${label}`); +}; + describeWithEnv("Bunny storage", STORAGE_TEST_ENV, () => { test("rejects incomplete Bunny credentials before loading the SDK", async () => { await runWithStorageConfig( @@ -151,4 +173,41 @@ describeWithEnv("Bunny storage", STORAGE_TEST_ENV, () => { }, ); }); + test("upload identifies its blocked subrequest", async () => { + await withBunnyStorageStub( + () => Response.json({ HttpCode: 201 }), + () => + expectBlockedOperation("storage upload", () => + uploadRaw(new Uint8Array([1]), "file.bin"), + ), + ); + }); + + test("download identifies its blocked subrequest", async () => { + await withBunnyStorageStub( + () => new Response(new Uint8Array([1])), + () => + expectBlockedOperation("storage download", () => + downloadRaw("file.bin"), + ), + ); + }); + + test("delete identifies its blocked subrequest", async () => { + await withBunnyStorageStub( + () => Response.json({ HttpCode: 200 }), + () => + expectBlockedOperation("storage delete", () => deleteFile("file.bin")), + ); + }); + + test("listing identifies its blocked subrequest", async () => { + await withBunnyStorageStub( + () => Response.json([]), + () => + expectBlockedOperation("storage directory listing", () => + listFiles("backup-"), + ), + ); + }); }); diff --git a/test/shared/storage/config.test.ts b/test/shared/storage/config.test.ts index 3ec824155..eb74ef965 100644 --- a/test/shared/storage/config.test.ts +++ b/test/shared/storage/config.test.ts @@ -1,6 +1,10 @@ import { expect } from "@std/expect"; import { it as test } from "@std/testing/bdd"; -import { getStorageBackend, runWithStorageConfig } from "#shared/storage.ts"; +import { + getStorageBackend, + runWithStorageConfig, + uploadRaw, +} from "#shared/storage.ts"; import { describeWithEnv } from "#test-utils/db.ts"; import { withEnv } from "#test-utils/env.ts"; import { @@ -10,6 +14,14 @@ import { import { STORAGE_TEST_ENV } from "./fixtures.ts"; describeWithEnv("storage environment config", STORAGE_TEST_ENV, () => { + test("an empty local path keeps storage disabled", async () => { + using _env = withEnv({ LOCAL_STORAGE_PATH: "" }); + expect(getStorageBackend()).toBe("none"); + await expect( + uploadRaw(new Uint8Array([1]), "empty-local-path-test/file"), + ).rejects.toThrow("Storage is not configured"); + }); + test("reads Bunny credentials from their environment keys", () => { using _env = withEnv({ STORAGE_ZONE_KEY: "env-key", diff --git a/test/shared/subrequest-budget.test.ts b/test/shared/subrequest-budget.test.ts new file mode 100644 index 000000000..44e208848 --- /dev/null +++ b/test/shared/subrequest-budget.test.ts @@ -0,0 +1,127 @@ +import { expect } from "@std/expect"; +import { describe, it as test } from "@std/testing/bdd"; +import { + BULK_REFUND_LIMIT, + BUNNY_SUBREQUEST_LIMIT, + countSubrequest, + getSubrequestRemaining, + getSubrequestUsage, + runWithSubrequestBudget, + withSubrequestAllowance, +} from "#shared/subrequest-budget.ts"; + +describe("combined subrequest budget", () => { + test("reports the full allowance outside a request scope", async () => { + expect(getSubrequestRemaining()).toEqual({ + database: BUNNY_SUBREQUEST_LIMIT, + external: BUNNY_SUBREQUEST_LIMIT, + total: BUNNY_SUBREQUEST_LIMIT, + }); + await withSubrequestAllowance( + { database: 1, external: 1, total: 2 }, + async () => { + expect(getSubrequestRemaining()).toEqual({ + database: 1, + external: 1, + total: 2, + }); + }, + ); + }); + + test("tracks database and external calls in one envelope", async () => { + await runWithSubrequestBudget(async () => { + countSubrequest("database", "claim"); + countSubrequest("external", "provider"); + expect(getSubrequestUsage()).toEqual({ + database: 1, + external: 1, + total: 2, + }); + await withSubrequestAllowance( + { database: 0, external: 0, total: 0 }, + async () => { + expect(getSubrequestRemaining()).toEqual({ + database: 0, + external: 0, + total: 0, + }); + }, + ); + }); + }); + + test("enforces each task allowance and its combined maximum", async () => { + await runWithSubrequestBudget(async () => { + await withSubrequestAllowance( + { database: 2, external: 1, total: 3 }, + async () => { + expect(getSubrequestRemaining()).toEqual({ + database: 2, + external: 1, + total: 3, + }); + countSubrequest("database", "first"); + countSubrequest("database", "second"); + countSubrequest("external", "third"); + expect(() => countSubrequest("database", "fourth")).toThrow( + "Subrequest allowance exceeded", + ); + }, + ); + }); + }); + + test("counts work already used before a nested allowance", async () => { + await runWithSubrequestBudget(async () => { + countSubrequest("database", "setup"); + await withSubrequestAllowance( + { database: 1, external: 0, total: 1 }, + async () => { + countSubrequest("database", "task"); + expect(getSubrequestUsage().total).toBe(2); + }, + ); + }); + }); + + test("enforces a per-kind limit before the combined limit", async () => { + await runWithSubrequestBudget(async () => { + await withSubrequestAllowance( + { database: 0, external: 5, total: 5 }, + async () => { + expect(() => countSubrequest("database", "blocked database")).toThrow( + "Subrequest allowance exceeded", + ); + }, + ); + }); + }); + + test("enforces the combined limit before either per-kind limit", async () => { + await runWithSubrequestBudget(async () => { + await withSubrequestAllowance( + { database: 2, external: 2, total: 1 }, + async () => { + countSubrequest("database", "first"); + expect(() => countSubrequest("external", "blocked total")).toThrow( + "Subrequest allowance exceeded", + ); + }, + ); + }); + }); + + test("reserves five provider calls for bounded bulk refunds", () => { + expect(BULK_REFUND_LIMIT).toBe(5); + }); + + test("does not count startup work outside a request scope", () => { + countSubrequest("external", "startup"); + expect(getSubrequestUsage()).toEqual({ + database: 0, + external: 0, + total: 0, + }); + }); +}); diff --git a/test/shared/update.test.ts b/test/shared/update.test.ts index 397c914a5..515fe467d 100644 --- a/test/shared/update.test.ts +++ b/test/shared/update.test.ts @@ -2,16 +2,40 @@ import { expect } from "@std/expect"; import { afterEach, describe, it as test } from "@std/testing/bdd"; import { stub } from "@std/testing/mock"; import { bunnyCdnApi } from "#shared/bunny-cdn.ts"; +import { execute, queryAll, queryOne } from "#shared/db/client.ts"; import { denoDeployApi } from "#shared/deno-deploy-api.ts"; import { + getSubrequestUsage, + runWithSubrequestBudget, + withSubrequestAllowance, +} from "#shared/subrequest-budget.ts"; +import { + CURRENT_SCRIPT_VERSION_KEY, deployLatestReleaseToDeno, deployRelease, + fetchLatestRelease, formatBuildDate, + GITHUB_RELEASES_URL, + GITHUB_REPO, isNewerVersion, + readRecordedScriptCommit, + recordScriptVersion, + setBuildCommitForTest, setBuildTimestampForTest, } from "#shared/update.ts"; +import { describeWithEnv } from "#test-utils/db.ts"; import { stubFetch } from "#test-utils/fetch-stub.ts"; -import { stubReleaseFetch } from "#test-utils/mocks.ts"; +import { MOCK_RELEASE, stubReleaseFetch } from "#test-utils/mocks.ts"; + +describe("update constants", () => { + test("points to the public tickets release page", () => { + expect(GITHUB_REPO).toBe("chobbledotcom/tickets"); + expect(GITHUB_RELEASES_URL).toBe( + "https://github.com/chobbledotcom/tickets/releases", + ); + expect(CURRENT_SCRIPT_VERSION_KEY).toBe("current_script_version"); + }); +}); describe("update", () => { afterEach(() => { @@ -55,8 +79,7 @@ describe("update", () => { describe("formatBuildDate", () => { test("formats an ISO timestamp for display", () => { const result = formatBuildDate("2026-03-28T14:30:22.000Z"); - expect(result).toContain("2026"); - expect(result).toContain("UTC"); + expect(result).toBe("Sat, 28 Mar 2026 14:30:22 UTC"); }); test("returns Development build for empty string", () => { @@ -65,6 +88,65 @@ describe("update", () => { }); }); +describe("fetchLatestRelease", () => { + test("sends the GitHub release media type", async () => { + using fetchStub = stubReleaseFetch(); + + await fetchLatestRelease(); + + const init = fetchStub.calls[0]!.args[1]; + expect(new Headers(init?.headers).get("Accept")).toBe( + "application/vnd.github.v3+json", + ); + }); + + test("returns every release field", async () => { + using _fetch = stubReleaseFetch(); + + expect(await fetchLatestRelease()).toEqual({ + assetUrl: + "https://github.com/chobbledotcom/tickets/releases/download/v2099-01-01-120000/bunny-script.ts", + name: "2099-01-01 - Big Update", + publishedAt: "2099-01-01T12:00:00Z", + tagName: "v2099-01-01-120000", + }); + }); + + test("preserves an empty matching asset URL", async () => { + using _fetch = stubFetch( + new Response( + JSON.stringify({ + ...MOCK_RELEASE, + assets: [{ browser_download_url: "", name: "bunny-script.ts" }], + }), + ), + ); + + expect((await fetchLatestRelease()).assetUrl).toBe(""); + }); + + test("throws the GitHub status when the release lookup fails", async () => { + using _fetch = stubFetch( + new Response(JSON.stringify(MOCK_RELEASE), { status: 503 }), + ); + + await expect(fetchLatestRelease()).rejects.toThrow( + "GitHub API returned 503", + ); + }); + + test("names a blocked GitHub release lookup", async () => { + await expect( + runWithSubrequestBudget(() => + withSubrequestAllowance( + { database: 50, external: 0, total: 0 }, + fetchLatestRelease, + ), + ), + ).rejects.toThrow("Blocked external operation: GitHub release lookup"); + }); +}); + describe("deployRelease", () => { test("downloads an asset URL and deploys to a Bunny script", async () => { using _fetch = stubFetch(new Response("console.log('asset')")); @@ -92,21 +174,150 @@ describe("deployRelease", () => { deployStub.restore(); } }); + + test("throws the asset status when the download fails", async () => { + using _fetch = stubFetch(new Response("Unavailable", { status: 503 })); + const deployStub = stub(bunnyCdnApi, "deployScriptCode", () => + Promise.resolve({ ok: true as const }), + ); + try { + await expect( + deployRelease("https://example.com/asset.ts"), + ).rejects.toThrow("Failed to download release asset: 503"); + } finally { + deployStub.restore(); + } + }); + + test("names a blocked GitHub release download", async () => { + await expect( + runWithSubrequestBudget(() => + withSubrequestAllowance({ database: 50, external: 0, total: 0 }, () => + deployRelease("https://example.com/asset.ts"), + ), + ), + ).rejects.toThrow("Blocked external operation: GitHub release download"); + }); }); describe("deployLatestReleaseToDeno", () => { test("fetches the latest release and deploys it to a Deno app", async () => { using _fetch = stubReleaseFetch(); const deployStub = stub(denoDeployApi, "deployCode", () => - Promise.resolve({ ok: true as const, value: "https://app.deno.dev" }), + Promise.resolve({ ok: true as const, value: undefined }), ); try { - const release = await deployLatestReleaseToDeno("app_123"); - expect(release.tagName).toBe("v2099-01-01-120000"); + await runWithSubrequestBudget(async () => { + const release = await deployLatestReleaseToDeno("app_123"); + expect(release.tagName).toBe("v2099-01-01-120000"); + expect(getSubrequestUsage()).toEqual({ + database: 0, + external: 2, + total: 2, + }); + }); expect(deployStub.calls).toHaveLength(1); expect(deployStub.calls[0]!.args[0]).toBe("app_123"); } finally { deployStub.restore(); } }); + + test("names a release with no downloadable asset", async () => { + using _fetch = stubFetch( + new Response(JSON.stringify({ ...MOCK_RELEASE, assets: [] })), + ); + + await expect(deployLatestReleaseToDeno("app_123")).rejects.toThrow( + "Release has no downloadable asset", + ); + }); +}); + +describeWithEnv("recordScriptVersion", { db: true }, () => { + const commitKey = "current_script_commit"; + const markerKeys = [commitKey, CURRENT_SCRIPT_VERSION_KEY]; + const clearMarkers = (): Promise => + execute("DELETE FROM settings WHERE key IN (?, ?)", markerKeys); + const markerRows = (): Promise<{ key: string; value: string }[]> => + queryAll<{ key: string; value: string }>( + "SELECT key, value FROM settings WHERE key IN (?, ?) ORDER BY key", + markerKeys, + ); + + afterEach(() => { + setBuildTimestampForTest(null); + setBuildCommitForTest(null); + }); + + test("stores the commit under its stable plaintext key", async () => { + setBuildCommitForTest("abc123def4567890"); + await recordScriptVersion(); + + expect( + await queryOne<{ value: string }>( + "SELECT value FROM settings WHERE key = 'current_script_commit'", + ), + ).toEqual({ value: "abc123def4567890" }); + }); + + test("does not create an empty commit marker for a version-only build", async () => { + setBuildTimestampForTest("2026-06-19T12:00:00Z"); + setBuildCommitForTest(""); + await recordScriptVersion(); + + expect( + await queryOne<{ value: string }>( + "SELECT value FROM settings WHERE key = 'current_script_commit'", + ), + ).toBeNull(); + }); + + test("stores the exact rows for each available build marker", async () => { + await clearMarkers(); + setBuildTimestampForTest(""); + setBuildCommitForTest(""); + await runWithSubrequestBudget(async () => { + await recordScriptVersion(); + expect(getSubrequestUsage()).toEqual({ + database: 0, + external: 0, + total: 0, + }); + }); + expect(await readRecordedScriptCommit()).toBe(""); + + setBuildTimestampForTest("2026-06-19T12:00:00Z"); + await recordScriptVersion(); + expect(await markerRows()).toEqual([ + { + key: CURRENT_SCRIPT_VERSION_KEY, + value: "2026-06-19T12:00:00Z", + }, + ]); + + await clearMarkers(); + setBuildTimestampForTest(""); + setBuildCommitForTest("abc123def4567890"); + await recordScriptVersion(); + expect(await markerRows()).toEqual([ + { key: commitKey, value: "abc123def4567890" }, + ]); + + await clearMarkers(); + await execute("INSERT INTO settings (key, value) VALUES (?, ?)", [ + commitKey, + "stale-commit", + ]); + setBuildTimestampForTest("2026-06-20T00:00:00Z"); + setBuildCommitForTest(""); + await recordScriptVersion(); + expect(await markerRows()).toEqual([ + { key: commitKey, value: "" }, + { + key: CURRENT_SCRIPT_VERSION_KEY, + value: "2026-06-20T00:00:00Z", + }, + ]); + }); }); diff --git a/test/test-utils/activity-log.ts b/test/test-utils/activity-log.ts index 58ccb3aa4..77ee4c41d 100644 --- a/test/test-utils/activity-log.ts +++ b/test/test-utils/activity-log.ts @@ -16,6 +16,7 @@ * path, should import the real readers from `#shared/db/activityLog.ts`. */ +import { encrypt } from "#shared/crypto/encryption.ts"; import type { ActivityLogEntry, ListingWithActivityLog, @@ -26,10 +27,33 @@ import { getListingActivityLog as realGetListingActivityLog, getListingWithActivityLogOrNull as realGetListingWithActivityLogOrNull, } from "#shared/db/activityLog.ts"; +import { execute, queryOne } from "#shared/db/client.ts"; +import { nowIso } from "#shared/now.ts"; import { withTestSession } from "#test-utils/session.ts"; export { logActivity } from "#shared/db/activityLog.ts"; +/** Insert a row encrypted with DB_ENCRYPTION_KEY (the pre-migration format). */ +export const insertLegacyActivity = async ( + message: string, +): Promise => { + const result = await execute( + "INSERT INTO activity_log (message, created, listing_id, attendee_id) VALUES (?, ?, NULL, NULL)", + [await encrypt(message), nowIso()], + ); + return Number(result.lastInsertRowid); +}; + +/** Raw (still-encrypted) stored message for an activity-log row. */ +export const rawActivityMessage = async (id: number): Promise => { + const row = await queryOne<{ message: string }>( + "SELECT message FROM activity_log WHERE id = ?", + [id], + ); + if (!row) throw new Error(`Activity log entry not found: ${id}`); + return row.message; +}; + export const getAllActivityLog = ( limit?: number, ): Promise => diff --git a/test/test-utils/builder-mocks.ts b/test/test-utils/builder-mocks.ts index 85e6c5f6e..575f4a8b7 100644 --- a/test/test-utils/builder-mocks.ts +++ b/test/test-utils/builder-mocks.ts @@ -176,9 +176,7 @@ export const stubDenoBuilderApis = (opts: DenoBuilderMockOptions = {}) => ({ ), ), deployStub: stub(denoDeployApi, "deployCode", () => - Promise.resolve( - opts.deployResult ?? okResult("https://tickets-test.deno.dev"), - ), + Promise.resolve(opts.deployResult ?? okResult(undefined)), ), encKeyStub: stub( builderApi, diff --git a/test/test-utils/db-helpers/built-sites.ts b/test/test-utils/db-helpers/built-sites.ts index 0a018e290..6bc453142 100644 --- a/test/test-utils/db-helpers/built-sites.ts +++ b/test/test-utils/db-helpers/built-sites.ts @@ -1,4 +1,7 @@ -import type { BuiltSite, BuiltSiteFormInput } from "#shared/db/built-sites.ts"; +import type { + BuiltSite, + BuiltSiteFormInput, +} from "#shared/db/built-sites/types.ts"; import { withEnv } from "../env.ts"; import { doAuthenticatedFormRequest } from "./request.ts"; diff --git a/test/test-utils/debug.ts b/test/test-utils/debug.ts index 35d9d98f5..c65d797e8 100644 --- a/test/test-utils/debug.ts +++ b/test/test-utils/debug.ts @@ -54,13 +54,6 @@ export const makeDebugState = ( provider: "", webhookConfigured: false, }, - prune: { - addresses: "Never", - logins: "Never", - payments: "Never", - sessions: "Never", - strings: "Never", - }, runtime: { arch: "", denoVersion: "", diff --git a/test/test-utils/factories.ts b/test/test-utils/factories.ts index d7c0ef23c..7335cbfc4 100644 --- a/test/test-utils/factories.ts +++ b/test/test-utils/factories.ts @@ -1,6 +1,6 @@ import type { PricedLine, PricedOrder } from "#shared/checkout-pricing.ts"; import type { BlindIndex } from "#shared/crypto/sealed.ts"; -import type { BuiltSite } from "#shared/db/built-sites.ts"; +import type { BuiltSite } from "#shared/db/built-sites/types.ts"; import type { ListingInput } from "#shared/db/listings/table.ts"; import type { Answer, @@ -227,6 +227,8 @@ export const testBuiltSite = ( readOnlyFrom: "", renewalToken: null, renewalTokenIndex: null, + scheduledTaskKey: null, + siteDataRevision: 0, siteUrl: "https://test.b-cdn.net", updates: "release", ...overrides, diff --git a/test/test-utils/hash.ts b/test/test-utils/hash.ts new file mode 100644 index 000000000..2ba93cb94 --- /dev/null +++ b/test/test-utils/hash.ts @@ -0,0 +1,8 @@ +/** Return the lowercase SHA-256 digest of a value's stable JSON form. */ +export const jsonHash = async (value: unknown): Promise => { + const bytes = new TextEncoder().encode(JSON.stringify(value)); + const digest = new Uint8Array(await crypto.subtle.digest("SHA-256", bytes)); + return Array.from(digest) + .map((byte) => byte.toString(16).padStart(2, "0")) + .join(""); +}; diff --git a/test/test-utils/internal.ts b/test/test-utils/internal.ts index 7f4fd65e0..f6f85ff15 100644 --- a/test/test-utils/internal.ts +++ b/test/test-utils/internal.ts @@ -100,7 +100,6 @@ export type PaymentProviderType = export type SessionMetadata = import("#shared/payments.ts").SessionMetadata; -export type { BuiltSiteFormInput } from "#shared/db/built-sites.ts"; export type { GroupInput } from "#shared/db/groups.ts"; export type { HolidayInput } from "#shared/db/holidays.ts"; export type { ListingInput } from "#shared/db/listings/table.ts"; diff --git a/test/test-utils/mocks.ts b/test/test-utils/mocks.ts index 704aa1089..81e1f1c0c 100644 --- a/test/test-utils/mocks.ts +++ b/test/test-utils/mocks.ts @@ -19,6 +19,12 @@ import { type TestRequestOptions, } from "#test-utils/internal.ts"; +export const restoreStubsAfterEach = (stubs: { restore(): void }[]): void => { + afterEach(() => { + for (const active of stubs.splice(0)) active.restore(); + }); +}; + export const mockRequestWithHost = ( path: string, host: string, diff --git a/test/test-utils/record-queries.ts b/test/test-utils/record-queries.ts index eae2c9d8f..d3aaaae92 100644 --- a/test/test-utils/record-queries.ts +++ b/test/test-utils/record-queries.ts @@ -19,7 +19,7 @@ export type DbCallHooks = { /** Take over the statement by returning a promise, or null to forward. */ execute: (statement: InStatement | string) => Promise | null; /** Observe a batch's statements before they forward. */ - batch: (statements: InStatement[]) => void; + batch: (statements: InStatement[], mode?: "write" | "read") => void; }; /** Swap the db client for a proxy that observes or intercepts statements @@ -30,7 +30,7 @@ export const wrapDbClient = (hooks: DbCallHooks): (() => void) => { setDb( proxyMembers(real, { batch: (statements: InStatement[], mode?: "write" | "read") => { - hooks.batch(statements); + hooks.batch(statements, mode); return real.batch(statements, mode); }, execute: wrapExecute( diff --git a/test/test-utils/scheduled.ts b/test/test-utils/scheduled.ts new file mode 100644 index 000000000..b7ba820bc --- /dev/null +++ b/test/test-utils/scheduled.ts @@ -0,0 +1,57 @@ +import { expect } from "@std/expect"; +import { stub } from "@std/testing/mock"; +import { bunnyHostingProvider } from "#shared/bunny-cdn.ts"; +import { toBase64Url } from "#shared/crypto/utils.ts"; +import { insertBuiltSite } from "#shared/db/built-sites.ts"; + +export const TEST_SCHEDULED_KEY = toBase64Url(new Uint8Array(32).fill(7)); + +export const SCHEDULED_OWNER_ENV = { + BUNNY_API_KEY: "test-key", + CAN_BUILD_SITES: "true", + SCHEDULED_TASK_KEY: TEST_SCHEDULED_KEY, +} as const; + +export const scheduledAuthorization = ( + key = TEST_SCHEDULED_KEY, +): Record => ({ authorization: `Bearer ${key}` }); + +export const expectScheduledResponse = async ( + response: Response, + status: number, +): Promise => { + expect(response.status).toBe(status); + expect(response.headers.get("cache-control")).toBe("no-store"); + expect(await response.text()).toBe(""); +}; + +export const stubBunnySchedulerSecrets = ( + stubs: { restore(): void }[], + names: string[], + result: { ok: true } | { error: string; ok: false }, +): void => { + stubs.push( + stub(bunnyHostingProvider, "getSecretNames", () => + Promise.resolve({ ok: true, value: names }), + ), + stub(bunnyHostingProvider, "setSecrets", () => + Promise.resolve(result.ok ? { ...result, value: undefined } : result), + ), + ); +}; + +export const insertScheduledTestSite = ( + active: string | null = TEST_SCHEDULED_KEY, +) => + insertBuiltSite( + "Child", + "child.example.test", + "", + "", + false, + "42", + undefined, + "bunny", + "bunny", + active, + ); diff --git a/test/test-utils/secure-admin.ts b/test/test-utils/secure-admin.ts new file mode 100644 index 000000000..037c7c8da --- /dev/null +++ b/test/test-utils/secure-admin.ts @@ -0,0 +1,16 @@ +import { handleRequest } from "#routes"; +import { mockRequestWithHost } from "#test-utils/mocks.ts"; +import { testCookie } from "#test-utils/session.ts"; + +export const secureAdminCookie = async (): Promise => { + const cookie = await testCookie(); + const token = cookie.split("=").slice(1).join("="); + return `__Host-session=${token}`; +}; + +export const secureAdminGet = ( + path: string, + host: string, + cookie: string, +): Promise => + handleRequest(mockRequestWithHost(path, host, { headers: { cookie } })); diff --git a/test/test-utils/user-owned-records.ts b/test/test-utils/user-owned-records.ts new file mode 100644 index 000000000..dd3a27a7e --- /dev/null +++ b/test/test-utils/user-owned-records.ts @@ -0,0 +1,43 @@ +import { encrypt } from "#shared/crypto/encryption.ts"; +import { getDb, insert, queryAll } from "#shared/db/client.ts"; +import { logisticsAgents } from "#shared/db/logistics-agents.ts"; +import { createSession } from "#shared/db/sessions.ts"; +import { userAgents } from "#shared/db/user-agents.ts"; + +export const addUserOwnedAccessRecords = async ( + userId: number, + label: string, +): Promise => { + const agent = await logisticsAgents.table.insert({ name: `${label} agent` }); + await userAgents.setIds(userId, [agent.id]); + await createSession( + `${label}-session`, + `${label}-csrf`, + Date.now() + 60_000, + null, + userId, + ); + await getDb().execute( + insert("api_keys", { + created: "2026-07-19T00:00:00.000Z", + key_index: `${label}-key-index`, + last_used: "", + name: await encrypt(`${label} key`), + user_id: userId, + wrapped_data_key: `${label}-wrapped-key`, + }), + ); +}; + +export const getUserOwnedRowSources = async ( + userId: number, +): Promise => + ( + await queryAll<{ source: string }>( + `SELECT 'api_keys' AS source FROM api_keys WHERE user_id = ? + UNION ALL SELECT 'sessions' FROM sessions WHERE user_id = ? + UNION ALL SELECT 'user_logistics_agents' FROM user_logistics_agents WHERE user_id = ? + UNION ALL SELECT 'users' FROM users WHERE id = ?`, + [userId, userId, userId, userId], + ) + ).map(({ source }) => source); diff --git a/test/ui/templates/admin/built-sites.test.ts b/test/ui/templates/admin/built-sites.test.ts index 3883c57ea..d05ae4033 100644 --- a/test/ui/templates/admin/built-sites.test.ts +++ b/test/ui/templates/admin/built-sites.test.ts @@ -6,294 +6,373 @@ import { SecretsPanel, UpdatePanel, } from "#templates/admin/built-sites/panels.tsx"; -import { adminBuiltSitesPage } from "#templates/admin/built-sites.tsx"; +import { + adminBuiltSiteDeletePage, + adminBuiltSiteNewPage, + adminBuiltSitesPage, + BuiltSiteEditPanel, + builtSiteToFieldValues, +} from "#templates/admin/built-sites.tsx"; import { setupTestEncryptionKey, withEnv } from "#test-utils/env.ts"; import { testBuiltSite, testListingWithCount } from "#test-utils/factories.ts"; const TEST_SESSION = { adminLevel: "owner" as const }; -beforeAll(async () => { - setupTestEncryptionKey(); - await signCsrfToken(); -}); - -describe("adminBuiltSitesPage", () => { - test("renders formatted deadline column", () => { - const site = testBuiltSite({ readOnlyFrom: "2099-06-01T00:00:00Z" }); - const html = adminBuiltSitesPage([site], TEST_SESSION); - expect(html).toContain("Read-only from"); - expect(html).toContain("in"); - expect(html).toContain("day"); +describe("built-site templates", () => { + beforeAll(async () => { + setupTestEncryptionKey(); + await signCsrfToken(); }); - test("renders 'never' for empty deadline", () => { - const site = testBuiltSite({ readOnlyFrom: "" }); - const html = adminBuiltSitesPage([site], TEST_SESSION); - expect(html).toContain("never"); - }); + describe("adminBuiltSitesPage", () => { + test("renders formatted deadline column", () => { + const site = testBuiltSite({ readOnlyFrom: "2099-06-01T00:00:00Z" }); + const html = adminBuiltSitesPage([site], TEST_SESSION); + expect(html).toContain("Read-only from"); + expect(html).toContain("in"); + expect(html).toContain("day"); + }); - test("links each site name to its entity page and has no list delete link", () => { - const site = testBuiltSite({ id: 7, name: "Linky", readOnlyFrom: "" }); - const html = adminBuiltSitesPage([site], TEST_SESSION); - expect(html).toContain('href="/admin/built-sites/7">Linky'); - expect(html).not.toContain("/admin/built-sites/7/delete"); - }); + test("renders 'never' for empty deadline", () => { + const site = testBuiltSite({ readOnlyFrom: "" }); + const html = adminBuiltSitesPage([site], TEST_SESSION); + expect(html).toContain("never"); + }); - test("warns when no qualifying renewal tier is configured", () => { - const site = testBuiltSite({ readOnlyFrom: "" }); - const html = adminBuiltSitesPage([site], TEST_SESSION, undefined, []); - expect(html).toContain("Renewal tiers"); - expect(html).toContain("No renewal tier listing is configured"); - expect(html).toContain("won't be able to renew"); - }); + test("links each site name to its entity page and has no list delete link", () => { + const site = testBuiltSite({ id: 7, name: "Linky", readOnlyFrom: "" }); + const html = adminBuiltSitesPage([site], TEST_SESSION); + expect(html).toContain('href="/admin/built-sites/7">Linky'); + expect(html).not.toContain("/admin/built-sites/7/delete"); + }); - test("keeps the read-only entity link but hides create actions", () => { - using _env = withEnv({ READ_ONLY_FROM: "2020-01-01T00:00:00.000Z" }); - const site = testBuiltSite({ id: 7, name: "Linky", readOnlyFrom: "" }); - const html = adminBuiltSitesPage([site], TEST_SESSION); - expect(html).toContain("Linky"); - expect(html).not.toContain('href="/admin/built-sites/new"'); - expect(html).toContain('href="/admin/built-sites/7"'); - }); + test("warns when no qualifying renewal tier is configured", () => { + const site = testBuiltSite({ readOnlyFrom: "" }); + const html = adminBuiltSitesPage([site], TEST_SESSION, undefined, []); + expect(html).toContain("Renewal tiers"); + expect(html).toContain("No renewal tier listing is configured"); + expect(html).toContain("won't be able to renew"); + }); - test("lists each tier with units sold from attendee_count", () => { - const monthly = testListingWithCount({ - attendee_count: 7, - hidden: true, - id: 11, - months_per_unit: 1, - name: "Monthly tier", - purchase_only: true, - unit_price: 500, + test("keeps the read-only entity link but hides create actions", () => { + using _env = withEnv({ READ_ONLY_FROM: "2020-01-01T00:00:00.000Z" }); + const site = testBuiltSite({ id: 7, name: "Linky", readOnlyFrom: "" }); + const html = adminBuiltSitesPage([site], TEST_SESSION); + expect(html).toContain("Linky"); + expect(html).not.toContain('href="/admin/built-sites/new"'); + expect(html).toContain('href="/admin/built-sites/7"'); }); - const annual = testListingWithCount({ - attendee_count: 2, - hidden: true, - id: 12, - months_per_unit: 12, - name: "Annual tier", - purchase_only: true, - unit_price: 5000, + + test("lists only Bunny hosting ids in the shared host marker", () => { + const html = adminBuiltSitesPage( + [ + testBuiltSite({ hostingId: "bunny-1", hostingProvider: "bunny" }), + testBuiltSite({ hostingId: "deno-1", hostingProvider: "deno" }), + testBuiltSite({ hostingId: "", hostingProvider: "bunny" }), + ], + TEST_SESSION, + ); + expect(html).toContain("bunny-1"); + expect(html).not.toContain("deno-1

"); }); - const site = testBuiltSite({ readOnlyFrom: "" }); - const html = adminBuiltSitesPage([site], TEST_SESSION, undefined, [ - monthly, - annual, - ]); - expect(html).toContain("Monthly tier"); - expect(html).toContain("Annual tier"); - // Units sold = attendee_count - expect(html).toContain(">7<"); - expect(html).toContain(">2<"); - // Linked back to the listing detail page - expect(html).toContain('href="/admin/listing/11"'); - expect(html).toContain('href="/admin/listing/12"'); - // Warning copy must not appear when tiers exist - expect(html).not.toContain("No renewal tier listing is configured"); - }); -}); -describe("renewalPanelFor — provisioned site", () => { - const provisionedSite = testBuiltSite({ - readOnlyFrom: "2027-01-15T00:00:00Z", - renewalToken: "real-customer-renewal-token", - renewalTokenIndex: "some-index", + test("lists each tier with units sold from attendee_count", () => { + const monthly = testListingWithCount({ + attendee_count: 7, + hidden: true, + id: 11, + months_per_unit: 1, + name: "Monthly tier", + purchase_only: true, + unit_price: 500, + }); + const annual = testListingWithCount({ + attendee_count: 2, + hidden: true, + id: 12, + months_per_unit: 12, + name: "Annual tier", + purchase_only: true, + unit_price: 5000, + }); + const site = testBuiltSite({ readOnlyFrom: "" }); + const html = adminBuiltSitesPage([site], TEST_SESSION, undefined, [ + monthly, + annual, + ]); + expect(html).toContain("Monthly tier"); + expect(html).toContain("Annual tier"); + // Units sold = attendee_count + expect(html).toContain(">7<"); + expect(html).toContain(">2<"); + // Linked back to the listing detail page + expect(html).toContain('href="/admin/listing/11"'); + expect(html).toContain('href="/admin/listing/12"'); + // Warning copy must not appear when tiers exist + expect(html).not.toContain("No renewal tier listing is configured"); + }); }); - test("shows renewal URL and rotate/bump/override/re-sync forms; no tier picker", () => { - const html = String(renewalPanelFor(provisionedSite)); - expect(html).toContain("Renewal URL"); - expect(html).toContain("rotate-renewal-token"); - expect(html).toContain("bump-deadline"); - expect(html).toContain("override-deadline"); - expect(html).toContain("re-sync-deadline"); - expect(html).toContain("Rotate token"); - expect(html).toContain("Re-sync deadline"); - expect(html).not.toContain("tier_listing_id"); - expect(html).not.toContain("set-renewal-tier"); - }); + describe("built-site forms", () => { + test("maps exact empty and populated field values", () => { + expect(builtSiteToFieldValues()).toEqual({ + assignable: "", + db_provider: "bunny", + db_token: "", + db_url: "", + hosting_id: "", + hosting_provider: "bunny", + name: "", + site_url: "", + updates: "release", + }); + expect( + builtSiteToFieldValues( + testBuiltSite({ + assignable: true, + dbProvider: "turso", + dbToken: "token", + dbUrl: "libsql://site", + hostingId: "app-1", + hostingProvider: "deno", + name: "Site", + siteUrl: "site.example", + updates: "alpha", + }), + ), + ).toEqual({ + assignable: "1", + db_provider: "turso", + db_token: "token", + db_url: "libsql://site", + hosting_id: "app-1", + hosting_provider: "deno", + name: "Site", + site_url: "site.example", + updates: "alpha", + }); + }); - test("renders the actual renewal URL (token, not placeholder)", () => { - const html = String(renewalPanelFor(provisionedSite)); - // The real token must appear inside a /renew/?t=… URL. The previous - // implementation rendered "" literally with a bogus host — guard - // against regression by asserting both the path shape and the token. - expect(html).toContain("/renew/?t=real-customer-renewal-token"); - expect(html).not.toContain("?t="); + test("renders new, edit, and delete destinations and labels", () => { + const site = testBuiltSite({ id: 42, name: "Delete " }); + const newPage = adminBuiltSiteNewPage(TEST_SESSION); + expect(newPage).toContain('action="/admin/built-sites"'); + expect(newPage).toContain("Add Built Site"); + const edit = String(BuiltSiteEditPanel({ site })); + expect(edit).toContain('action="/admin/built-sites/42/edit"'); + expect(edit).toContain("Save Changes"); + const deletePage = adminBuiltSiteDeletePage(site, TEST_SESSION); + expect(deletePage).toContain('action="/admin/built-sites/42/delete"'); + expect(deletePage).toContain("Delete Built Site"); + expect(deletePage).toContain("Delete <site>"); + expect(deletePage).not.toContain("button-danger"); + }); }); - test("labels the shared bump/override forms inline (no headings)", () => { - const html = String(renewalPanelFor(provisionedSite)); - expect(html).toContain('