Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -233,6 +233,14 @@ jobs:
- name: 'Type check: va-apple-music-url-remediation (BS#2009)'
run: npm run typecheck --workspace=jobs/va-apple-music-url-remediation

# Same BS#2009 gap as above: this job carries the Track 2d
# precondition gate that stands between a rerun and an unrecoverable
# legal-name loss (DJ real-name PII safeguards plan) — the one job in
# jobs/** where shipping uncompiled-but-green is the worst possible
# place for that gap.
- name: 'Type check: auth-user-name-backfill'
run: npm run typecheck --workspace=jobs/auth-user-name-backfill

- name: Lint
run: npm run lint

Expand Down
1 change: 1 addition & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ npm workspaces:
| `@wxyc/flowsheet-april-gap-import` | `jobs/flowsheet-april-gap-import/` | One-shot dry-run-by-default import (BS#2119): backfill the closed BS#351 residue — 403 `FLOWSHEET_ENTRY_PROD` rows tubafrenzy holds that Backend never received because the pre-fix ETL silently dropped every track entry with `START_TIME=0`. Default scope is the unambiguous 2026-04-16 → 2026-04-20 window (399 rows / 15 shows, `GAP_IMPORT_WINDOW_START`/`END` widen it); the 4 post-Phase-3 August rows are deliberately excluded pending a per-row provenance check (see #1543). Insert-only, `ON CONFLICT (legacy_entry_id) DO NOTHING`, never `DO UPDATE`. Reuses `jobs/flowsheet-etl`'s pure mappers (extracted to `transform.ts`/`show-id-map.ts`/`fetch-legacy.ts`'s `fetchLegacyEntriesInWindow` by this same issue, since importing `job.ts` itself self-invokes the ETL) — never `flowsheet-etl`/`legacy-linkage-resolve` directly. Batched inserts with cooperative live-DJ pause + inter-batch gap. **Four refusals before any write** (all also fire in dry-run): Backend-side id-count floor, cohort-size ceiling, an upstream `GAP_IMPORT_MIN_CANDIDATE_COUNT` floor (default 1 — zero candidates is a bad window, not an empty one; candidates come from tubafrenzy and don't depend on Backend state), and a `GAP_IMPORT_MAX_NULL_KEY_ROWS` orphan guard (default 0) counting **non-marker** rows the target shows already hold with `legacy_entry_id IS NULL` — both the cohort diff and the `ON CONFLICT` target key on that column and a unique index doesn't constrain NULLs, so a dj-site row whose back-stamp was skipped (the `legacy-mirror-reconcile` Sweep 2 orphan class) is invisible to both and would be inserted twice. `dj_name` comes from the canonical `resolveShowDjName` (`@wxyc/database` `dj-name.ts`, extracted from `flowsheet.service.ts` by the BS#2119 review), never a re-derived `COALESCE` — the donor's copy predates `dj_name_override` (BS#1321) and omits the literal-"Anonymous" filter (BS#1286). |
| `@wxyc/va-apple-music-url-remediation` | `jobs/va-apple-music-url-remediation/` | One-shot V/A Apple-URL remediation (BS#2000) for the deep-links BS persisted from LML's pre-LML#1139 V/A-blind Apple **track** matcher (the `Various Artists - ` prefix scores ~85 between any two V/A credits, and the LML#782 album-dropped fallback re-admitted winners on a vacuous artist + a generic standard's title). **Two arms, because the two tables have different recovery paths.** `flowsheet` gets a real LML re-verify — one lookup per DISTINCT `(artist, album, track)` (BS#1192: Apple URLs are track-aware), fanned to every row with that triple — because it has no status column and the worker never revisits `enriched_match`, so a null there is terminal. `album_metadata` gets **zero LML calls**: it is invalidated to `apple_music_url=NULL` + `apple_music_status='unresolved'` + `streaming_reask_attempts=0` so the BS#1915 hourly `streaming-reask.ts` sweep re-adjudicates it through the guarded matcher (it is album-keyed while the URL is a _track_ deep-link, so there is no honest triple to re-query with). **Scope correction to the issue body, which said not to touch `album_metadata`:** `flowsheet.service.ts` serves `coalesce(album_metadata.apple_music_url, flowsheet.apple_music_url)` and `enrich.ts` writes BOTH from the same track probe, so for a linked row the DJ-visible value is the album_metadata one. **Phase order is load-bearing — flowsheet FIRST**, since nulling `album_metadata` unmasks flowsheet's value via that coalesce (album phase is skipped entirely if flowsheet fails). Three-way verdict (`url` / `none` / `indeterminate`): a `none` needs THREE consecutive null passes and `indeterminate` (shed, BS#1293 skip, empty `results`, transport error) writes nothing — LML#904 measured ~56% of Apple probes nulling on LML's own self-throttle, so a single null is not evidence (same BS#1915 doctrine, bought procedurally since `flowsheet` lacks the status column). In-band rescue-rate detector aborts the run when observed throttle-nulls exceed `VA_REMEDIATION_MAX_RESCUE_RATE`; an indeterminate triple is skipped-and-accounted, never halted-in-place (the BS#1011 wedge). Coarse SQL net over `fold_artist_name` (the fold MUST be in SQL — `lower('Vàrious Artists')` matches nothing) + the `isVariousArtistsCredit` arbiter in `va-artist.ts`, which shares `foldArtistName` with the net and deliberately disagrees with all three existing repo V/A predicates (two substring, one that calls `Various Artists-Rock-Y` not-various). Compare-and-set UPDATE (this job overwrites a NON-null value while two other writers touch the column); flowsheet UPDATE omits `updated_at` (trigger 0084); ANALYZE after. Dry-run by default and makes ZERO LML calls. Hard-gated on LML#1139's guard **and** its cache purge being deployed, plus a confirmed `LML_APPLE_MUSIC_RATE_PER_MIN` roll-up. Cooperative live-DJ pause (BS#2009): both phases probe once per page, before the page loads (never once per row); a throwing probe is fail-open (logged, captured, treated as no activity) rather than aborting the run and losing its `last_id` resume cursors — ported from the `streaming-url-remediation` / `flowsheet-ghost-row-sweep` donors' `waitForQuietPeriod` + `safeProbe`. |
| `@wxyc/metadata-no-match-digest` | `jobs/metadata-no-match-digest/` | Daily cron digest: reads the `cronjob_runs` watermark, queries `flowsheet` for rows that flipped to `metadata_status='enriched_no_match'` since the last run (filtered/sorted on `updated_at`, never `metadata_attempt_at` — the live CDC enrichment worker deliberately leaves that NULL on no-match rows), and emails one HTML+text digest to `DIGEST_RECIPIENT_EMAIL` (default `jake@wxyc.org`) via a self-contained SES sender (`email.ts`, deliberately not `@wxyc/authentication` — see the job's README): catalog/rotation-linked misses listed in full (`format.ts` Section A, feeds the `rotation.discogs_release_id` backfill), freeform misses aggregated by artist with a top-25 + "…and N more" cap (Section B), each line with a synthesized Discogs search URL. Zero misses sends no email but still advances the watermark; a send failure leaves the watermark untouched so the next run retries the same window. First run bounds the window to the last 24h. Read-only against `flowsheet`; no schema migration. All times rendered Pacific (`PT` label); the fixed-UTC cron schedule (`07 15 * * *`) means the rendered wall-clock hour drifts across the DST boundary (documented in the job's README). DB-only — no `@wxyc/lml-client` dependency, exempt from the LML cron-spacing policy (`docs/ops-cron-scheduling.md`). |
| `@wxyc/auth-user-name-backfill` | `jobs/auth-user-name-backfill/` | One-shot backfill (DJ real-name PII safeguards plan, Track 2d): rewrites `auth_user.name` to `resolveDjDisplayName(dj_name) ?? username` for the ~139 rows that predate the `databaseHooks.user.create.before`/`update.before` choke point (`shared/authentication/src/derive-user-display-name.ts`), skipping anonymous users and the `'Auto DJ'` service account. Dry-run by default (`--execute` to write). Opens with a machine-enforced preserve-first precondition gate (`decide.ts`'s `violatesPreserveFirstPrecondition`) that aborts non-zero — in both dry-run and execute mode — if any row still holds its ONLY copy of a legal name in `name` (real_name blank, name not `'Anonymous'`/`'Auto DJ'`/username/the on-air handle): **must NOT run with `--execute` before plan step 2a** (the reviewed manual SQL that copies `name -> real_name` for exactly those rows) has run against the target database — the gate makes that ordering machine-enforced rather than operator-trusted. |

### API Server (`apps/backend`)

Expand Down
37 changes: 37 additions & 0 deletions Dockerfile.auth-user-name-backfill
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
#Build stage
FROM node:24-alpine AS builder

WORKDIR /auth-user-name-backfill-builder

COPY ./package.json ./package-lock.json ./
COPY ./tsconfig.base.json ./
COPY ./shared/database ./shared/database
COPY ./jobs/auth-user-name-backfill ./jobs/auth-user-name-backfill

RUN npm ci && npm run build --workspace=@wxyc/database --workspace=@wxyc/auth-user-name-backfill

#Production stage
FROM node:24-alpine AS prod

WORKDIR /auth-user-name-backfill

COPY ./package* ./
COPY ./jobs/auth-user-name-backfill/package* ./jobs/auth-user-name-backfill/
COPY ./shared/database/package* ./shared/database/

RUN npm install --omit=dev

COPY --from=builder ./auth-user-name-backfill-builder/jobs/auth-user-name-backfill/dist ./jobs/auth-user-name-backfill/dist
COPY --from=builder ./auth-user-name-backfill-builder/shared/database/dist ./shared/database/dist

# Tiny single-table workload (~139 candidate rows) — no bulk-update tuning
# needed here, unlike the batched flowsheet/shows backfills.
ENV DB_APPLICATION_NAME=wxyc-auth-user-name-backfill

# ENTRYPOINT + empty CMD so docker-level args (`--execute`) pass through to
# the job rather than replacing the launcher — same reasoning as
# Dockerfile.legacy-dj-name-remediation. With `CMD ["npm", "start"]`,
# `docker run <image> --execute` would replace the whole command instead of
# extending it.
ENTRYPOINT ["node", "/auth-user-name-backfill/jobs/auth-user-name-backfill/dist/job.js"]
CMD []
11 changes: 7 additions & 4 deletions apps/auth/app.ts
Original file line number Diff line number Diff line change
Expand Up @@ -255,10 +255,14 @@ app.post('/auth/admin/provision-user', async (req, res) => {
return res.status(403).json({ error: 'Forbidden: admin role required' });
}

// Validate required fields
// Validate required fields. `name` is deliberately NOT in this list
// (DJ real-name PII safeguards plan, Track 2c) — provisionUser() derives
// the stored name itself from djName/username, never from a
// client-supplied value. A still-supplied `name` is accepted-and-ignored
// for the deploy overlap window while dj-site still sends it.
const body = (req.body ?? {}) as Record<string, unknown>;
const { email, username, name, organizationSlug, role, realName, djName } = body;
const missing = ['email', 'username', 'name', 'organizationSlug', 'role'].filter(
const { email, username, organizationSlug, role, realName, djName } = body;
const missing = ['email', 'username', 'organizationSlug', 'role'].filter(
(field) => !body[field] || typeof body[field] !== 'string'
);
if (missing.length > 0) {
Expand All @@ -273,7 +277,6 @@ app.post('/auth/admin/provision-user', async (req, res) => {
const result = await provisionUser({
email: email as string,
username: username as string,
name: name as string,
organizationSlug: organizationSlug as string,
role: role as string,
realName: realName as string | undefined,
Expand Down
11 changes: 9 additions & 2 deletions apps/auth/create-auto-dj-user.ts
Original file line number Diff line number Diff line change
Expand Up @@ -77,8 +77,15 @@ export const createAutoDjUser = async (): Promise<void> => {
await provisionUser({
email,
username: 'autodj',
name: 'Auto DJ', // required (notNull); internal, never surfaced publicly
djName: 'Auto DJ', // the public HANDLE — what appears on-air
// `name` is required (notNull) but is NOT the invariant carrier here —
// `djName` below is. deriveUserNameOnCreate (databaseHooks.user.create.before,
// shared/authentication/src/derive-user-display-name.ts) derives the stored
// `name` from `djName`, falling back to `username` only when `djName` is
// absent/blank/'Anonymous'. Deleting `djName` while trusting a comment that
// called `name` load-bearing would surface this account as its username
// ('autodj') instead of the intended 'Auto DJ' handle.
name: 'Auto DJ',
djName: 'Auto DJ', // the public HANDLE — what appears on-air, and what derives `name` above
organizationSlug,
role: 'dj',
password,
Expand Down
Loading
Loading