Skip to content

feat(sing): auto-resolve failed singer-submission downloads (v0.87.0) - #202

Merged
beveradb merged 9 commits into
mainfrom
feat/sess-20260717-0153-singer-submission-validation
Jul 17, 2026
Merged

feat(sing): auto-resolve failed singer-submission downloads (v0.87.0)#202
beveradb merged 9 commits into
mainfrom
feat/sess-20260717-0153-singer-submission-validation

Conversation

@beveradb

Copy link
Copy Markdown
Contributor

Auto-resolving singer submissions (download validation + fallback)

Motivated by the 2026-07-09 live incident: a singer requested "Beetlejuice – Say My Name", the picked YouTube version was a private video, the download failed with a dead red ❌, and the KJ had to fix it by hand mid-show.

What changed

When an approved singer submission's YouTube download fails, the download worker now auto-heals instead of dead-ending:

  • Unavailable (private/deleted/blocked) → advance to the next-best candidate version of the same song.
  • Transient (timeout / HTTP 429 / bgutil down / network / any unknown error) → retry the same candidate a bounded number of times before advancing, so a flaky network never discards a good video.
  • Bounded to 3 candidates. On success the request is rebound to the version that landed (so /my-requests shows the right one) and the singer gets a push; only a fully-exhausted request surfaces the terminal ❌ + an honest "KJ notified" push.

How it works

  • New pure, exhaustively-tested sing_resolve.py (classify unavailable-vs-transient; candidate iteration with caps) — the download attempt is the probe.
  • media.download_video records its failure reason on media._last_error (it previously swallowed the error); the single-threaded _download_worker reads and classifies it.
  • approve_sing_request attaches a ranked YouTube candidate list to the queue item, built via the existing _pick_version_from_kj_pick translator. _preserve_versions_meta keeps the versions[] snapshot through kj_pick binding so multi-version songs (the common case) have alternates.
  • Web Push copy for resolved_alt / unavailable outcomes.

Scope / notes

  • v1 falls back across YouTube-type candidates only; cross-source (local/Divebar) fallback is a documented follow-up.
  • Deploy: backend change → requires systemctl restart kj-controller (interrupts playback — deploy between songs). No DB migration; new queue-item fields are in-memory and ignored on rollback.

Testing

  • sing_resolve 100% covered (incl. the incident's private-video string); push_dispatcher 90%.
  • Integration tests: advance-on-unavailable, terminal-when-all-dead, transient-retries-same-candidate, URL-form dedup, and an end-to-end reproduction of the 2026-07-09 incident.
  • Full unit+integration suite green (e2e failures are environmental — no browser here).

Design + plan: docs/archive/2026-07-09-singer-submission-validation-{design,plan}.md.

@coderabbitai ignore

beveradb and others added 9 commits July 17, 2026 01:58
Step-by-step plan derived from the approved design spec: new pure
sing_resolve module (error classification + candidate iteration),
fallback wired into the existing _download_worker error branch reusing
_pick_version_from_kj_pick for version translation, async + notify.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
sing_resolve.classify_error decides unavailable (advance to next
candidate) vs transient (retry same); next_candidate_index bounds
attempts by MAX_CANDIDATES. Unknown/empty errors default to transient so
a network fluke never discards a good candidate. Fully unit-tested,
including the 2026-07-09 private-video incident string.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
download_video swallows yt-dlp errors and returns (None, None); the
sing-request fallback worker needs the reason to classify unavailable vs
transient. Set self._last_error at each failure return, reset at start,
cleared on success. Downloads are serialized through the single worker so
the shared attribute is race-free.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire sing_resolve into the download worker: on a failed sing-request
download, retry the same candidate on a transient blip or advance to the
next ranked YouTube candidate on an unavailable one, rebinding the
request source so /my-requests reflects the version that landed. Only a
truly exhausted request surfaces the terminal red. approve_sing_request
attaches the ranked candidate list (built via the existing
_pick_version_from_kj_pick translator) to the youtube queue item.

Adds resolved_alt/unavailable Web Push copy and generalises
notify_request_decision's step mapping. Integration tests cover advance,
terminal-when-all-dead, and transient-retries-same-candidate.

v1 falls back across YouTube-type candidates only; cross-source
(local/divebar) fallback is a documented follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Multi-version songs (the incident case) route through kj_pick, and
binding a version rewrote source_meta to the picked version's meta —
dropping the versions[] snapshot the fallback worker needs. Merge the
snapshot back in at both binding sites (admin approve route +
resolve_kj_pick_best auto-bind) so approve_sing_request can build a real
candidate list. Without this the fallback was dead for the main case.

Pivot from the planned client change: direct youtube/kn picks are only
ever single-version (no alternates), so attaching versions there adds
nothing; the value is entirely in preserving the kj_pick snapshot. Adds
an end-to-end test reproducing the 2026-07-09 private-video incident.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ARCHITECTURE: new sing_resolve module row + a 'Singer submission download
fallback' subsection. App CHANGELOG: dated entry with deploy note.
TROUBLESHOOTING: what an auto-swapped version / 'unavailable' means for
the KJ. Plan: mark implemented + record as-built deviations (D1-D5).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ifier

Review follow-ups:
- _build_sing_fallback_candidates now dedups on the canonical YouTube
  video id (via youtube_id_from_url) instead of the raw URL, so the same
  dead video in youtu.be vs watch?v= form can't burn two candidate slots.
- Drop the over-broad 'does not exist' UNAVAILABLE pattern (kept the
  specific 'video does not exist') to avoid misclassifying a transient
  'URL does not exist' server blip as unavailable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…53-singer-submission-validation

# Conflicts:
#	kj-controller/docs/CHANGELOG.md
#	kj-controller/pyproject.toml
@beveradb
beveradb merged commit 1493f5d into main Jul 17, 2026
1 check passed
@beveradb
beveradb deleted the feat/sess-20260717-0153-singer-submission-validation branch July 17, 2026 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant