feat(singers): device-session provenance + smarter Merge modal (v0.91.0)

docs/ARCHITECTURE.md

@@ -87,7 +87,8 @@ KJ Controller is a web-based karaoke show management application. A Flask backen
 | `sleep_mode.py` | ~100 | `SleepManager` class: enter/exit low-power sleep mode, stop services, unmount SSD |
 | `push_dispatcher.py` | ~200 | `PushDispatcher` class: VAPID config, subscription scan, ladder decision (`now_singing`/`up_next`/`up_in_2`), dedup via `last_sent_state`, 500ms debounce, `ThreadPoolExecutor` send pool. Pure helpers at module level (`decide_ladder_step`, `next_entry_for_phone`, `render_payload`). |
 | `sing.py` | ~280 | Public `/sing/*` blueprint (landing, search, submit, status, rules, now, manifest, sw, push subscribe/unsubscribe) + token-gate decorator + per-IP rate limiter + host-based route guard + QR-overlay auto-sync helper |
-| `sing_store.py` | ~260 | `SingStore` class: SQLite CRUD for `sing_requests` + `sing_push_subscriptions` + event-token helpers (regenerate / enable / auto-approve) on `rotation_meta` |
+| `sing_store.py` | ~260 | `SingStore` class: SQLite CRUD for `sing_requests` (incl. `user_agent` device capture + `get_requests_for_entries` night-scoped session lookup) + `sing_push_subscriptions` + event-token helpers (regenerate / enable / auto-approve) on `rotation_meta` |
+| `ua_parse.py` | ~130 | Pure best-effort User-Agent parser (`parse_user_agent`, `summarize`) → friendly browser/OS/device labels; no dependency |
 | `routes.py` | ~1000 | Flask Blueprint with all route handlers (includes `/rotation/requests/*` admin endpoints for the public request form). Hosts the playability gates: tier-1 inline `_playability_gate` (link/upload/download hard-block) + tier-2 async render verification (`_enqueue_tier2` → single-worker queue → `_run_tier2_check` against the active renderer, stamps `playability_warning`) |
 | `wait_estimate.py` | ~80 | Pure function `compute_estimate(entries, target_id, cfg)` producing `{position, expected_s, range_low_s, range_high_s, spread_source, close_to_front, now_singing}`. Uses tonight's sung-entry variance for the range; falls back to a configurable minimum spread. |
 | `sing_resolve.py` | ~120 | Pure decision logic for singer-submission download fallback: `classify_error` (unavailable → advance to next candidate vs transient → retry same) and `next_candidate_index` (bounded by `MAX_CANDIDATES`). No yt-dlp/network/Flask deps → exhaustively unit-tested. |
@@ -409,6 +410,8 @@ Additional `static-sing/` assets added in sub-project #4:
 
 **Duet partners + multi-song done screen (2026-05-15):** Singers can attach up to 3 duet partners (name + optional phone) on the confirm screen via a new `additional_singers TEXT NULL` column on `sing_requests` (JSON array). `POST /sing/submit` validates the field (max 3, name required, phone format optional). `approve_sing_request` builds a `singers=[primary, …partner_names]` list and passes it to `rotation.add_entry(...)` — the existing `singers_json` plumbing on `rotation_entries` joins names with ` & ` for the legacy `singer` text column and persists the structured list. KJ admin approval card renders a duet block with `sms:` links for partner phones. The singer's done screen now lists all their submitted requests via a new `GET /sing/my-requests?ids=…` endpoint (max 20 ids per call, returns `{now_playing, requests:[{request, estimate?, performed}]}`) and includes a "+ Request another song" button that resets song-picking state while preserving identity. Request ids are tracked in `localStorage` (`sing_my_request_ids`) scoped per token so yesterday's ids don't leak into a new event. Partner phones are display-only (no push subscriptions for partners) — the KJ texts them manually from the admin card.
 
+**Singer session provenance + smarter Merge (2026-08-13):** Motivated by the "two Chailas" incident (a self-registered singer vs a duet-partner label typed into someone else's request looked like two people). Every `/sing/submit` and `/sing/requests/<id>/change` now stores the submitting device's `User-Agent` on a new `sing_requests.user_agent` column. `routes._add_singer_session_info(singer_stats)` (called alongside `_add_last_sang_to_singer_stats` in `GET /rotation` and every singer action response) attaches a `session` block to each singer by matching their rotation-entry ids to linked requests via `SingStore.get_requests_for_entries(entry_ids, night_started)` (night-scoped, fails closed like the SMS phone lookup): `origin` is `singer_ui` (owns a linked request whose `singer_name` is theirs → `has_device`, carries `device`/`phone`/`request_count`/`sources`), `duet_partner` (only appears in another request's `additional_singers`), or `kj_added` (no linked request). The frontend shows a 📱 icon only on `singer_ui` singers (click → device-details popup) and replaces the inline Merge dropdown with a modal: searchable singer list (device-linked first), then a confirm step spelling out KEEP vs REMOVE, combined sung/queued totals, and a real-device warning + **Swap** that defaults the keeper to the phone-linked singer. `ua_parse.py` turns the raw UA into a friendly summary (Android UAs also yield the model, e.g. SM-S911B; iOS exposes only "iPhone").
+
 **Done-screen ordering + sung-song clearing (2026-07-17, v0.88.0):** The "Your songs tonight" list previously rendered in submission order (the client passes its `localStorage` id list to `/my-requests`, which echoes them back in that order) and never dropped sung songs (a `sing_request` stays status `approved` after its rotation entry is sung, and `get_rotation()` returns the ACTIVE queue only, so the sung entry got a `position=None` estimate and lingered as "Added to the queue" all night). Fix: `/my-requests` now sets a per-item `performed` boolean — for a linked entry not in the active queue it looks up `rotation.store.get_entry(...)` and marks `performed=True` when the entry is Done/Left (no estimate attached). The client (`sing.js`) sorts the active list into sung order (now singing → queue position → awaiting-KJ) and moves performed songs into a collapsed **"✓ Already sung tonight (N)"** section that's read-only (no cancel/change/reorder). `performed` songs are excluded from `_liveSongs`, so the persistent "🎤 My songs (N)" bar count and boot smart-restore ignore them (a singer who has sung everything lands on the request screen, not a stale list).
 
 ### Singer Web Push (sub-project #4)

docs/archive/2026-08-13-singer-session-provenance-plan.md

@@ -0,0 +1,87 @@
+# Singer session provenance + smarter Merge — plan
+
+**Date:** 2026-08-13
+**Branch:** `feat/singer-session-provenance`
+
+## Motivation
+
+Live-show investigation (the "two Chailas" case): the KJ Singers list can show the
+same person twice when one name is a **self-registered singer** (own phone session)
+and the other is a **duet-partner label** typed into someone else's submission, or a
+**KJ-hand-added** entry. Today nothing in the UI distinguishes these, and Merge is a
+bare name dropdown that doesn't say which singer is retained or what history moves.
+
+## Goals
+
+1. Show a **phone/device icon** beside singers who have a real device session
+   (they personally submitted ≥1 request from the singer UI). Duet-partner labels and
+   KJ-added singers get no icon.
+2. Clicking the icon opens a **device-details popup** with whatever we captured
+   (parsed browser/OS/device + raw User-Agent, phone on file, first submitted,
+   request count, source types).
+3. Replace the inline Merge dropdown with a proper **modal**: searchable/scrollable
+   singer list, clear "who is retained" preview, a real-device warning + Swap
+   direction, and an explicit confirm.
+
+## Data model
+
+- **New column** `sing_requests.user_agent TEXT` (additive migration in
+  `SingStore.init_schema`, mirrors `sing_push_subscriptions.user_agent`).
+- `SingStore.create_request(..., user_agent=None)` stores it.
+- `sing.py` `/submit` and `/requests/<id>/change` pass
+  `request.headers.get("User-Agent", "")[:500]`.
+- Client Hints (Sec-CH-UA-Model) deferred — the singer PWA is usually a single
+  visit, so `Accept-CH` wouldn't be honoured on the first load. UA parsing +
+  raw string is enough for v1.
+
+## Backend — session enrichment
+
+- New `ua_parse.py`: pure `parse_user_agent(ua) -> {browser, os, device, is_mobile}`
+  (heuristic, best-effort, always keeps raw). Unit-tested.
+- `SingStore.get_requests_for_entries(entry_ids, night_started)` → linked requests
+  (id, linked_entry_id, singer_name, phone, user_agent, created_at, source_type,
+  additional_singers), night-scoped (mirrors the SMS phone lookup guard).
+- `routes._add_singer_session_info(singer_stats, app)` decorator, called alongside
+  `_add_last_sang_to_singer_stats` in `get_rotation` and `_singer_action_response`.
+  Per singer it attaches:
+  ```
+  session: {
+    origin: 'singer_ui' | 'duet_partner' | 'kj_added',
+    has_device: bool,           # true only for singer_ui
+    phone: str,                 # from newest own request (may be '')
+    device: {browser, os, device, is_mobile, raw},  # newest own request UA
+    request_count: int,
+    first_request_at: str|null,
+    sources: [str, ...],        # distinct source_types
+  }
+  ```
+  Classification: match the singer's rotation-entry ids to linked requests.
+  - own request whose `singer_name` == singer name → `singer_ui` (has_device).
+  - else appears in a linked request's `additional_singers` → `duet_partner`.
+  - else no linked request → `kj_added`.
+
+## Frontend
+
+- `buildSingerRow`: when `session.has_device`, insert a 📱 icon button after the name
+  → `openSingerDeviceModal(singer)`.
+- `openSingerDeviceModal`: reuses the songs-modal backdrop pattern; shows parsed
+  device, phone, first-seen, request count, sources, and raw UA (monospace).
+- Replace `showMergeDropdown` → `openMergeModal(singer)`:
+  - search box + scrollable list of all other singers (self excluded); each option
+    shows name + 📱 badge + sung/queued counts; done singers de-emphasised.
+  - selecting a target reveals a **confirmation panel**: keeper badge on the retained
+    name, combined-history sentence, real-device warning when merging a phone-linked
+    singer into a non-linked one, **Swap direction**, and Confirm.
+- CSS for icon, device modal, merge modal (dark theme, existing tokens).
+
+## Tests
+
+- `ua_parse` unit tests (iOS Safari, Android Chrome w/ model, desktop, junk).
+- Migration idempotency + UA persisted on submit (endpoint test).
+- Session classification: singer_ui / duet_partner / kj_added on a seeded rotation.
+
+## Docs
+
+- `docs/ARCHITECTURE.md` (session provenance flow), `kj-controller/docs/CHANGELOG.md`,
+  version bump.
+```

kj-controller/docs/CHANGELOG.md

@@ -4,6 +4,15 @@ Dated entries, newest first. Each entry notes any required deploy steps.
 
 ---
 
+## 2026-08-13 - Singer session provenance + smarter Merge (v0.91.0)
+
+**Deploy:** backend (`sing_store.py`, `sing.py`, `routes.py`, new `ua_parse.py`) → **requires `systemctl restart kj-controller`** (interrupts playback — deploy between songs). Also frontend (`app.js`, `style.css`). **DB migration:** additive `sing_requests.user_agent` column (auto-applied on boot; safe to roll back — the column is simply ignored).
+
+- **Why:** the "two Chailas" incident — the KJ Singers list showed the same person twice because one entry was her own self-registered singer ("Chaila R", from her phone) and the other was a **duet-partner label** ("Chaila") that another singer typed into their own submission. Nothing in the UI distinguished a real device session from a duet label or a KJ-hand-added entry, and Merge didn't say which singer was kept.
+- **Device capture:** every singer-UI `/submit` and `/change` now records the submitting device's `User-Agent` on `sing_requests`. A small heuristic parser (`ua_parse.py`) turns it into a friendly "iPhone · Safari · iOS 17.4" summary (Android UAs also expose the model, e.g. SM-S911B).
+- **Provenance on the Singers list:** each singer now carries a `session` block classifying them as `singer_ui` (own device — gets a 📱 icon), `duet_partner` (credited in someone else's request), or `kj_added` (typed in by the KJ). Only real device sessions show the 📱 icon; clicking it opens a **device-details popup** (parsed device, phone on file, first submitted, song count, sources, raw User-Agent).
+- **Merge redesign:** the inline dropdown is now a **modal** with a searchable/scrollable singer list (real-device sessions sorted first, each showing device/sung/queued badges). Selecting a partner opens a **confirm step** that spells out exactly which name is KEPT vs REMOVED, the combined sung/queued totals, and a note that a phone-linked keeper inherits the other's history. When you'd merge a phone-linked singer *away* into a non-linked one, it warns and offers **Swap** — defaulting the keeper to the device-linked singer (the KJ's stated preference).
+
 ## 2026-07-17 - Auto Order — fair automatic rotation reordering (v0.90.0)
 
 **Deploy:** backend (`routes.py`, `rotation.py`, `rotation_store.py`, `sing_store.py`, `sing.py`, new `auto_order.py`) → **requires `systemctl restart kj-controller`** (interrupts playback — deploy between songs). Also frontend (`app.js`, `index.html`). No DB migration (reuses the `rotation_meta` key/value table).

kj-controller/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "kj-controller"
-version = "0.90.0"
+version = "0.91.0"
 description = "Web-based karaoke show management with mpv + VLC playback"
 requires-python = ">=3.11"
 

kj-controller/routes.py

@@ -2909,12 +2909,97 @@ def _add_last_sang_to_singer_stats(singer_stats, rotation):
         singer["last_sang_minutes"] = mins
 
 
+def _add_singer_session_info(singer_stats, app=None):
+    """Attach ``session`` provenance to each singer stat dict.
+
+    Answers "how did this singer come to exist?" so the KJ can tell a real
+    singer-UI device session apart from a duet-partner label or a KJ-hand-added
+    entry — and pick the right keeper when merging. Best-effort: any failure
+    leaves ``session`` absent rather than breaking the rotation poll.
+
+    Classification per singer (matching their rotation-entry ids to linked
+    ``sing_requests``, night-scoped):
+      - ``singer_ui``    — owns >=1 linked request whose ``singer_name`` is theirs
+                           -> ``has_device`` True; carries device/phone details.
+      - ``duet_partner`` — only appears in another request's ``additional_singers``.
+      - ``kj_added``     — has entries but no linked request (KJ typed them in).
+    """
+    from ua_parse import parse_user_agent
+
+    app = app or current_app._get_current_object()
+    sing_store = getattr(app, "sing_store", None)
+    if sing_store is None:
+        return
+    try:
+        night_started = sing_store.get_night_started_at()
+        all_ids = []
+        for s in singer_stats:
+            for e in (s.get("entries") or []):
+                if e.get("id") is not None:
+                    all_ids.append(e["id"])
+        reqs = sing_store.get_requests_for_entries(all_ids, night_started)
+    except Exception:
+        return
+
+    # entry_id -> linked requests (query already returns newest id first).
+    by_entry = {}
+    for r in reqs:
+        by_entry.setdefault(r["linked_entry_id"], []).append(r)
+
+    for singer in singer_stats:
+        try:
+            name_lower = singer["name"].strip().lower()
+            own = []
+            is_duet_partner = False
+            for e in (singer.get("entries") or []):
+                for r in by_entry.get(e.get("id"), []):
+                    if (r.get("singer_name") or "").strip().lower() == name_lower:
+                        own.append(r)
+                    else:
+                        for p in (r.get("additional_singers") or []):
+                            if isinstance(p, dict) and (p.get("name") or "").strip().lower() == name_lower:
+                                is_duet_partner = True
+
+            if own:
+                own.sort(key=lambda r: r.get("id") or 0, reverse=True)
+                newest = own[0]
+                # Prefer the newest non-empty phone (singers sometimes drop the
+                # number on later submits after giving it the first time).
+                phone = ""
+                for r in own:
+                    if (r.get("phone") or "").strip():
+                        phone = r["phone"].strip()
+                        break
+                singer["session"] = {
+                    "origin": "singer_ui",
+                    "has_device": True,
+                    "phone": phone,
+                    "device": parse_user_agent(newest.get("user_agent")),
+                    "request_count": len(own),
+                    "first_request_at": min(r.get("created_at") or "" for r in own) or None,
+                    "sources": sorted({r.get("source_type") for r in own if r.get("source_type")}),
+                }
+            else:
+                singer["session"] = {
+                    "origin": "duet_partner" if is_duet_partner else "kj_added",
+                    "has_device": False,
+                    "phone": "",
+                    "device": parse_user_agent(""),
+                    "request_count": 0,
+                    "first_request_at": None,
+                    "sources": [],
+                }
+        except Exception:
+            continue
+
+
 def _singer_action_response(rotation):
     """Build standard response for singer action routes."""
     entries = rotation.get_rotation()
     _decorate_rotation_entries(entries, rotation)
     singer_stats = rotation.get_singer_stats()
     _add_last_sang_to_singer_stats(singer_stats, rotation)
+    _add_singer_session_info(singer_stats)
     return jsonify({"success": True, "entries": entries, "singer_stats": singer_stats})
 
 
@@ -3148,6 +3233,7 @@ def get_rotation():
         _decorate_rotation_entries(entries, rotation)
         singer_stats = rotation.get_singer_stats()
         _add_last_sang_to_singer_stats(singer_stats, rotation)
+        _add_singer_session_info(singer_stats)
         return jsonify({
             "entries": entries,
             "singer_stats": singer_stats,

kj-controller/sing.py

@@ -588,6 +588,7 @@ def submit():
         source_meta=source_meta,
         notes=notes,
         additional_singers=additional,
+        user_agent=request.headers.get("User-Agent", "")[:500],
     )
 
     auto_approved = False
@@ -1020,6 +1021,7 @@ def change_request(req_id):
         source_type=source_type, source_ref=source_ref, source_meta=source_meta,
         token=req["token"], additional_singers=req.get("additional_singers"),
         supersedes_request_id=req_id,
+        user_agent=request.headers.get("User-Agent", "")[:500],
     )
     return jsonify({"success": True, "request": {
         **_public_request_view(new_req), "edit_token": new_req.get("edit_token")}})