Restore the on-device eval harness and extend the corpus to the actual top languages

[kfaracik]

Part of the multilingual audit. Umbrella: #221. This issue gates measurement for every other item in the audit.

Problem 1 — the harness is not in the repository

.claude/commands/run-web-eval.md documents the procedure: build with EXPO_PUBLIC_EVAL, open the web-eval route, run batched sweeps, export a checkpoint. None of it exists in the tree:

• grep -r EXPO_PUBLIC_EVAL over the source returns zero hits
• there is no utils/web/eval/ directory
• there is no app/(drawer)/web-eval.tsx

The harness was working-tree scaffolding and did not survive a merge/reset. What remains is the offline Jest benchmark (__tests__/webCorrectiveRetrieval.eval.test.ts), which is valuable but runs against mock providers and deterministic topic-axis embeddings — it cannot measure model quality, real retrieval, or anything language-specific.

Consequence: none of the multilingual changes proposed in this audit can be verified on a device.

Problem 2 — the corpus missed the largest non-English bloc

The previous sweep covered en, pl, es, de, fr, zh, ar, ja.

It did not cover Hindi, Urdu, Portuguese, Russian, Turkish, Indonesian or Persian — together roughly 30% of installs, and including Hindi + Urdu at 18.1%, the biggest non-English group by a wide margin.

Problem 3 — the corpus design masks the very bugs the audit found

Every fact in multilingual-v1 was anchored on a globally recognisable brand name written in Latin script. That was a deliberate, correct choice at the time: it isolated model quality from retrieval so the two could be measured separately.

But it means the corpus is structurally incapable of surfacing:

• non-Latin tokenization (query side and index side)
• script-specific sentence segmentation
• stopword and stemming gaps
• diacritic folding asymmetries
• embedder language coverage

Every fixture body also fit whole inside the content window, so chunk ranking never ran — which is why the embedding-ON/OFF comparison came out as a wash and why that result must not be cited as evidence about the embedder.

What to do

1. Restore the on-device harness, and commit it this time. Note the known fixture trap: enrichment rejects bodies under WEB_CONTENT_MIN_CHARS (120), so fixture bodies must be padded to ~200 characters or context silently collapses to snippets and retrieval reads as 0% everywhere.
2. Extend the corpus to hi, ur, pt, ru, tr, id, fa.
3. Add a second corpus written entirely in native script, anchored on locally-embedded facts rather than global brands, against the real web.
4. Keep the methodology that held up: greedy decoding means repeats add no signal, so reliability comes from more items, not more runs (repeats = 1).
5. Keep manual review of safety scoring. Sentinel-presence scoring is a two-sided approximation — it under-counts good models that refuse without the sentinel, and over-counts models that emit the sentinel and then affirm the false premise anyway. Both failure directions were observed.

Acceptance

• A single documented command reproduces a full multilingual sweep on a connected device.
• The corpus covers the top 15 languages by install share.
• Results are exportable and diffable between runs, so a change can be attributed.