Skip to content

i18n(privacy): propagate self-hosted image URLs into all 18 translations (+ fix a latent 54-file deletion) - #1910

Merged
zksquirrel merged 3 commits into
ZecHub:mainfrom
bloxster:i18n/image-migration-parity
Jul 29, 2026
Merged

i18n(privacy): propagate self-hosted image URLs into all 18 translations (+ fix a latent 54-file deletion)#1910
zksquirrel merged 3 commits into
ZecHub:mainfrom
bloxster:i18n/image-migration-parity

Conversation

@bloxster

@bloxster bloxster commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

The self-hosted-image migration (#1888, commit 3ec05852) rewrote image references in site/** from third-party hosts to /content-images/…, but never touched translations/**. Since the frontend prefers a curated translation over the English source and does no runtime image rewriting, every non-English reader was still hotlinking the original hosts — so the privacy leak the migration set out to close stayed live across the majority of the site's content surface.

This also fixes a latent data-loss bug in the curated/translation bookkeeping (commit 1), which would have deleted 54 good translations on the next sync run.

1 — f2215e8c restore curated/translation parity after the case-only renames

Three source pages were renamed for capitalization on main:

Old New
Accept_Payments_AS_A_Merchant.md Accept_Payments_As_A_Merchant.md
Journalist_privacy_setup.md Journalist_Privacy_Setup.md
Private_Community_treasury.md Private_Community_Treasury.md

The source moved, but translation/curated-pages.txt, the 54 translated files (3 pages × 18 locales) and their 54 manifest entries all kept the old names. On a case-sensitive filesystem that leaves curated-pages.txt pointing at three paths that do not exist, breaking the curated ⊆ site invariant, and the staleness detector reports all three as orphans in every locale.

That misclassification is destructive rather than cosmetic. The sync agent's orphan path rmSyncs the translation file and drops the manifest entry, and the de-curation stage then removes the page from the curated list once no locale references it. An unscoped sync run would have deleted 54 good translations and silently cut coverage from 181 to 178 pages, while the new capitalized names never entered curation at all.

Renames the 54 translated files to match, re-keys the 54 manifest entries (values untouched, provenance preserved), fixes the three curated-pages.txt lines.

2 — 9ca89c97 propagate self-hosted image URLs into all 18 translations

Before this change, translated files still referenced:

Host Files
i.ibb.co 540
raw.githubusercontent.com 288
pbs.twimg.com 144
hackmd.io 72
img.youtube.com 72
github.com/ZecHub/zechub/assets 54

Mechanical URL substitution only — no re-translation, no model involved. The old→new pairs are taken from the migration commit itself (1276 pairs across 250 pages), so the mapping is authoritative rather than reconstructed.

Applied: 9458 URL replacements across 1908 files. The diff is +10205/−10205 — byte-for-byte symmetric, no line added or removed in any file. Alt text, captions and link text are untouched (spot-verified across ar, zh, ru, sw and the NLLB/GT locales).

Each file had to pass four post-conditions or it was left alone:

  1. no mapped old URL remains
  2. image-reference count unchanged
  3. line count unchanged
  4. every differing line is reproducible by applying only the mapped substitutions to the original line

Zero gate failures occurred.

Deliberately not propagated: the 74 pairs whose migrated value is /content-images/_unavailable.svg, plus any target with no corresponding file in the frontend's public/content-images/. In those cases the migration could not fetch the image, so the English side now shows a placeholder. Rather than copy a placeholder into 18 locales, those pairs were skipped and the affected pages stay flagged stale, so the decision stays visible instead of being silently stamped fresh.

Note that most of these placeholders reflect a pre-existing loss rather than anything the migration broke: 30 of the 74 replaced relative paths from a Notion export whose asset directory was never committed to this repo, so those images were already 404ing before any of this work. The remaining 44 were hotlinks that failed to fetch. Repairing them is separate work — see the companion PR.

3 — 9e07db3e re-stamp provenance for the image-only pages

The 94 curated pages whose English changed only by image URL are genuinely current again, so their manifest provenance is re-stamped for all 18 locales (1692 entries). Without this the detector keeps reporting them stale and the sync agent would spend ~1700 work items of LLM calls re-translating prose that never changed.

Hashes come from the repo's own translation/lib/normalize-hash.mjs, so they match what the detector and seeder produce.

Four conditions had to hold before any entry was stamped:

  • A no old (pre-migration) URL from that page's mapping survives in the translation
  • B every /content-images/… URL in the translation is one the English page also references (no stray targets)
  • C the entry is not flagged edited (never overwrite a hand-edited page)
  • D the page's English change since the seed was image-URL-only

Zero gate failures. Condition D specifically excluded four pages that were both image-churned and prose-edited (Wallets, Pepper_Sync, Zcash_Shielded_Assets, Raspberry_Pi_4_Full_Node) — stamping those would have marked genuinely stale prose as fresh and hidden it from the detector permanently.

Net effect

Detector, per locale:

before after
orphan 3 0
stale 108 17

The remaining 17 are exactly the work that still needs doing — 6 pages with genuine prose changes (normal sync), and 11 pages held pending the upstream image repair.

translation/check-invariants.mjsManifest invariants hold: 181 curated pages, 18 locales.

Reviewer note

translation/sync-state.json shows ~3384 insertions/deletions because the file was re-serialised with sorted keys after the rename re-keying. The entry count is unchanged (3258) and no data was lost — only key order and the re-stamped src/src_commit values differ.

Provenance

The analysis behind this was adversarially reviewed by two independent models before any file was touched; both confirmed the 102-image-only/6-prose split under stricter methods than the original, and both rejected an earlier plan of mine to strip images from the canonical hash (it would have made all future image-only drift invisible). The 54-translation deletion trap in commit 1 was surfaced by that review, not by the original analysis.

🤖 Generated with Claude Code

bloxster and others added 3 commits July 28, 2026 13:02
…names

Three source pages were renamed for capitalization on main:

  Accept_Payments_AS_A_Merchant.md -> Accept_Payments_As_A_Merchant.md
  Journalist_privacy_setup.md      -> Journalist_Privacy_Setup.md
  Private_Community_treasury.md    -> Private_Community_Treasury.md

The source moved, but `translation/curated-pages.txt`, the 54 translated
files (3 pages x 18 locales) and their 54 manifest entries all kept the old
names. On a case-sensitive filesystem that leaves `curated-pages.txt`
pointing at three paths that do not exist, so the `curated subset of site`
invariant is broken, and the staleness detector reports the three pages as
ORPHANS in every locale.

That misclassification is destructive, not cosmetic: the sync agent's orphan
path (`processOrphan`) rmSync's the translation file and drops the manifest
entry, and the de-curation stage then removes the page from the curated list
once no locale references it. An unscoped sync run would therefore have
deleted 54 good translations and silently cut coverage from 181 to 178 pages,
while the new capitalized names never entered curation at all.

This renames the 54 translated files to match, re-keys the 54 manifest
entries (values untouched, so provenance is preserved), and fixes the three
curated-pages.txt lines.

Verified after the change:
  - translation/check-invariants.mjs: "Manifest invariants hold: 181 curated
    pages, 18 locales"
  - detect-staleness.mjs: orphan=0 in all 18 locales (was 3 each)
  - the 3 pages now register as ordinary stale entries; their English diff
    since the manifest was seeded is image-URL-only (10 URLs each, from the
    self-hosted-images migration), so no re-translation is required

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The self-hosting migration (3ec0585, PR ZecHub#1888) rewrote image references in
`site/**` from third-party hosts to `/content-images/...`, but never touched
`translations/**`. Because the frontend prefers a curated translation over the
English source and performs no runtime image rewriting, every non-English
reader was still hotlinking the original hosts — i.e. the privacy leak the
migration set out to close remained live across the majority of the site's
content surface.

Before this change, translated files still referenced:

    i.ibb.co                            540 files
    raw.githubusercontent.com           288 files
    pbs.twimg.com                       144 files
    hackmd.io                            72 files
    img.youtube.com                      72 files
    github.com/ZecHub/zechub/assets      54 files

Mechanical URL substitution only — no re-translation, no model involved. The
old->new pairs are taken from the migration commit itself (1276 pairs across
250 pages), so the mapping is authoritative rather than reconstructed.

Applied: 9458 URL replacements across 1908 files. The diff is +10205/-10205 —
byte-for-byte symmetric, no lines added or removed in any file. Alt text,
captions and link text are untouched (verified across ar/zh/ru/sw and the
NLLB/GT locales).

Each file had to pass four post-conditions or it was left alone:
  1. no mapped old URL remains
  2. image-reference count unchanged
  3. line count unchanged
  4. every differing line is reproducible by applying only the mapped
     substitutions to the original line
Zero gate failures occurred.

Deliberately NOT propagated: the 74 pairs whose migrated value is
`/content-images/_unavailable.svg`, plus any target with no corresponding file
in the frontend's `public/content-images/`. For those the migration lost the
image, so English is privacy-clean but broken while the translation still
holds a working URL. Copying the placeholder would degrade 18 locales to match
a regression. Those 19 pages keep their working URLs and stay flagged stale, so
they remain visible until the English side is fixed upstream.

Residual after this change: 3 pages x 18 locales (Creators_and_Tips,
Visualizing_the_Zcash_Network, ZK_Shielded_Asset_Platforms) still reference
pbs.twimg.com / raw.githubusercontent.com. Two of the three are at parity with
English, which still hotlinks the same asset; all three trace back to
`_unavailable.svg` losses and are fixed by re-migrating those images upstream.

Verified: translation/check-invariants.mjs -> "Manifest invariants hold: 181
curated pages, 18 locales".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
With the self-hosted image URLs now propagated into the translated trees, the
94 curated pages whose English changed ONLY by image URL are genuinely current
again. This re-stamps their manifest provenance (`src` = normalized hash of the
current English, `src_commit` = fd4274d) for all 18 locales: 1692 entries.

Without this the detector keeps reporting those pages stale and the sync agent
would spend LLM calls re-translating prose that never changed — ~1700 work
items of pure waste.

Hashes are computed with the repo's own translation/lib/normalize-hash.mjs, so
they match what the detector and the seeder produce.

Four conditions had to hold before any entry was stamped:
  A. no old (pre-migration) URL from that page's mapping survives in the
     translation
  B. every /content-images/... URL in the translation is one the English page
     also references (no stray or invented targets)
  C. the entry is not flagged `edited` (never overwrite a hand-edited page)
  D. the page's English change since the seed was image-URL-only
Zero gate failures. Condition D specifically excluded four pages that were both
image-churned AND prose-edited (Wallets, Pepper_Sync, Zcash_Shielded_Assets,
Raspberry_Pi_4_Full_Node) — stamping those would have marked genuinely stale
prose as fresh and hidden it from the detector permanently.

Detector, per locale, before -> after this branch:
  orphan 3 -> 0        (the case-rename fix)
  stale  108 -> 17     (111 after renames reclassified, then -94 stamped)

The remaining 17 are exactly the work that still needs doing:
  6 pages with genuine prose changes -> normal LLM sync
  11 pages held because the migration lost their image (`_unavailable.svg`);
     their translations keep working URLs and stay visible until the English
     side is repaired upstream

Verified: check-invariants.mjs -> "Manifest invariants hold: 181 curated pages,
18 locales".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@bloxster
bloxster force-pushed the i18n/image-migration-parity branch from 9e07db3 to e2cdc1e Compare July 28, 2026 11:30
@bloxster

Copy link
Copy Markdown
Contributor Author

Force-pushed to correct a defect I introduced in the first push: the rewrite script read and wrote in Python text mode, which silently converted 10 translated files from CRLF to LF. Those files' entire contents showed as changed even though only their image URLs differed.

Fixed by redoing those 10 substitutions in binary mode from the pre-change bytes. CRLF file count is back to 28, matching the original tree, and git diff --ignore-all-space against the previous push reports zero differences — so the URL substitutions were always correct, only the line endings were wrong.

The privacy commit's diffstat drops from +10205/-10205 to +9369/-9369 across the same 1908 files. That is now consistent with the 9458 URL replacements (a few lines carry more than one image URL), where the earlier figure was inflated by ~750 line-ending-only changes.

Re-verified after the rebase: check-invariants.mjsManifest invariants hold: 181 curated pages, 18 locales; detector unchanged at stale=17 orphan=0 per locale.

Worth noting for anyone writing similar tooling against this repo: 28 files under translations/ use CRLF, and Python's universal-newline read strips \r before your own diff checks can see it — my four post-condition gates all passed while the damage was happening.

@zksquirrel
zksquirrel merged commit c81b373 into ZecHub:main Jul 29, 2026
1 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants