Skip to content

Latest commit

 

History

History
331 lines (284 loc) · 18.3 KB

File metadata and controls

331 lines (284 loc) · 18.3 KB

Interface languages

vutuv's interface ships in English (en), German (de), French (fr) and Italian (it). This document is about that interface: the chrome, the flashes, the error pages and the transactional mail. What members write is a different subsystem with its own rules — see translations.md.

Adding a language touches Gettext catalogs, two email template trees, a CLDR backend and a handful of places that still spell the locales out by hand. The checklist at the bottom lists them; the sections above say why.

Where a request gets its language

VutuvWeb.Plug.Locale resolves it once per request, in this order:

  1. the signed-in member's users.locale, if set;
  2. otherwise the first Accept-Language entry whose base subtag the installation serves (de-AT counts as de);
  3. otherwise en.

The answer goes into Gettext for the request process and into the session, because a LiveView runs in a process the plug never touched and would otherwise render English chrome under a German page (VutuvWeb.LiveLocale, VutuvWeb.Live.InitAssigns). The same plug resolves the two other "where the reader is" questions, the date shape (Vutuv.DateRegions) and the time zone (Vutuv.ViewerClock).

It runs on the :browser pipeline, so the machine documents (robots.txt, llms.txt, the sitemaps) answer every visitor the same bytes. /site.webmanifest is the exception: its long-press shortcut labels are read by a person, so VutuvWeb.PageController arms this plug for that action alone and the answer carries Vary: accept-language. Arm it for another such document the same way, per action — pipeline-wide it would make every cached machine document vary by a header none of them reads today.

Agent formats ignore all of it. VutuvWeb.AgentDocs renders .md/.txt/ .json/.xml in English unless the URL asks otherwise with ?lang=, so one URL always answers with the same bytes whether or not anybody is signed in. ?lang= does not work on HTML pages: a browser gets what it asked for in its header.

The locale list, and where it lives

config :vutuv, VutuvWeb.Endpoint,
  locales: ~w(en de fr it)

in config/config.exs, and only there. config/prod.exs used to repeat it, which was not merely redundant: config.exs imports the env file last and deep merges onto the same Endpoint keyword list, so the copy changed nothing while it agreed — and would have served production fewer languages than the suite ever tested the moment it fell behind. It is also the one place Vutuv.SiteLocalesChokepointTest cannot see, because that test greps lib/**. Note that the key sits inside the endpoint block rather than as a top-level :vutuv key. Application.get_env(:vutuv, :locales, default) compiles, runs, answers the default, and is wrong — silently. It cost a day when the pre-translation sweep read it that way (#1570) and it is still wrong in lib/vutuv/jobs/job_posting.ex, where the hardcoded default happens to match the config. Read it like everyone else does:

{:ok, config} = Application.fetch_env(:vutuv, VutuvWeb.Endpoint)
config[:locales]

Five readers then follow the config on their own: Plug.Locale, NodeInfo's langs (Vutuv.NodeInfo), the Mastodon API's /api/v2/instance, Vutuv.Translations (see below), and page_locale_render_test.exs, which loops the configured locales so a new language is covered by public-page render tests the moment it is added.

Beware the second source of truth beside it: Gettext.known_locales/1 reports whatever PO directories exist under priv/gettext. AgentDocs keys off that, everything else off :locales. A half-added language therefore shows up in the agent formats before it shows up in the UI.

Surface 1: the Gettext catalogs

Everything on screen resolves against priv/gettext/<locale>/LC_MESSAGES/: default.po for the interface and errors.po for changeset messages. Flash messages are ordinary gettext/1 calls, so they are covered by default.po and need nothing of their own. So is the text JavaScript puts on screen: the hooks read their wording from data-label-* attributes the server rendered, and no user-facing string is spelled out in assets/js.

A missing msgstr falls back to the msgid, which is English. An incomplete catalog is therefore a cosmetic problem and never an outage — the opposite of the email templates below.

The extract/merge traps

mix gettext.extract --merge is the most dangerous command in this subsystem, in three separate ways:

  • It fuzzy-fills. A brand-new msgid does not arrive empty; it arrives holding the translation of whatever existing string the tool thought looked similar. Real examples from one run: "Now" became "Nein" (no), "Not yours yet." became "Noch keine Reposts.". Each is flagged fuzzy and nothing fails the build, so confident nonsense ships while every English test stays green. After every merge, read git diff priv/gettext and treat every fuzzy entry as untranslated.
  • Grep for ", fuzzy", not "#, fuzzy". The flag line reads #, elixir-autogen, elixir-format, fuzzy, so the obvious pattern matches nothing and reports a clean file.
  • An empty-looking msgstr "" followed by indented lines is a multi-line translation, not a missing one. Writing into it appends a second copy that gettext concatenates into one doubled sentence. That is how the logged-out landing page came to 500 for every German visitor while curl / looked fine.

Every translated catalog is fuzzy-free, so a grep -c ", fuzzy" priv/gettext/{de,fr,it}/LC_MESSAGES/default.po answering anything but 0 comes from your own merge rather than from old debt. The en catalog is different: its entries carry no translation at all (English falls back to the msgid), so the flags left on 310 of them mean nothing.

Complete is a separate claim, and only French is. Measured on 2026-09-20: fr has 0 untranslated entries, de 2 and it 20 — the Italian ones are almost all of the post reach-analysis page (templates/post/analytics.html.heex), so a whole recent feature renders in English for an Italian reader today. The gate below does not catch them because those surfaces are outside its @covered list, which is deliberate.

Count an untranslated entry by joining the field, never by grepping the line. grep -c '^msgstr ""' answers 333 for de and 694 for it against the real 2 and 20, because a multi-line translation opens with exactly that line and carries its text on the indented lines below — the same shape that doubled the German consent sentence above. The line count is still sound as a before and after comparison of one merge (unchanged means unchanged, which is how the one silently-emptied msgid gets caught), but never read it as a total.

test/vutuv_web/translated_catalogs_test.exs fails the build on either half of that — a missing translation or a fuzzy flag — for every translated locale, but only over the surfaces named in its @covered list. Everything outside that list is still yours to notice: the catalogs carry a long tail of untranslated legacy prose, and gating all of it would only get the test deleted.

And one trap grep cannot see at all: a msgid is a key, not a phrase. gettext("Following") is translated "Folge ich" (I follow), written for a member's own list; reusing it under an organization's navigation made the page claim the reader follows those accounts. When the same English word is spoken by a different voice, give it its own msgid.

Surface 2: emails

Mail does not go through Gettext for its body. Every message has two body files per locale:

lib/vutuv_web/templates/email/<name>_<locale>.text.eex the text/plain part
lib/vutuv_web/templates/email_body/<name>_<locale>.html.heex the text/html alternative

Subjects are Gettext, rendered inside Gettext.with_locale/3 for the recipient. test/vutuv/notifications/email_html_drift_test.exs fails the build if one half of a pair is missing.

Three things are not templates and are easy to miss:

  • _signature_<locale>.text.eex on the text side has no HTML twin. The HTML signature is a pair of function clauses in VutuvWeb.EmailComponents (signature_line1/1, signature_line2/1), so a new language is an Elixir edit there.
  • _footer.text.eex is shared by every locale and is English prose. It names the operator and is the same in every message.
  • The salutation every body opens with is email_greeting/1 in VutuvWeb.UserHelpers, one clause per locale, falling through to "Hi". German greets with surname and honorific because the UI says Sie; each language decides that for itself. email_unsubscribe_note/1 in EmailComponents is a second such clause list.
  • Four mails are deliberately German-only, because they go to the operator and never to a member: account_deleted_notice, ad_booking, daily_report, organization_operator_notice. The two admin moderation mails are not in that group; they follow the admin's own locale.

Ordering matters, and getting it wrong is an outage. Emailer.get_locale/1 falls back to en only for locales that are not in :locales. The moment a locale is in the config, VutuvWeb.EmailText.render/2 looks for <name>_<locale>.text and raises on the missing atom. Config and templates ship in the same deploy, always.

EmailText builds one function per file at compile time, so editing a template recompiles the module but adding one would not, and CI caches _build. __mix_recompile__?/0 closes that hole (#1086); still, verify a new template with mix compile on an already-built tree, never only after --force.

Surface 3: things that are formatted, not translated

A language is not only words. These read the locale and would quietly produce English output for a third one:

  • VutuvWeb.UI's number_separators/0 answers both separators as one pair, because they invert together. It is a three-way case and French is why: it takes German's decimal comma but groups thousands with a narrow no-break space (U+202F), so it belongs to neither branch of the de it against everything-else split this used to be. The space must be the narrow no-break one — a plain space lets a browser end a line inside 60 023. A number formatted with the wrong rules is misread, not untidy.
  • The decimal separator alone goes through VutuvWeb.UI.decimal_separator/0. Three call sites format a figure themselves and each used to carry its own copy of the rule (views/qualification_html.ex and views/job_reference_html.ex, which hold two identical copies of a file_size_label/1 that predates UI.file_size/1, and views/admin/newsletter_html.ex's click rate). A half-right copy reads exactly like a whole one.
  • VutuvWeb.UI's relative day labels — "Yesterday"/"Gestern" is an if, not a Gettext string.
  • Vutuv.Countries carries 249 ISO 3166-1 codes with one short name per interface language, plus the four region presets (EU, EMEA, MENA, APAC). A new language is a new column in both tables and one clause in localized/4.
  • Vutuv.Search's @field_ops holds the query operators (tag:, ort:, city: …) in every interface language, because the search help page prints the examples in the reader's language and an example that does not parse is worse than none.
  • Vutuv.DateRegions already maps the common language subtags to a date shape, Italian and Spanish included, so it usually needs nothing.
  • Vutuv.Cldr compiles in only the configured locales. ex_cldr bundles en and und and downloads the rest at compile time, which once let a GitHub incident block a production build. Compile once with network access and commit the generated priv/cldr/locales/<locale>.json, or Vutuv.CldrLocaleBundleTest fails the build (#1545).

What a new locale costs at runtime

With TRANSLATE_POSTS=true, Vutuv.Translations pre-translates every local post into every locale the installation serves, so adding one queues a machine translation per post through Ollama. Nothing to write, but the config line has a GPU bill attached.

What is deliberately not translated

  • The legal pages. Impressum, Datenschutzerklärung and Nutzungsbedingungen are database rows edited at /admin/legal, one body per slug with no locale column. They stay in the language the operator wrote them.
  • Operator mail, as above.
  • Member content. Posts are translated on demand by a model, never by us; that is translations.md.

Adding a language

  1. config/config.exs: add the code to locales: in the endpoint block. That is the only file — see "The locale list" above for why config/prod.exs no longer carries a copy.

  2. lib/vutuv/cldr.ex: add it to locales:, compile once online, commit priv/cldr/locales/<locale>.json.

  3. mix gettext.merge priv/gettext --locale <locale>, then translate default.po and errors.po and clear every fuzzy flag. test/vutuv_web/translated_catalogs_test.exs needs nothing: its locale list is derived from the config, because a list that drives iteration and falls behind does not go red, it quietly stops checking a whole catalog.

  4. Both email bodies for every member-facing mail, plus _signature_<locale>.text.eex, the two EmailComponents clauses and an email_greeting/1 clause.

  5. priv/help/{markdown,mastodon}_<locale>.md — read with File.read! at compile time, so a missing file breaks the build, not a request.

  6. Widen what is genuinely per-language. Most of the old list here is gone: Vutuv.Languages.site_locales/0 is now the one reader of the locale list and Vutuv.SiteLocalesChokepointTest fails the build on a second copy, so the language picker, the newsletter groups and the job form follow the config by themselves. What is left is per-language content and per-language rules: number_separators/0 (see above), the relative day labels (relative_yesterday/1), @language_names in VutuvWeb.AgentDocs (the endonym), Vutuv.Search's query operators — note the search help page prints its examples as gettext msgids, so whatever the catalog tells a reader to type has to be a key in @field_ops — and Vutuv.Countries, one column per locale in both tables plus a localized/5 and a normalize_locale/1 clause. Two files are allowed to hold the locale list as a compile-time literal because each reads a committed file while compiling, and the chokepoint test checks them instead: lib/vutuv/cldr.ex and lib/vutuv_web/controllers/help_controller.ex.

    The shape that hides from the grep is a lookup with a default, and French fell through two of them. Vutuv.Newsletters.email_locale/1 had a clause for "de" and one for "it", so every French member received the English newsletter while newsletter_fr.text.eex sat unreachable; the /settings/preferences interface-language <select> spelled three <option>s, so a member whose browser sends de could never pin French at all. Neither raised, neither logged, and the chokepoint test cannot see either — it keys on the assignment of a locale list, and a case answering a default for an unknown locale is indistinguishable from one answering a default for a locale nobody serves. Both read site_locales/0 now (through Vutuv.Languages.render_locale/1, which is "a locale we hold templates for", the rendering half of user_locale/1). The guard that does catch this shape is a test that iterates site_locales/0 and asserts a distinct, non-fallback answer per locale — open_graph_test.exs with Map.fetch!, the per-locale newsletter bodies in newsletters_test.exs, the help pages in help_controller_test.exs. Write one beside any new per-locale table.

    The country column is worth generating rather than typing: ex_cldr already carries the names for a locale it compiles in (Vutuv.Cldr.Territory.from_territory_code/2). One normalization is not optional — CLDR writes the typographic apostrophe (U+2019) and Countries.fold/1 folds accents but not apostrophes, so a name carrying it is unreachable by anybody typing '.

  7. Plural rules are not the same in every locale, and French is the one that differs here. en, de and it send 0 to the plural form; French sends it to the singular (plural=(n>1)), so msgstr[0] has to read correctly for "0 jour" as well as "1 jour". Wording copied from the German singular is wrong for zero. Gettext.Plural.plural("fr", 0) answers 0; check a new locale there rather than assuming.

    The concrete trap that follows from it: English spells the 1 out in its singular msgid, because English only ever shows that form for n == 1 (msgid "1 connection" beside msgid_plural "%{formatted} connections"). Translate that literally and French renders "1 relation" for zero connections, which the /:slug/connections heading really does — seven entries had it. Put the placeholder in the singular instead ("%{formatted} relation"); the call site passes its binding for both forms, and ngettext auto-binds %{count}. To find them: msgid_plural has a placeholder, msgstr[0] has a digit and none.

  8. A locale added is a locale that stops being an example of an unknown one. Six tests used "fr" that way and adding French turned three of them red, leaving the rest quietly asserting the wrong thing. Two shapes: a stand-in for a language this installation does not serve — take the unassigned "zz", never the next language somebody is going to add — and a stand-in for a curated language that is not a site locale, where "zz" will not do because it is not in Languages.codes(): derive it, hd(Languages.codes() -- Languages.site_locales()). A third shape is a test that spells the served list inside an expected string (llms.txt's ?lang= sentence); anchor such an assertion on the part that carries its point instead, since deriving the list there only re-implements the line under test.

  9. Verify in a browser, not only in the suite: the Swoosh test adapter never renders a real mail, and Phoenix.ConnTest defaults to English. Walk login, posting and following with Accept-Language: <locale>, trigger a flash and a 404, and read both parts of the login PIN mail in /sent_emails.