Skip to content

@lde/search: preserve all label language variants in text projection + resolution #591

Description

@ddeboer

Problem

A label-source label text field declares a fixed locale set (locales: [nl, en]), and projection emits only those declared locales. But label data is routinely untagged or tagged in another language:

foaf:name "Bibliothèque"@fr
dct:title "Thésaurus"          # untagged

Every such value is silently dropped, so the facet bucket / reference renders as a bare IRI instead of a label.

The Dataset Register works around this by re-tagging each subject into explicit @nl / @en with a nl → en → first-of-any fallback. That is lossy: it discards the real language, and a subject with both @fr and @de keeps only the first-seen.

Desired behaviour

Preserve every language variant, so [LanguageString!] returns them all, best-first per Accept-Language, and the consumer picks. Once projection keeps all languages, the DR re-tagging workaround disappears.

Two spots hardcode the declared locales and both must change:

  1. ProjectionapplyText (@lde/search) iterates field.locales only.
  2. Resolution — the Typesense engine’s localizedValue reads only the field.locales display fields; labelValue additionally falls back to the bare label for untagged (und). So even if the collection stored label_fr, the engine would not read it today.

Two possible shapes

  • (a) open / dynamic locale set — project label_<lang> for whatever languages are present and resolve them back into the full LanguageString[]. Keeps per-language search tokenisation (matters for label-collection typeahead, Search-within-a-facet (facet_query typeahead) for high-cardinality facets beyond the maxFacetValues cap #533). (Leaning this way for label collections that are searched, not only displayed.)
  • (b) language-map field — store the label as one { lang: value } map and pass it through at resolution. Simplest; loses per-language search tokens, acceptable while a label-source label is display-only.

Decide per use: a display-only reference label can take (b); a searchable label collection needs (a).

Scope

  • @lde/search: applyText projects all present languages (not just declared ones) for the chosen shape.
  • @lde/search-typesense: localizedValue / labelValue resolve the full variant set back into LanguageString[], best-first per Accept-Language.
  • Remove the DR-side re-tagging workaround once projection preserves languages.

Relates

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Fields

    No fields configured for Task.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions