Add mathspeak localization to Mathquill #324

sclower · 2025-07-13T02:14:29Z

Description

Mathquill's screen reader messages, known internally as "mathspeak," have always been hard-coded to English. We are receiving more requests to make them respect the user's language, and this work shows how we could fulfill these requests.

Some notable changes include:

Factors out messages to individual English and Spanish catalogs. Note: the latter needs human verification for accuracy but appears to be reasonable enough to keep for testing purposes.
Messages are bundled with the build, so no external files need be distributed.
User can set Mathquill's language: e.g. mq.config({ language: 'es' }). At present, only English and Spanish are recognized, but the system is configured to allow for more, including locale-specific variants.
New Mathquill.L10n API allows the user to check for language availability, set, and get the active language, as well as provide a callback to fire when the language changes (so that, for example, related DOM can be updated).
Dynamic language switching: Default aria label and Mathspeak are recomputed when language changes. Note that user-supplied custom aria labels persist.
Per-instance localization: Each MathField can have independent language settings if desired.
All common symbols, blocks, and structural descriptions are localizable.
Custom fraction, root, and exponent shorthand speech is preserved with some subtle improvements introduced.
Tweaked all mixed-case constructs to be separate words, e.g. "StartFraction" is now "Start Fraction" (tests updated accordingly)
Common function names with existing screen reader-friendly replacements (e.g. "sine" for "sin") are localizable. TBD if this is worth keeping or if callers should be expected to provide all auto operators and language-specific alternatives themselves.
Added documentation and tests for new API methods

Open Questions

The Fluent library adds more size to the generated bundle which also inflates anything embedding Mathquill:

Mathquill.js: 439K before, 652K after
Mathquill.min.js: 149K before, 210K after

This is fairly significant, and it might be worth exploring a dedicated make target which allows external callers to pass in an existing FluentJS bundle.

Support currently exists to localize some common function names (see above). For stand-alone Mathquill instantiation where the list of auto operator names is not replaced, I think this will work out well. Life could become much more confusing for callers which include custom keywords which also require localization.
Fluent syntax has proven to be powerful and flexible. Is the system overkill for this project?

Note: Claude Code was used for parts of this work, especially in assisting with the refactoring of English messages into the Fluent catalogs and translation into Spanish. I have reviewed and tweaked any new code which was generated along the way.

Mathquill's screen reader messages, known internally as "mathspeak," have always been hard-coded to English. We are receiving more requests to make them respect the user's language, and this work aims to do just that. Some notable changes include: - Factors out messages to individual English and Spanish catalogs. Note: the latter needs human verification for accuracy but appears to be reasonable enough to keep for testing purposes. - Messages are bundled with the build, so no external files need be distributed. - User can set Mathquill's language: e.g. mq.config({ language: 'es' }). At present, only English and Spanish are recognized, but the system is configured to allow for more, including locale-specific variants. - Dynamic language switching: Default aria label and Mathspeak are recomputed when language changes. Note that user-supplied custom aria labels persist. - Per-instance localization: Each MathField can have independent language settings if desired. - All common symbols, blocks, and structural descriptions are localizable. - Custom fraction, root, and exponent shorthand speech is preserved with some subtle improvements introduced. - Tweaked all mixed-case constructs to be separate words, e.g. "StartFraction" is now "Start Fraction" (tests updated accordingly)

- Added new Mathquill.L10n API and moved the previously global Window functions here - Removed temporary any typings - Updated documentation

src/publicapi.ts

timstallmann · 2025-07-21T12:49:55Z

src/publicapi.ts

        var processor = (optionProcessors as any)[name]; // TODO - validate option processors better
        (currentOptions as any)[name] = processor ? processor(value) : value; // TODO - think about typing better
+
+        // Handle language changes by updating the global language manager


unclear to me if this is needed? we're already doing it in MQ.config. I'm just not clear what the codepaths are here

timstallmann · 2025-07-21T12:51:18Z

src/publicapi.ts

      this.revert = function () {
        ctrlr.removeMouseEventListener();
+        // Clean up language change callback
+        if (ctrlr.unregisterLanguageChange) {


why is this named unregisterLanguageChange?

src/services/fluent-bundle.js

src/services/locale-imports.ts

src/shared_types.d.ts

timstallmann · 2025-07-21T12:55:21Z

src/shared_types.d.ts

 type NodeRef = MQNode | 0;
+
+// Handle timeout types for both browser and Node.js environments
+type TimeoutId = number | NodeJS.Timeout;


is this type addition necessary for the localization stuff?

test/unit/localization-fluent.test.js

timstallmann · 2025-07-22T12:08:10Z

src/locale/es/messages.ftl

+plus = más
+positive = positivo
+minus = menos
+negative = negativo


pretty sure this is also menos in practice. I kind of think we shouldn't be deploying translations that claude generated, especially for math where there's a lot of language patterns which are math-specific. would it be possible to send the english ftl file off to localizers, or just get someone with professional translation experience to review the es translations?

e.g. see https://hinative.com/questions/4805117

timstallmann · 2025-07-22T12:40:25Z

script/generate-locale-imports.js

@@ -0,0 +1,182 @@
+#!/usr/bin/env node


thinking about how we already have a version of this in knox, which is also in typescript. I dunno if it makes sense to try to standardize, mainly just thinking out loud

src/commands/math/commands.ts

src/locale/en/messages.ftl

src/services/localization.ts

This also ensures that nodes whose content doesn't update with language changes (specifically binomials, text, and style blocks) are updated by the controller.

- More mathspeak template optimization - Added more localizable messages for block types and brackets - Now using menos instead of negativo to describe negative numbers in Spanish - Removed duplicate strings where possible

timstallmann · 2025-07-23T13:28:36Z

src/animate.ts

 const animate = (function () {
  // IIFE exists just to hang on to configured rafShim and cancelShim
-  // functions
+  // functions. Both return/accept number tokens for cross-environment compatibility.


can we revert this file to main? I'm having trouble seeing how this change is necessary? but maybe it is?

timstallmann · 2025-07-23T13:30:23Z

src/commands/math.ts

    if (this.controller) autoOps = this.controller.options.autoOperatorNames;
+    const outerThis = this; // Capture outer 'this' for use in callback
    return (
      this.foldChildren<string[]>([], function (speechArray, cmd) {


if this were an arrow function we wouldn't need the outerThis abstraction, would be clearer.

timstallmann · 2025-07-23T13:35:06Z

src/commands/math/advancedSymbols.ts

-);
-LatexCmds.lbrack = bindVanillaSymbol('[', 'left bracket');
-LatexCmds.rbrack = bindVanillaSymbol(']', 'right bracket');
+class LeftBrace extends VanillaSymbol {


is the idea that not all of the symbols in here get mathspeak localizations yet? e.g. \\simeq?

timstallmann · 2025-07-23T13:36:51Z

src/commands/math/commands.ts


+  mathspeak() {
+    const localization = getControllerLocalization(this);
+    this.mathspeakTemplate = [


could this use createMathspeakTemplate?

timstallmann · 2025-07-23T13:37:10Z

src/commands/math/commands.ts

+
      mathspeak(opts?: MathspeakOptions) {
+        const localization = getControllerLocalization(this);
+        this.mathspeakTemplate = [


could this use createMathspeakTemplate?

timstallmann · 2025-07-23T13:37:34Z

src/commands/math/commands.ts

-            'StartNestedFraction,',
-            'NestedOver',
-            ', EndNestedFraction'
+            localization.formatMessage('start-nested-fraction') + ',',


could this use createMathspeakTemplate?

timstallmann · 2025-07-23T13:37:38Z

src/commands/math/commands.ts

          ];
        } else {
-          this.mathspeakTemplate = ['StartFraction,', 'Over', ', EndFraction'];
+          this.mathspeakTemplate = [


could this use createMathspeakTemplate?

timstallmann · 2025-07-23T13:40:07Z

src/commands/math/commands.ts

+        localization.formatMessage('end-cube-root')
+      );
    } else {
      return (


should this also use a mathspeak template or do we want to be directly concatenating the string for some reason?

timstallmann · 2025-07-23T13:40:42Z

src/controller.ts

  readonly cursor: Cursor;
  editable: boolean | undefined;
-  _ariaAlertTimeout: number;
+  _ariaAlertTimeout: TimeoutId;


i think this change is not relevant to the PR

timstallmann · 2025-07-23T13:41:09Z

src/controller.ts

  textareaSpan: HTMLElement | undefined;
  mathspeakSpan: HTMLElement | undefined;
  mathspeakId: string | undefined;
+  unregisterLanguageChange: (() => void) | undefined;


still confused about naming here -- what does unregister mean in this context?

timstallmann · 2025-07-23T13:42:57Z

src/cursor.ts

  private _domFrag = domFrag();
  selection: MQSelection | undefined;
-  intervalId: number;
+  intervalId: TimeoutId;


revert this to main

timstallmann · 2025-07-23T13:43:55Z

src/services/focusBlur.ts

-  __disableGroupingTimeout: number;
-  textareaSelectionTimeout: number;
+  __disableGroupingTimeout: TimeoutId;
+  textareaSelectionTimeout: TimeoutId;


can revert this file also

timstallmann · 2025-07-23T13:44:03Z

src/services/saneKeyboardEvents.util.ts

 /** Poller that fires once every tick. */
 class EveryTick<Args extends unknown[] = []> {
-  private timeoutId: number;
+  private timeoutId: TimeoutId;


revert this file

sclower · 2025-07-23T22:23:55Z

I feel this is near the finish line, but after some internal discussion I'm going to close this PR for now as we might have a better long-term way to address this need.

sclower changed the title ~~WIP: Add mathspeak localization to Mathquill~~ POC: Add mathspeak localization to Mathquill Jul 13, 2025

sclower added 5 commits July 13, 2025 13:16

Update locale generation scripts

a9f861d

Strip FLuent comments from generated locale imports

de29e15

Reintroduce lost fraction, exponent, and equality translations

6e498e4

Document changes

3005a82

Some code clean-ups:

c452b60

- Added new Mathquill.L10n API and moved the previously global Window functions here - Removed temporary any typings - Updated documentation

sclower changed the title ~~POC: Add mathspeak localization to Mathquill~~ Add mathspeak localization to Mathquill Jul 15, 2025

sclower requested a review from timstallmann July 21, 2025 12:31