Skip to content

feat: add PARAPHRASE_MULTILINGUAL_MINILM_L12_V2 text embeddings model - #1115

Merged
msluszniak merged 3 commits into
mainfrom
@ms/paraphrase-multilingual-minilm
May 7, 2026
Merged

feat: add PARAPHRASE_MULTILINGUAL_MINILM_L12_V2 text embeddings model#1115
msluszniak merged 3 commits into
mainfrom
@ms/paraphrase-multilingual-minilm

Conversation

@msluszniak

@msluszniak msluszniak commented Apr 30, 2026

Copy link
Copy Markdown
Member

Description

Adds the paraphrase-multilingual-MiniLM-L12-v2 sentence-transformer model — the second multilingual embeddings model after distiluse, completing #945. Ships only the XNNPACK 8da4w variant under MODEL_REGISTRY.ALL_MODELS (see "Why a single variant" below).

384-d output, max 126 tokens, 50+ languages. Tokenizer is Unigram + Precompiled normalizer + Metaspace decoder — requires the bumped pytorch/extension/llm/tokenizers runtime from #1114, so this PR blocks on that landing first and should be rebased onto main once #1114 merges.

HF repo: software-mansion/react-native-executorch-paraphrase-multilingual-MiniLM-L12-v2 (v0.9.0 tag, layout mirrors distiluse).

Why a single variant
TLDR 8da4w works faster then all and was also one of the smallest, without loss in precision.
Longer answer:
unlike distiluse, where Core ML fp32 won iPhone thanks to ANE acceleration, benchmarks on iPhone 17 Pro + OnePlus 12 (~80-token input, 50 measured forwards after 3 warmups) showed the XNNPACK 8da4w variant Pareto-dominates the other three on both platforms: faster than XNNPACK fp32, Core ML fp32 and Core ML fp16 on iPhone, and ~36% smaller steady-state memory footprint than the next-best variant. Likely cause: paraphrase-multilingual-MiniLM-L12-v2 is a smaller model (~118 M params, 12 layers) where Core ML's runtime doesn't push enough work onto ANE for the precision-conversion overhead to pay off. fp16 being slower than fp32 on Core ML for this model is a tell that the runtime is falling back to slower compute units. Shipping only _8DA4W keeps the public surface aligned with the data; if a future Core ML or model update flips the verdict, easy to add the other variants back.

Memory methodology note — the new paraphrase row in docs/docs/02-benchmarks/memory-usage.md reports RSS / phys_footprint deltas from a clean app baseline (loaded − idle), captured on-device at the same conceptual point. The existing distiluse rows there (36 / 44 MB) come from an older measurement pass with a different (and not reconstructable from the diff) methodology, so the two rows are not directly comparable. A separate pass to re-measure distiluse and other rows with the same methodology would be a good follow-up.

Introduces a breaking change?

  • Yes
  • No

Type of change

  • Bug fix (change which fixes an issue)
  • New feature (change which adds functionality)
  • Documentation update (improves or adds clarity to existing documentation)
  • Other (chores, tests, code style improvements etc.)

Tested on

  • iOS
  • Android

Testing instructions

  1. cd apps/text-embeddings && npx expo run:ios (or run:android).
  2. Pick "Multilingual Paraphrase (8da4w)" in the model picker.
  3. Add a sentence in one language, query with an aligned sentence in another (e.g. Polish "Słoneczko" against "It's so sunny outside!"). The cross-lingual pair should top the matches.

Related issues

Closes the paraphrase-multilingual half of #945 (the distiluse half landed in #1098).

Checklist

  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have updated the documentation accordingly
  • My changes generate no new warnings

Additional notes

Blocks on #1114.

@msluszniak msluszniak self-assigned this Apr 30, 2026
@msluszniak msluszniak added the feature PRs that implement a new feature label Apr 30, 2026
@msluszniak msluszniak linked an issue Apr 30, 2026 that may be closed by this pull request
@msluszniak msluszniak added the blocked Issue blocked by some problems (but not other issue, use relationship -> blocker instead) label Apr 30, 2026
@msluszniak
msluszniak force-pushed the @bo/bumpTokenizerCapabilities branch from 78b5a13 to f1341d2 Compare April 30, 2026 12:14
@msluszniak
msluszniak force-pushed the @ms/paraphrase-multilingual-minilm branch from 9cd7623 to 27e7204 Compare April 30, 2026 12:19
Base automatically changed from @bo/bumpTokenizerCapabilities to main May 6, 2026 09:12
msluszniak added 2 commits May 6, 2026 13:57
Benchmarks on iPhone 17 Pro and OnePlus 12 (~80-token input, 50 forwards)
showed XNNPACK 8da4w Pareto-dominates the other three variants on both
platforms — different from distiluse, where Core ML won iPhone via ANE.
Drop the bare _V2, _COREML and _COREML_FP16 exports; ship only _8DA4W.
Adds the corresponding rows to model-size, inference-time, and memory-usage
docs (memory column uses a different methodology than the older distiluse
rows, see PR body).
@msluszniak
msluszniak force-pushed the @ms/paraphrase-multilingual-minilm branch from 27e7204 to 2cfefc9 Compare May 6, 2026 11:59
@msluszniak
msluszniak marked this pull request as ready for review May 6, 2026 12:00
@msluszniak

msluszniak commented May 6, 2026

Copy link
Copy Markdown
Member Author

Would you rather keep 8da4w suffix or you are in favour of dropping it? cc: @chmjkb @NorbertKlockiewicz @barhanc

@msluszniak
msluszniak requested a review from barhanc May 6, 2026 18:19

@barhanc barhanc left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Model works, just this small comment regarding naming.

/**
* @category Models - Text Embeddings
*/
export const PARAPHRASE_MULTILINGUAL_MINILM_L12_V2_8DA4W = {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think for now it should be PARAPHRASE_MULTILINGUAL_MINILM_L12_V2_QUANTIZED so that we maintain consistent naming (and we should probably retroactively change the DISTILUSE_BASE_MULTILINGUAL_CASED_V2_8DA4W also). We had some discussion about model registry and naming (#612) so we can change the symbols all in one go when implementing that.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed, I think that answers your question @msluszniak

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, so I'll change the name for this model and for distiluse we will fix this in one holistic change.

Per #1115 review: align the public symbol with the broader naming
discussion (#612) so the suffix describes the trait (quantized) rather
than the specific scheme (8da4w).
@msluszniak
msluszniak requested a review from barhanc May 7, 2026 10:57
@msluszniak msluszniak removed the blocked Issue blocked by some problems (but not other issue, use relationship -> blocker instead) label May 7, 2026
@msluszniak
msluszniak merged commit 35e33b3 into main May 7, 2026
5 checks passed
@msluszniak
msluszniak deleted the @ms/paraphrase-multilingual-minilm branch May 7, 2026 11:56
IgorSwat pushed a commit that referenced this pull request May 11, 2026
…#1115)

## Description

Adds the `paraphrase-multilingual-MiniLM-L12-v2` sentence-transformer
model — the second multilingual embeddings model after distiluse,
completing #945. Ships **only the XNNPACK 8da4w variant** under
`MODEL_REGISTRY.ALL_MODELS` (see "Why a single variant" below).

384-d output, max 126 tokens, 50+ languages. Tokenizer is Unigram +
Precompiled normalizer + Metaspace decoder — **requires the bumped
`pytorch/extension/llm/tokenizers` runtime from #1114**, so this PR
blocks on that landing first and should be rebased onto main once #1114
merges.

HF repo:
[software-mansion/react-native-executorch-paraphrase-multilingual-MiniLM-L12-v2](https://huggingface.co/software-mansion/react-native-executorch-paraphrase-multilingual-MiniLM-L12-v2)
(`v0.9.0` tag, layout mirrors distiluse).

**Why a single variant** — 
TLDR 8da4w works faster then all and was also one of the smallest,
without loss in precision.
Longer answer:
unlike distiluse, where Core ML fp32 won iPhone thanks to ANE
acceleration, benchmarks on iPhone 17 Pro + OnePlus 12 (~80-token input,
50 measured forwards after 3 warmups) showed the XNNPACK 8da4w variant
Pareto-dominates the other three on both platforms: faster than XNNPACK
fp32, Core ML fp32 *and* Core ML fp16 on iPhone, and ~36% smaller
steady-state memory footprint than the next-best variant. Likely cause:
paraphrase-multilingual-MiniLM-L12-v2 is a smaller model (~118 M params,
12 layers) where Core ML's runtime doesn't push enough work onto ANE for
the precision-conversion overhead to pay off. fp16 being slower than
fp32 on Core ML for this model is a tell that the runtime is falling
back to slower compute units. Shipping only `_8DA4W` keeps the public
surface aligned with the data; if a future Core ML or model update flips
the verdict, easy to add the other variants back.

**Memory methodology note** — the new paraphrase row in
`docs/docs/02-benchmarks/memory-usage.md` reports RSS / `phys_footprint`
deltas from a clean app baseline (loaded − idle), captured on-device at
the same conceptual point. The existing distiluse rows there (36 / 44
MB) come from an older measurement pass with a different (and not
reconstructable from the diff) methodology, so the two rows are not
directly comparable. A separate pass to re-measure distiluse and other
rows with the same methodology would be a good follow-up.

### Introduces a breaking change?

- [ ] Yes
- [x] No

### Type of change

- [ ] Bug fix (change which fixes an issue)
- [x] New feature (change which adds functionality)
- [ ] Documentation update (improves or adds clarity to existing
documentation)
- [ ] Other (chores, tests, code style improvements etc.)

### Tested on

- [x] iOS
- [x] Android

### Testing instructions

1. `cd apps/text-embeddings && npx expo run:ios` (or `run:android`).
2. Pick **"Multilingual Paraphrase (8da4w)"** in the model picker.
3. Add a sentence in one language, query with an aligned sentence in
another (e.g. Polish "Słoneczko" against "It's so sunny outside!"). The
cross-lingual pair should top the matches.

### Related issues

Closes the paraphrase-multilingual half of #945 (the distiluse half
landed in #1098).

### Checklist

- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have updated the documentation accordingly
- [x] My changes generate no new warnings

### Additional notes

Blocks on #1114.
IgorSwat pushed a commit that referenced this pull request May 14, 2026
…#1115)

## Description

Adds the `paraphrase-multilingual-MiniLM-L12-v2` sentence-transformer
model — the second multilingual embeddings model after distiluse,
completing #945. Ships **only the XNNPACK 8da4w variant** under
`MODEL_REGISTRY.ALL_MODELS` (see "Why a single variant" below).

384-d output, max 126 tokens, 50+ languages. Tokenizer is Unigram +
Precompiled normalizer + Metaspace decoder — **requires the bumped
`pytorch/extension/llm/tokenizers` runtime from #1114**, so this PR
blocks on that landing first and should be rebased onto main once #1114
merges.

HF repo:
[software-mansion/react-native-executorch-paraphrase-multilingual-MiniLM-L12-v2](https://huggingface.co/software-mansion/react-native-executorch-paraphrase-multilingual-MiniLM-L12-v2)
(`v0.9.0` tag, layout mirrors distiluse).

**Why a single variant** — 
TLDR 8da4w works faster then all and was also one of the smallest,
without loss in precision.
Longer answer:
unlike distiluse, where Core ML fp32 won iPhone thanks to ANE
acceleration, benchmarks on iPhone 17 Pro + OnePlus 12 (~80-token input,
50 measured forwards after 3 warmups) showed the XNNPACK 8da4w variant
Pareto-dominates the other three on both platforms: faster than XNNPACK
fp32, Core ML fp32 *and* Core ML fp16 on iPhone, and ~36% smaller
steady-state memory footprint than the next-best variant. Likely cause:
paraphrase-multilingual-MiniLM-L12-v2 is a smaller model (~118 M params,
12 layers) where Core ML's runtime doesn't push enough work onto ANE for
the precision-conversion overhead to pay off. fp16 being slower than
fp32 on Core ML for this model is a tell that the runtime is falling
back to slower compute units. Shipping only `_8DA4W` keeps the public
surface aligned with the data; if a future Core ML or model update flips
the verdict, easy to add the other variants back.

**Memory methodology note** — the new paraphrase row in
`docs/docs/02-benchmarks/memory-usage.md` reports RSS / `phys_footprint`
deltas from a clean app baseline (loaded − idle), captured on-device at
the same conceptual point. The existing distiluse rows there (36 / 44
MB) come from an older measurement pass with a different (and not
reconstructable from the diff) methodology, so the two rows are not
directly comparable. A separate pass to re-measure distiluse and other
rows with the same methodology would be a good follow-up.

### Introduces a breaking change?

- [ ] Yes
- [x] No

### Type of change

- [ ] Bug fix (change which fixes an issue)
- [x] New feature (change which adds functionality)
- [ ] Documentation update (improves or adds clarity to existing
documentation)
- [ ] Other (chores, tests, code style improvements etc.)

### Tested on

- [x] iOS
- [x] Android

### Testing instructions

1. `cd apps/text-embeddings && npx expo run:ios` (or `run:android`).
2. Pick **"Multilingual Paraphrase (8da4w)"** in the model picker.
3. Add a sentence in one language, query with an aligned sentence in
another (e.g. Polish "Słoneczko" against "It's so sunny outside!"). The
cross-lingual pair should top the matches.

### Related issues

Closes the paraphrase-multilingual half of #945 (the distiluse half
landed in #1098).

### Checklist

- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have updated the documentation accordingly
- [x] My changes generate no new warnings

### Additional notes

Blocks on #1114.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature PRs that implement a new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request: multilingual text embeddings model

3 participants