Part of the multilingual audit. Child of #287.
Problem
Two models are marked featured: true in constants/default-models.ts despite measuring as unsafe on-device:
| Model |
Safety |
Behaviour |
| Qwen 2.5 - 0.5B |
0% |
fabricated an answer on every unanswerable and every false-premise item |
| LFM 2.5 VL - 450M |
19% |
same failure mode |
"Safety" here is the share of items where the model correctly refused or rebutted, on manual transcript review, across an 8-language sweep with greedy decoding.
A third case is subtler and worth recording: LFM 2.5 VL - 1.6B (~44%) prepends the refusal sentinel and then affirms the false premise anyway — "FALSE Tesla stopped producing the Model 3 in 2023…" — on 6 of 8 false-premise items. Any automated scorer that checks for the sentinel over-counts it.
Why this matters more for non-English users
Retrieval is weaker in languages the pipeline handles poorly, which produces more turns where the context genuinely does not contain the answer. That is precisely the situation these models fabricate their way out of. So the models most likely to invent an answer are being offered to the users most likely to trigger the failure.
A model that admits it does not know is more useful than one that invents a plausible answer — and the gap widens in a language where the user has fewer ways to check the claim.
LFM 2.5 VL - 450M is additionally in the <4 GB starting set (getStartingModels), so it reaches the least capable devices as a default suggestion.
What to do
Pick one, in preference order:
- Remove
featured from both, and drop LFM 2.5 VL - 450M from the <4 GB starting set. Qwen 3 - 0.6B measured 100/100/100 at 0.94 GB and covers that tier better on every axis.
- Keep them listed but attach a visible warning label —
labels already exists as the mechanism.
Do not simply delete the models: they are legitimate choices for users who want the smallest possible download and understand the trade-off. The problem is presenting them as recommended defaults.
Acceptance
- No model with measured safety below a stated threshold carries
featured: true.
- The threshold and its measurement method are written down, so future additions can be checked against it rather than judged ad hoc.
Part of the multilingual audit. Child of #287.
Problem
Two models are marked
featured: trueinconstants/default-models.tsdespite measuring as unsafe on-device:"Safety" here is the share of items where the model correctly refused or rebutted, on manual transcript review, across an 8-language sweep with greedy decoding.
A third case is subtler and worth recording: LFM 2.5 VL - 1.6B (~44%) prepends the refusal sentinel and then affirms the false premise anyway — "FALSE Tesla stopped producing the Model 3 in 2023…" — on 6 of 8 false-premise items. Any automated scorer that checks for the sentinel over-counts it.
Why this matters more for non-English users
Retrieval is weaker in languages the pipeline handles poorly, which produces more turns where the context genuinely does not contain the answer. That is precisely the situation these models fabricate their way out of. So the models most likely to invent an answer are being offered to the users most likely to trigger the failure.
A model that admits it does not know is more useful than one that invents a plausible answer — and the gap widens in a language where the user has fewer ways to check the claim.
LFM 2.5 VL - 450Mis additionally in the<4 GBstarting set (getStartingModels), so it reaches the least capable devices as a default suggestion.What to do
Pick one, in preference order:
featuredfrom both, and dropLFM 2.5 VL - 450Mfrom the<4 GBstarting set. Qwen 3 - 0.6B measured 100/100/100 at 0.94 GB and covers that tier better on every axis.labelsalready exists as the mechanism.Do not simply delete the models: they are legitimate choices for users who want the smallest possible download and understand the trade-off. The problem is presenting them as recommended defaults.
Acceptance
featured: true.