Add English to Dutch (#59)

* Add English to Dutch * Update evaluation results [skip ci] * Update model registry [skip ci] Co-authored-by: CircleCI evaluation job <ci-models-evaluation@firefox-translations>
mozilla · Jul 18, 2022 · 5ec2829 · 5ec2829
1 parent 46f6780
commit 5ec2829
Show file tree

Hide file tree

Showing 15 changed files with 57 additions and 8 deletions.
diff --git a/README.md b/README.md
@@ -90,6 +90,7 @@ Suffix of the model file in the registry:
 - Icelandic -> English
 - Norwegian Nynorsk -> English
 - Ukrainian <-> English
+- Dutch <- English
 
 ## Upcoming
-- Dutch <-> English
+- Dutch -> English
diff --git a/evaluation/dev/en-nl/flores-dev.bergamot.nl.bleu b/evaluation/dev/en-nl/flores-dev.bergamot.nl.bleu
@@ -0,0 +1 @@
+27.6
diff --git a/evaluation/dev/en-nl/flores-dev.google.nl.bleu b/evaluation/dev/en-nl/flores-dev.google.nl.bleu
@@ -0,0 +1 @@
+29.4
diff --git a/evaluation/dev/en-nl/flores-dev.microsoft.nl.bleu b/evaluation/dev/en-nl/flores-dev.microsoft.nl.bleu
@@ -0,0 +1 @@
+29.3
diff --git a/evaluation/dev/en-nl/flores-test.bergamot.nl.bleu b/evaluation/dev/en-nl/flores-test.bergamot.nl.bleu
@@ -0,0 +1 @@
+27.2
diff --git a/evaluation/dev/en-nl/flores-test.google.nl.bleu b/evaluation/dev/en-nl/flores-test.google.nl.bleu
@@ -0,0 +1 @@
+29.1
diff --git a/evaluation/dev/en-nl/flores-test.microsoft.nl.bleu b/evaluation/dev/en-nl/flores-test.microsoft.nl.bleu
@@ -0,0 +1 @@
+28.6
diff --git a/evaluation/dev/img/avg.png b/evaluation/dev/img/avg.png
diff --git a/evaluation/dev/img/en-nl.png b/evaluation/dev/img/en-nl.png
diff --git a/evaluation/dev/results.md b/evaluation/dev/results.md
@@ -57,11 +57,11 @@ Both absolute and relative differences in BLEU scores between Bergamot and other
 
 ## avg
 
-| Translator/Dataset | en-uk | fa-en | ru-en | en-ru | uk-en | en-fa | is-en |
-| --- | --- | --- | --- | --- | --- | --- | --- |
-| bergamot | 28.00 | 28.70 | 33.37 | 30.47 | 35.65 | 17.30 | 23.50 |
-| google | 32.40 (+4.40, +15.71%) | 36.05 (+7.35, +25.61%) | 36.53 (+3.15, +9.45%) | 33.72 (+3.25, +10.67%) | 38.90 (+3.25, +9.12%) | 27.70 (+10.40, +60.12%) | 34.95 (+11.45, +48.72%) |
-| microsoft | 31.05 (+3.05, +10.89%) | 36.15 (+7.45, +25.96%) | 36.87 (+3.50, +10.49%) | 33.68 (+3.21, +10.53%) | 39.00 (+3.35, +9.40%) | 20.50 (+3.20, +18.50%) | 34.90 (+11.40, +48.51%) |
+| Translator/Dataset | en-uk | fa-en | ru-en | en-ru | uk-en | en-fa | en-nl | is-en |
+| --- | --- | --- | --- | --- | --- | --- | --- | --- |
+| bergamot | 28.00 | 28.70 | 33.37 | 30.47 | 35.65 | 17.30 | 27.40 | 23.50 |
+| google | 32.40 (+4.40, +15.71%) | 36.05 (+7.35, +25.61%) | 36.53 (+3.15, +9.45%) | 33.72 (+3.25, +10.67%) | 38.90 (+3.25, +9.12%) | 27.70 (+10.40, +60.12%) | 29.25 (+1.85, +6.75%) | 34.95 (+11.45, +48.72%) |
+| microsoft | 31.05 (+3.05, +10.89%) | 36.15 (+7.45, +25.96%) | 36.87 (+3.50, +10.49%) | 33.68 (+3.21, +10.53%) | 39.00 (+3.35, +9.40%) | 20.50 (+3.20, +18.50%) | 28.95 (+1.55, +5.66%) | 34.90 (+11.40, +48.51%) |
 
 ![Results](img/avg.png)
 
@@ -125,6 +125,16 @@ Both absolute and relative differences in BLEU scores between Bergamot and other
 
 ![Results](img/en-fa.png)
 
+## en-nl
+
+| Translator/Dataset | flores-dev | flores-test |
+| --- | --- | --- |
+| bergamot | 27.60 | 27.20 |
+| google | 29.40 (+1.80, +6.52%) | 29.10 (+1.90, +6.99%) |
+| microsoft | 29.30 (+1.70, +6.16%) | 28.60 (+1.40, +5.15%) |
+
+![Results](img/en-nl.png)
+
 ## is-en
 
 | Translator/Dataset | flores-dev | flores-test |

diff --git a/evaluation/prod/results.md b/evaluation/prod/results.md
@@ -24,7 +24,7 @@ BLEU Score |	Interpretation
 Source: https://cloud.google.com/translate/automl/docs/evaluate#bleu
 
 
-BLEU is the most popular becnhmark in academia, so using BLEU allows us also to compare with reserach papers results and competitions (see [Conference on Machine Translation (WMT)](http://statmt.org/wmt21/)).
+BLEU is the most popular becnhmark in academia, so using BLEU allows us also to compare with reserach papers results and competitions (see [Conference on Machine Translation Conference (WMT)](http://statmt.org/wmt21/)).
 
 Read [this article](https://www.rws.com/blog/understanding-mt-quality-bleu-scores/) to better understand what BLEU is and why it is not perfect.
 
@@ -253,4 +253,4 @@ Both absolute and relative differences in BLEU scores between Bergamot and other
 | google | 24.70 (+0.40, +1.65%) | 38.60 (-1.40, -3.50%) | 24.10 (+0.70, +2.99%) | 33.70 (+0.60, +1.81%) | 28.80 (+0.60, +2.13%) | 28.90 (+2.20, +8.24%) | 23.70 (+0.10, +0.42%) | 26.50 (-0.30, -1.12%) | 43.50 (-1.00, -2.25%) | 30.90 (+1.10, +3.69%) | 36.50 (+0.80, +2.24%) | 42.30 (+3.50, +9.02%) | 47.80 (+0.10, +0.21%) | 31.50 (-0.50, -1.56%) | 23.60 (+0.60, +2.61%) | 43.70 (+4.90, +12.63%) |
 | microsoft | 25.30 (+1.00, +4.12%) | 40.50 (+0.50, +1.25%) | 23.70 (+0.30, +1.28%) | 34.30 (+1.20, +3.63%) | 28.80 (+0.60, +2.13%) | 28.20 (+1.50, +5.62%) | 24.00 (+0.40, +1.69%) | 27.20 (+0.40, +1.49%) | 43.80 (-0.70, -1.57%) | 32.20 (+2.40, +8.05%) | 36.10 (+0.40, +1.12%) | 42.90 (+4.10, +10.57%) | 48.70 (+1.00, +2.10%) | 33.10 (+1.10, +3.44%) | 23.90 (+0.90, +3.91%) | 44.00 (+5.20, +13.40%) |
 
-![Results](img/en-de.png)
+![Results](img/en-de.png)
diff --git a/models/dev/ennl/lex.50.50.ennl.s2t.bin.gz b/models/dev/ennl/lex.50.50.ennl.s2t.bin.gz
diff --git a/models/dev/ennl/model.ennl.intgemm.alphas.bin.gz b/models/dev/ennl/model.ennl.intgemm.alphas.bin.gz
diff --git a/models/dev/ennl/vocab.ennl.spm.gz b/models/dev/ennl/vocab.ennl.spm.gz
diff --git a/registry.json b/registry.json
@@ -480,6 +480,29 @@
       "modelType": "dev"
     }
   },
+  "ennl": {
+    "model": {
+      "name": "model.ennl.intgemm.alphas.bin",
+      "size": 17140899,
+      "estimatedCompressedSize": 13081379,
+      "expectedSha256Hash": "906690a58a0d72aff28bd4b941cbd0984d1e0a62958c0b21aebae378a656d822",
+      "modelType": "dev"
+    },
+    "lex": {
+      "name": "lex.50.50.ennl.s2t.bin",
+      "size": 4494892,
+      "estimatedCompressedSize": 2454349,
+      "expectedSha256Hash": "f780a6d74af4b141f551dcc0da56bab44a05a90ef53d63381269710f35eaa41b",
+      "modelType": "dev"
+    },
+    "vocab": {
+      "name": "vocab.ennl.spm",
+      "size": 807541,
+      "estimatedCompressedSize": 411799,
+      "expectedSha256Hash": "43ba3922c3bba2b76ca2e2124837c96518b0e31300b7d6d5ccce55ee10d86393",
+      "modelType": "dev"
+    }
+  },
   "enru": {
     "model": {
       "name": "model.enru.intgemm.alphas.bin",