Implement new translation tasks for google WMT24++ datasets

[grzegorz-aniol]

Dataset reference: https://huggingface.co/datasets/google/wmt24pp
Paper: https://arxiv.org/abs/2502.12404

[CLAassistant]

All committers have signed the CLA.

[djstrong]

I have run it but see some warnings:

2025-12-27:12:23:33 WARNING  [api.task:728] [Task: wmt24pp-en-pl_PL] metric bleu is defined, but aggregation is not. using default aggregation=bleu
2025-12-27:12:23:33 WARNING  [api.task:740] [Task: wmt24pp-en-pl_PL] metric bleu is defined, but higher_is_better is not. using default higher_is_better=True
2025-12-27:12:23:33 WARNING  [api.task:728] [Task: wmt24pp-en-pl_PL] metric ter is defined, but aggregation is not. using default aggregation=ter
2025-12-27:12:23:33 WARNING  [api.task:740] [Task: wmt24pp-en-pl_PL] metric ter is defined, but higher_is_better is not. using default higher_is_better=True
2025-12-27:12:23:33 WARNING  [api.task:728] [Task: wmt24pp-en-pl_PL] metric chrf is defined, but aggregation is not. using default aggregation=chrf
2025-12-27:12:23:33 WARNING  [api.task:740] [Task: wmt24pp-en-pl_PL] metric chrf is defined, but higher_is_better is not. using default higher_is_better=True

[grzegorz-aniol]

I have run it but see some warnings:

2025-12-27:12:23:33 WARNING  [api.task:728] [Task: wmt24pp-en-pl_PL] metric bleu is defined, but aggregation is not. using default aggregation=bleu
2025-12-27:12:23:33 WARNING  [api.task:740] [Task: wmt24pp-en-pl_PL] metric bleu is defined, but higher_is_better is not. using default higher_is_better=True
2025-12-27:12:23:33 WARNING  [api.task:728] [Task: wmt24pp-en-pl_PL] metric ter is defined, but aggregation is not. using default aggregation=ter
2025-12-27:12:23:33 WARNING  [api.task:740] [Task: wmt24pp-en-pl_PL] metric ter is defined, but higher_is_better is not. using default higher_is_better=True
2025-12-27:12:23:33 WARNING  [api.task:728] [Task: wmt24pp-en-pl_PL] metric chrf is defined, but aggregation is not. using default aggregation=chrf
2025-12-27:12:23:33 WARNING  [api.task:740] [Task: wmt24pp-en-pl_PL] metric chrf is defined, but higher_is_better is not. using default higher_is_better=True

@djstrong I think it's not an issue but I fixed this adding explicit aggregation metric in config file. Thanks to this, TER is correctly reported in output table as ↓ (lower is better)

[grzegorz-aniol]

@baberabb May I draw your attention on this PR? I added new set of tools for google's wmt24++ translation datasets. Do you see something missing, can it be merged? Thank you

[djstrong]

Maybe let's make some tests without chat templates, so it works well with base models too?

[grzegorz-aniol]

Maybe let's make some tests without chat templates, so it works well with base models too?

Ok, I will have to adjust a prompt a bit, to make test working well even without chat template.

@baberabb I will come back to you after verification by some models. Thanks.