[Feature] Evaluate multiple metrics simultaneously

[ezeli]

Describe the feature

Is it possible to evaluate multiple metrics at the same time, such as bleu and rouge?

Will you implement it?

• I would like to implement this feature and create a PR!

[tonysy]

You can modify the evaluator for the specific dataset and support multiple metrics simultaneously.

[ezeli]

Thanks for your reply, but I think assembling multiple evaluators should be a basic ability.

[acylam]

Thanks for your reply, but I think assembling multiple evaluators should be a basic ability.

@ezeli You can evaluate multiple metrics for multiple datasets by appending each dataset + evaluator_cfg combination to the dataset config list. Note that each evaluator must support the corresponding dataset in each pair for this to work.