AfroBench: How Good are Large Language Models on African Languages?

[JessicaOjo]

Overview

AfroBench is a large scale benchmark of African languages on LLMs, covering 15 tasks and 22 datasets.

Features

The PR features 19 new tasks/datasets all in the AfroBench subfolder
It also updates the irokobench tasks - AfriXNLI, AfriMGSM and AfriMMLU to reflect recent experiments performed on these tasks

Testing

The PR has been tested locally against several open source models

Documentation

Adequate documentation has been provided on each task ReadMe pages.

[CLAassistant]

All committers have signed the CLA.

[baberabb]

Hi! Thanks for your substantial PR. I'll try to review it soon! Just need everyone in the commit history to pacify the CLI assistant by signing the agreement. cc: @theyorubayesian, @dadelani

[JessicaOjo]

Hi! Thanks for your substantial PR. I'll try to review it soon! Just need everyone in the commit history to pacify the CLI assistant by signing the agreement. cc: @theyorubayesian, @dadelani

Thanks for the comment everyone in the commit history has signed the agreement.

[StellaAthena]

Hi! Thanks for your substantial PR. I'll try to review it soon! Just need everyone in the commit history to pacify the CLI assistant by signing the agreement. cc: @theyorubayesian, @dadelani

Thanks for the comment everyone in the commit history has signed the agreement.

It looks like a bunch of tests are failing. Can you run the pre-commit script and see about addressing those issues?

[JessicaOjo]

Hi! Thanks for your substantial PR. I'll try to review it soon! Just need everyone in the commit history to pacify the CLI assistant by signing the agreement. cc: @theyorubayesian, @dadelani

Thanks for the comment everyone in the commit history has signed the agreement.

It looks like a bunch of tests are failing. Can you run the pre-commit script and see about addressing those issues?

Hi Stella,

I've addressed the pertinent issues, but I’m unsure if the remaining ones are within my scope to fix:

• Linters: The files are being formatted, but I’m not sure what’s causing the failure.
• External LM test: It’s failing due to an accelerate error—specifically, 'clear_device_cache' from accelerate.utils.memory.
• Scan for changed task test: This seems to be failing at the test script level.
  Let me know if you have any suggestions on how I can resolve these. Thanks!