Skip to content

Pull requests: EleutherAI/lm-evaluation-harness

New pull request New
104 Open 1,420 Closed
104 Open 1,420 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Add support for quantization_config
#2842 opened Mar 25, 2025 by jerryzh168 Loading…
1
Add simple Dockerfile and instructions
#2837 opened Mar 24, 2025 by kiersten-stokes Loading…
2
feat: Numeric bench
#2835 opened Mar 24, 2025 by Gresham429 Loading…
4
Add MkDocs Documentation with GitHub Actions Deployment
#2832 opened Mar 23, 2025 by Slyracoon23 Loading…
2
Adds MMLU CoT, gsm8k and arc_challenge for llama instruct
#2829 opened Mar 21, 2025 by anmarques Loading…
1
AfroBench: How Good are Large Language Models on African Languages?
#2825 opened Mar 20, 2025 by JessicaOjo Loading…
5
Increase default max_gen_toks to 2048 and max_length to 8192 for MMLU Pro tests
#2824 opened Mar 20, 2025 by dazipe Loading…
1
1
[leaderboard] math - sync with repo
#2817 opened Mar 19, 2025 by baberabb Loading…
E3 c v3 name entity recognition
#2812 opened Mar 18, 2025 by sfarzi Loading…
2
Adding ACPBench task
#2807 opened Mar 17, 2025 by harshakokel Loading…
6 tasks done
3
Add new task named e3c_v3_re
#2806 opened Mar 17, 2025 by sfarzi Loading…
3
Add GigaChat models
#2805 opened Mar 17, 2025 by seldereyy Loading…
1
Fixed a timeout issue in api_models.py when using async requests
#2795 opened Mar 14, 2025 by dazipe Loading…
1
Add GSM8K Platinum
#2771 opened Mar 7, 2025 by Qubitium Loading…
3
paws-x fix formatting
#2759 opened Mar 5, 2025 by baberabb Loading…
1
New benchmark: CaselawQA
#2739 opened Feb 26, 2025 by RicardoDominguez Loading…
11
Add support for sequence labeling
#2718 opened Feb 20, 2025 by jogonba2 Loading…
Add AIBE task and utilities
#2712 opened Feb 18, 2025 by parimalthakre01 Loading…
1
feat: Add mmlu-redux and it's spanish transaltion as generative task definitions
#2705 opened Feb 16, 2025 by luiscosio Loading…
2
Add Task (Financial mmlu ko)
#2699 opened Feb 14, 2025 by choics2623 Loading…
1
Add generation variants of some tasks
#2688 opened Feb 11, 2025 by baberabb Loading…
Convert multiple_choice to gen tasks
#2670 opened Feb 4, 2025 by baberabb • Draft
Add from dataframe
#2655 opened Jan 25, 2025 by AMindToThink Loading…
2
Include all test files in sdist
#2634 opened Jan 19, 2025 by booxter Loading…
3
fix: nemo eval in containers with TransformerEngine > 1.10 won't error
#2621 opened Jan 10, 2025 by terrykong • Draft
1
ProTip! Find all pull requests that aren't related to any open issues with -linked:issue.