Skip to content

Issues: open-compass/opencompass

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

234 Open 350 Closed
234 Open 350 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[Bug] 缺乏game24以及其他数据集
#1807 opened Jan 7, 2025 by nuochenpku
2 tasks done
@liushz
[Bug] SUPERGLUE-RTE评测任务无反应
#1805 opened Jan 4, 2025 by shedding-ash
2 tasks done
@tonysy
[Feature] 支持基于ray的多卡多机评测
#1804 opened Jan 4, 2025 by wenxueru
1 task
@tonysy
[Feature] 如何添加评价指标
#1803 opened Jan 3, 2025 by Lichunyan3
1 task
@tonysy
2
[Bug] SuperGLUE_BoolQ_ppl Unable to load file
#1799 opened Jan 2, 2025 by shutttttdown
2 tasks done
@acylam
[Bug] gsm8k_gen_17d0dc-BOT思考提示语错误
#1788 opened Dec 27, 2024 by TBoyBack
2 tasks done
@liushz
2
[Bug] Please make sure ./data/safety.txt is correct
#1787 opened Dec 27, 2024 by wccccp
2 tasks done
[Feature] 是否打算支持mamba或者mamba2的测评吗
#1778 opened Dec 25, 2024 by fxnie
1 task done
@MaiziXiao
2
[Bug] torch.OutOfMemoryError
#1773 opened Dec 23, 2024 by QingChengLineOne
2 tasks done
@bittersweet1999
3
[Bug] 推理完成了，但是因为插件版本，在做测评的时候报错了，但是预测的哪些文件都在，怎么重新继续做预测？还有就是运行的时候默认好像是多有的数据集都会测试
#1761 opened Dec 14, 2024 by Lelege0
2 tasks done
@tonysy
1
[Bug] commonsenseqa_gen_c946f2中的commonsenseqa_datasets数据集报错 TypeError: 'ListWrapper' object is not iterable
#1760 opened Dec 14, 2024 by hanbaoergogo
2 tasks done
@tonysy
2
[Bug] cannot use GPU when conduct evaluation
#1759 opened Dec 12, 2024 by takagi97
2 tasks done
@acylam
[Bug] 提示新写的Dataset未注册
#1756 opened Dec 11, 2024 by ezeli
2 tasks done
@MaiziXiao
[Bug] 无法多卡并行评测数据
#1755 opened Dec 11, 2024 by luhairong11
2 tasks done
@MaiziXiao
3
[Bug] 自定义数据集，模型进行评测结果为0分
#1752 opened Dec 11, 2024 by lomoonmoonbird
2 tasks done
@MaiziXiao
4
[Feature] Evaluate multiple metrics simultaneously
#1748 opened Dec 9, 2024 by ezeli
1 task
@bittersweet1999
3
[Bug] NPU 运行报错 core dump
#1747 opened Dec 9, 2024 by SefaZeng
2 tasks done
@bittersweet1999
1
[Bug] 自定义模型评测配置
#1746 opened Dec 9, 2024 by ucassqs
2 tasks done
@bittersweet1999
HumanEvalPlusEvaluator使用的evalplus版本是多少的
#1743 opened Dec 9, 2024 by wanggei
1 task
@bittersweet1999
[Bug] Unsupported operand types in OpenAISDK
#1736 opened Dec 4, 2024 by maanasharma5
2 tasks done
@MaiziXiao
1
[Bug] 微调过的Qwen2.5-7b输出全是感叹号/The output of fine-tuning Qwen2.5-7b is '！！！！！！'
#1732 opened Dec 3, 2024 by noname342
2 tasks done
@liushz
8
[Bug] 在无gpu的机器上执行case，运行时报错数据集未注册（其实已经注册）
#1725 opened Nov 29, 2024 by Caeser-SONG
2 tasks done
@tonysy
1
[Bug] windows下数据集的位置以及检测结果为0.0
#1723 opened Nov 29, 2024 by Dbgsaoge
2 tasks done
@tonysy
您好，请问L-Eval的主观题最终得分是使用rougeLsum这个分数吗？还有就是L-Eval数据集缺少了codeU和sci_fi有相关的评测配置文件么？
#1721 opened Nov 28, 2024 by 13416157913
2 tasks done
@acylam
[Feature] 请问使用API评测如何支持自定义数据集？
#1718 opened Nov 28, 2024 by Jimmy-L99
1 task
@acylam
1
ProTip! no:milestone will show everything without a milestone.