Skip to content

Issues: modelscope/evalscope

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

21 Open 96 Closed
21 Open 96 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

opencompass 支持的数据集比较多，但evalscope使用opencompass作为后端，有些数据集却不支持
#268 opened Dec 30, 2024 by weibingo
5 tasks
1
MMLU 测评结果与官方差距较大
#267 opened Dec 30, 2024 by Haruka1307
1
请问测评自定义模型时是否可以指定model_name
#266 opened Dec 29, 2024 by Haruka1307
3
想请问一下在评估reranker模型想要加入难负例数据，格式也是corpus、query、qerls吗？
#263 opened Dec 26, 2024 by ZHAOFEGNSHUN
1
RAGAS generate testset task 生成问题时报错 Documents appears to be too short (ie 100 tokens or less) bug Something isn't working rageval
#255 opened Dec 19, 2024 by simonqian
2 of 6 tasks
@Yunnglin
4
问答评估 native
#233 opened Dec 9, 2024 by lizhen-lizhen
@Yunnglin
1
使用opencompass评测后端结果和官网结构不一样 opencompass
#229 opened Dec 5, 2024 by yosure
4 tasks
@wangxingjun778
1
当模型推理结果为空时会报错 native
#228 opened Dec 3, 2024 by ChavesLiu
@Yunnglin
4
评估越来越慢，gpu利用率低 native
#227 opened Dec 3, 2024 by ChavesLiu
2 tasks done
@wangxingjun778
1
采用 Openai server 测试，能够产生 inference 结果，但是 eval 输出结果为空 opencompass
#208 opened Nov 22, 2024 by HaoWuSR
9 tasks
@wangxingjun778
2
使用python代码如何统一指定离线data目录？ documentation Improvements or additions to documentation
#201 opened Nov 13, 2024 by charliedream1
opencompass数据集中safety板块数据集支持 enhancement New feature or request
#197 opened Nov 12, 2024 by A-Zi-cong-xiao-jiu-hen-ke-ai
4
评估提示模板 enhancement New feature or request
#195 opened Nov 12, 2024 by lizhen-lizhen
1
OpenCompass打印的测试集列表和网址给出的不一致，导致无法测试，比如cmmlu enhancement New feature or request
#191 opened Nov 9, 2024 by charliedream1
@wangxingjun778
11
能否给个longwriter和toolbench使用openai接口测试的例子 documentation Improvements or additions to documentation
#189 opened Nov 9, 2024 by charliedream1
2
输出结果没有分数 bug Something isn't working opencompass
#181 opened Nov 8, 2024 by Leo20100307
9 tasks
@wangxingjun778
6
evalscope perf 测试sglang 部署的openai api server 无法输出结果 perf question Further information is requested
#128 opened Sep 12, 2024 by hetian127
@Yunnglin
10
请问，评估支持使用昇腾910NPU嘛？ good first issue Good for newcomers
#123 opened Sep 9, 2024 by yiyayieryo
4
Baseline模型对比模式结果错误
#118 opened Aug 29, 2024 by stay-leave
2
请教一下，如何使用openai兼容格式的大模型作为评估模型来进行两模型在自定义数据集上的评估任务呢？
#115 opened Aug 26, 2024 by EvilCalf
3
model_id为local ckpt路径时，infer template映射和解析出现问题
#24 opened Mar 26, 2024 by wangxingjun778
ProTip! What’s not been updated in a month: updated:<2024-11-30.