Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

使用opencompass评测后端结果和官网结构不一样 #229

Open
4 tasks
yosure opened this issue Dec 5, 2024 · 1 comment
Open
4 tasks

使用opencompass评测后端结果和官网结构不一样 #229

yosure opened this issue Dec 5, 2024 · 1 comment
Assignees

Comments

@yosure
Copy link

yosure commented Dec 5, 2024

问题描述 / Issue Description

请简要描述您遇到的问题。 / Please briefly describe the issue you encountered.

使用的工具 / Tools Used

  • Native / 原生框架
  • Opencompass backend
    ##使用脚本
    官方示例脚本
1733364264424

运行环境 / Runtime Environment

  • 操作系统 / Operating System:

    • Ubuntu
  • Python版本 / Python Version:

    • 3.10

实际评测结果
1733364034570
官网结果
1733364083710

没有对数据集结果进行分类和汇总取average

@wangxingjun778
Copy link
Collaborator

OC更新以后mmlu等包含子集的summary没有生效,我们处理一下。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants