Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

请问,评估支持使用昇腾910NPU嘛? #123

Open
yiyayieryo opened this issue Sep 9, 2024 · 4 comments
Open

请问,评估支持使用昇腾910NPU嘛? #123

yiyayieryo opened this issue Sep 9, 2024 · 4 comments
Labels
good first issue Good for newcomers

Comments

@yiyayieryo
Copy link

请问,评估支持使用昇腾910 NPU嘛?

@wangxingjun778
Copy link
Collaborator

暂时不可以,评估的流程依赖推理服务。目前看来主流的推理部署框架比如vLLM、LMDeploy等,支持910的feature暂时还没有见到release出来,可能有在适配中。

@yiyayieryo
Copy link
Author

感谢指导啊!
另外请问下,咱们评估框架本地评估的话,使用的是什么推理框架呢?
比如本地评估加载千问模型文件,是怎么加载模型文件起推理服务呢?

@Yunnglin
Copy link
Collaborator

使用的是transformer+torch框架进行推理的
本地加载模型的话传入模型路径即可,参考:https://evalscope.readthedocs.io/zh-cn/latest/user_guides/offline_evaluation.html

@XYZliang
Copy link

你使用别的推理框架,例如mindie,或者vllm-npu去启动openai推理接口,再进行测试不就行了

@Yunnglin Yunnglin added the good first issue Good for newcomers label Nov 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

4 participants