We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
请问,评估支持使用昇腾910 NPU嘛?
The text was updated successfully, but these errors were encountered:
暂时不可以,评估的流程依赖推理服务。目前看来主流的推理部署框架比如vLLM、LMDeploy等,支持910的feature暂时还没有见到release出来,可能有在适配中。
Sorry, something went wrong.
感谢指导啊! 另外请问下,咱们评估框架本地评估的话,使用的是什么推理框架呢? 比如本地评估加载千问模型文件,是怎么加载模型文件起推理服务呢?
使用的是transformer+torch框架进行推理的 本地加载模型的话传入模型路径即可,参考:https://evalscope.readthedocs.io/zh-cn/latest/user_guides/offline_evaluation.html
transformer
torch
你使用别的推理框架,例如mindie,或者vllm-npu去启动openai推理接口,再进行测试不就行了
No branches or pull requests
请问,评估支持使用昇腾910 NPU嘛?
The text was updated successfully, but these errors were encountered: