Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]v3.11.8 版本 GPU 透传到 虚拟机后, 设备无法正常运行 #21826

Open
khw934 opened this issue Dec 17, 2024 · 3 comments
Open
Labels

Comments

@khw934
Copy link

khw934 commented Dec 17, 2024

问题描述/What happened:

GPU 透传到 虚拟机后, 设备无法正常运行, 报错如下

/opt/conda/lib/python3.8/site-packages/torch/cuda/init.py:52: UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 802: system not yet initialized (Triggered internally at /opt/conda/conda-bld/pytorch_1607370172916/work/c10/cuda/CUDAFunctions.cpp:100.)
return torch._C._cuda_getDeviceCount() > 0

image

环境/Environment:

  • OS (e.g. cat /etc/os-release):
  • Kernel (e.g. uname -a):
  • Host: (e.g. dmidecode | egrep -i 'manufacturer|product' |sort -u)
  • Service Version (e.g. kubectl exec -n onecloud $(kubectl get pods -n onecloud | grep climc | awk '{print $1}') -- climc version-list):
@khw934 khw934 added the bug Something isn't working label Dec 17, 2024
@zexi
Copy link
Member

zexi commented Dec 18, 2024

@khw934 执行 nvidia-smi 能正常输出吗?

@khw934
Copy link
Author

khw934 commented Dec 18, 2024

@khw934 执行 nvidia-smi 能正常输出吗?

@zexi
Copy link
Member

zexi commented Dec 24, 2024

@khw934 没看到回复

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants