Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GPU PyTorch slower than cpu #127

Open
bsmy12 opened this issue Nov 15, 2024 · 1 comment
Open

GPU PyTorch slower than cpu #127

bsmy12 opened this issue Nov 15, 2024 · 1 comment

Comments

@bsmy12
Copy link

bsmy12 commented Nov 15, 2024

Hi, I am running the standard PyTorch TAPIR model on GPU but noticed that the inference speed is significantly slower than when using cpu. This seems backwards. I am running on 400 frames with only 2 query points. The frame size is 512x512x3. I see that it is in fact running on the GPU due to the usage percentage increasing and the memory increases as well. I am also running on a NVIDIA GeForce RTX 2080. Any guidance on what could be happening here?

CPU: ~325s to complete inference
GPU: ~1025s to complete inference

@yangyi02
Copy link
Collaborator

Hi @bsmy12 , I am trying to reproduce your problem using our colab, but I cannot.

Here is the runtime I got on a L4 GPU (T4 GPU is running out of memory) on your setup: 2 query points, 400 frames, 512x512x3 frame size. On L4 GPU, inference cost 9 seconds.
Screenshot 2024-11-29 at 21 24 27

Unfortunately we don't have NVIDIA GeForce RTX 2080 at this moment. I suspect that an outdated or mismatched NVIDIA driver or CUDA toolkit can severely impact GPU performance.

Suggestion: Ensure the RTX 2080 system uses a compatible driver and CUDA version for the PyTorch setup. Check the PyTorch compatibility table https://github.com/pytorch/pytorch/blob/main/RELEASE.md#release-compatibility-matrix

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants