Multi GPUs
#163
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
{{ message }}
-
Is anyone running the Command-R model via external ollama server with this?
Whenever I set a larger context size and send something to ollama from the llama conversation integration, it seems like the model is loaded on just 1 GPU for some reason... I never actually get a response though and the model is unloaded after some time.
If I then use open webui with the same model, I can see all GPUs being used (same context size set) and the model works fine within open webui.
Beta Was this translation helpful? Give feedback.
All reactions