OpenGL isn't accelerated by default with Nvidia drivers

[CendioOssman]

Describe the bug
If I run an OpenGL application with an Nvidia card for acceleration, then the application will still be unaccelerated. This can also be seen using glxinfo which shows llvmpipe

To Reproduce
Steps to reproduce the behavior:

1. Run glxinfo
2. See OpenGL renderer string: llvmpipe (LLVM 18.1.6, 256 bits)

Expected behavior
See OpenGL renderer string: NVIDIA GeForce RTX 4060/PCIe/SSE2.

Client (please complete the following information):
No client needed.

Server (please complete the following information):

• OS: Fedora 40
• VNC server: TigerVNC
• VNC server version: 1.14.0 beta
• Server downloaded from: Built using contrib spec file
• Server was started using: Xvnc :2

Additional context
The issue seems to be that it still tries the mesa drivers. I get this error during startup of the application:

glx: failed to create dri3 screen
failed to load driver: nouveau
DRM kernel driver 'nvidia-drm' in use. NVK requires nouveau.
glx: failed to create dri3 screen
failed to load driver: nouveau

If I force glvnd to pick the Nvidia driver, then everything works:

$ __GLX_VENDOR_LIBRARY_NAME=nvidia DISPLAY=:2 glxinfo | grep renderer
OpenGL renderer string: NVIDIA GeForce RTX 4060/PCIe/SSE2

No such issue appears for Vulkan. I guess there is a better mechanism there.

[CendioOssman]

I'm guessing the problem is that Xvnc is still using Mesa for indirect rendering, and hence it will say mesa when you query the GLX extension for which driver to use. Perhaps we can override this from the DRI3 code somehow when we see that we load the Nvidia driver?

[dcommander]

Does DRI3 even work with the nVidia proprietary drivers? My understanding was that they use DRI2, which is tied to the physical X server (via the NV-GLX extension) and can't work in a virtual X server such as Xvnc, Xvfb, etc.

[CendioOssman]

Works nicely during our testing here. I'm afraid I'm not at the machine presently, so I can't check which Nvidia driver version. I think it's the latest, though.

[CendioOssman]

I've been able to come back to testing here, and superficially everything looks like it is working on Fedora 40 with RTX 4060 and Nvidia driver 555.58.02 installed from RPMFusion.

But looking closer at it, rendering performance is not very good, and CPU usage is high. I think the Nvidia driver is falling back to software rendering, even if it presents everything the same as when things as GPU accelerated.

So there does indeed seem to be things that need to be fixed on Nvidia's side, and solving the GLVND issue here will not be sufficient.

[dcommander]

That is consistent with my understanding. DRI3 allocates GPU buffers in the OpenGL front end (i.e. in the application), whereas DRI2 (what nVidia's drivers currently use) allocates GPU buffers in the OpenGL back end (i.e. in the X server.) Thus, they would need to change their driver architecture and release a Mesa-compatible DRI driver in order to support DRI3. By manipulating __GLX_VENDOR_LIBRARY_NAME, you were able to use nVidia's OpenGL front end, but the OpenGL back end was probably still llvmpipe. (nvidia-smi is a good tool to verify whether an application is actually using an nVidia GPU.)