Linux-->Linux: Extend multitouch/GII implementation to cover eGalax touchscreen devices

[binzhai]

No description provided.

[dcommander]

Yes, it does support multitouch when using the UltraVNC Viewer. Isn't pinch zoom a form of multitouch?

[binzhai]

Does turbovnc viewer support multi-touch? If so, how to enable it? ?

[dcommander]

See my comment above.

[binzhai]

UltraVNC Viewer supports multi-touch, but UltraVNC Viewer only has a windows version. If using turboVNC viewer on Ubuntu, I found that there is no multi-touch function

[dcommander]

The Linux TurboVNC Viewer supports multitouch with the TurboVNC Server, but only with touchscreen devices. It uses TurboVNC's remote X Input functionality, so the touchscreen must be working properly on the client before it can be expected to work in the TurboVNC session. Send me the TurboVNC session log and the output of the viewer with -loglevel 150, and perhaps I can help you figure out why it isn't working.

[dcommander]

To clarify my comment above:

• The Linux TurboVNC Viewer translates client-side X Input multitouch events into a TurboVNC-specific protocol that piggybacks on the GII RFB extension.
• The UltraVNC Viewer (Windows only) translates client-side Windows multitouch events into an UltraVNC-specific protocol that piggybacks on the GII RFB extension.
• The TurboVNC Server translates multitouch events received via either protocol into X Input multitouch events in the TurboVNC session.
• In other words, multitouch should work with the TurboVNC Server when using the TurboVNC Viewer on Linux clients or the UltraVNC Viewer on Windows clients.
• However, our multitouch implementation has only been tested with specific touchscreen devices. Whether or not it works with a particular touchscreen device on a Linux client depends on how the touchscreen interfaces through X Input.

The multitouch feature was implemented in TurboVNC under contract with companies who have a specific need for it with specific devices. Thus, it isn't guaranteed to work with all devices, but if you provide the diagnostic information I requested, then I might be able to figure out how to extend the feature to support your device. If you don't provide that information, then there is nothing more I can do for you.

[binzhai]

Sorry to reply you only now.
What's happening to me right now is:
1: turbovnc viewer + turbovnc server cannot respond to multi-finger events, such as pinch zooming, the same operation, using UltraVNC Viewer is fine

[binzhai]

log of turbovnc viewer：
libjawt.so path: /usr/lib/jvm/java-11-openjdk-amd64/lib
Log setting: 150
main: start called
CConn: connected to host 192.168.68.234 port 5901
CConnection: reading protocol version
CConnection: Server supports RFB protocol version 3.8
CConnection: Using RFB protocol version 3.8
CConnection: processing security types message
CConnection: Server offers security type None(1)
CConnection: Server offers security type Tight(16)
CConnection: Choosing security type Tight(16)
CConnection: processing security message
Tight: Choosing security type None(1)
CConnection: processing security result message
CConnection: Authentication success!
CConnection: reading server initialisation
CConnection: initialisation done
DesktopWindow: GraphicsDevice supports HW acceleration.
PlatformPixelBuffer: Native pixel format is depth 24 (32bpp) little-endian rgb888
PlatformPixelBuffer: Native pixel format is depth 24 (32bpp) little-endian rgb888
CConn: Using pixel format depth 24 (32bpp) little-endian rgb888
CConn: Requesting Tight encoding
CConn: Screen 0 work area: 72, 27 3768 x 2133
CConn: Spanned work area: 72, 27 3768 x 2133
Viewport: Set geometry to 996, 567 1920 x 1053
TurboVNC Helper: Disabling X11 full-screen mode for window 0x02000056
CMsgReader: Server supports GII version 1
CConn: Enabling GII
ExtInputDevice: DEVICE:
ExtInputDevice: name = Linux 4.9.125 with mtu3 Composite Gadget(HID + MS)
ExtInputDevice: id = 19
ExtInputDevice: vendorID = 4242
ExtInputDevice: productID = 4
ExtInputDevice: canGenerate = 0x00002C00
ExtInputDevice: numRegisters = 20
ExtInputDevice: numValuators = 4
ExtInputDevice: numButtons = 7
ExtInputDevice: absolute = true
ExtInputDevice: VALUATOR:
ExtInputDevice: index = 0
ExtInputDevice: longName = Abs MT Position X
ExtInputDevice: shortName = 0
ExtInputDevice: rangeMin = 0
ExtInputDevice: rangeCenter = 32767
ExtInputDevice: rangeMax = 65535
ExtInputDevice: siUnit = 3
ExtInputDevice: siAdd = 0
ExtInputDevice: siMul = 0
ExtInputDevice: siDiv = 0
ExtInputDevice: siShift = 0
ExtInputDevice: VALUATOR:
ExtInputDevice: index = 1
ExtInputDevice: longName = Abs MT Position Y
ExtInputDevice: shortName = 1
ExtInputDevice: rangeMin = 0
ExtInputDevice: rangeCenter = 32767
ExtInputDevice: rangeMax = 65535
ExtInputDevice: siUnit = 3
ExtInputDevice: siAdd = 0
ExtInputDevice: siMul = 0
ExtInputDevice: siDiv = 0
ExtInputDevice: siShift = 0
ExtInputDevice: VALUATOR:
ExtInputDevice: index = 2
ExtInputDevice: longName = TURBOVNC FAKE TOUCH ID
ExtInputDevice: shortName = TFTI
ExtInputDevice: rangeMin = 0
ExtInputDevice: rangeCenter = 1073741823
ExtInputDevice: rangeMax = 2147483647
ExtInputDevice: siUnit = 0
ExtInputDevice: siAdd = 0
ExtInputDevice: siMul = 0
ExtInputDevice: siDiv = 0
ExtInputDevice: siShift = 0
ExtInputDevice: VALUATOR:
ExtInputDevice: index = 3
ExtInputDevice: longName = TURBOVNC FAKE TOUCH TYPE
ExtInputDevice: shortName = TFTT
ExtInputDevice: rangeMin = 0
ExtInputDevice: rangeCenter = 2
ExtInputDevice: rangeMax = 5
ExtInputDevice: siUnit = 0
ExtInputDevice: siAdd = 0
ExtInputDevice: siMul = 0
ExtInputDevice: siDiv = 0
ExtInputDevice: siShift = 0
ExtInputDevice: DEVICE:
ExtInputDevice: name = eGalax Inc. eGalaxTouch P86H128 8600 v00_M01 k4.18.200
ExtInputDevice: id = 20
ExtInputDevice: vendorID = 4242
ExtInputDevice: productID = 4
ExtInputDevice: canGenerate = 0x00002C00
ExtInputDevice: numRegisters = 10
ExtInputDevice: numValuators = 4
ExtInputDevice: numButtons = 7
ExtInputDevice: absolute = true
ExtInputDevice: VALUATOR:
ExtInputDevice: index = 0
ExtInputDevice: longName = Abs MT Position X
ExtInputDevice: shortName = 0
ExtInputDevice: rangeMin = 0
ExtInputDevice: rangeCenter = 32767
ExtInputDevice: rangeMax = 65535
ExtInputDevice: siUnit = 3
ExtInputDevice: siAdd = 0
ExtInputDevice: siMul = 0
ExtInputDevice: siDiv = 0
ExtInputDevice: siShift = 0
ExtInputDevice: VALUATOR:
ExtInputDevice: index = 1
ExtInputDevice: longName = Abs MT Position Y
ExtInputDevice: shortName = 1
ExtInputDevice: rangeMin = 0
ExtInputDevice: rangeCenter = 32767
ExtInputDevice: rangeMax = 65535
ExtInputDevice: siUnit = 3
ExtInputDevice: siAdd = 0
ExtInputDevice: siMul = 0
ExtInputDevice: siDiv = 0
ExtInputDevice: siShift = 0
ExtInputDevice: VALUATOR:
ExtInputDevice: index = 2
ExtInputDevice: longName = TURBOVNC FAKE TOUCH ID
ExtInputDevice: shortName = TFTI
ExtInputDevice: rangeMin = 0
ExtInputDevice: rangeCenter = 1073741823
ExtInputDevice: rangeMax = 2147483647
ExtInputDevice: siUnit = 0
ExtInputDevice: siAdd = 0
ExtInputDevice: siMul = 0
ExtInputDevice: siDiv = 0
ExtInputDevice: siShift = 0
ExtInputDevice: VALUATOR:
ExtInputDevice: index = 3
ExtInputDevice: longName = TURBOVNC FAKE TOUCH TYPE
ExtInputDevice: shortName = TFTT
ExtInputDevice: rangeMin = 0
ExtInputDevice: rangeCenter = 2
ExtInputDevice: rangeMax = 5
ExtInputDevice: siUnit = 0
ExtInputDevice: siAdd = 0
ExtInputDevice: siMul = 0
ExtInputDevice: siDiv = 0
ExtInputDevice: siShift = 0
TurboVNC Helper: Listening for XInput events on :0 (window 0x02000056)
ScreenSet: LAYOUT RECEIVED:
ScreenSet: 64 (0x40): 1920x994+0+0 (flags 0x0)
ScreenSet: LAYOUT SENT:
ScreenSet: 64 (0x40): 1920x994+0+0 (flags 0x0)
Viewport: Successfully created device 1 (Linux 4.9.125 with mtu3 Composite Gadget(HID + MS))
Viewport: Successfully created device 2 (eGalax Inc. eGalaxTouch P86H128 8600 v00_M01 k4.18.200)
CConn: Enabling continuous updates
ScreenSet: LAYOUT RECEIVED:
ScreenSet: 64 (0x40): 1920x994+0+0 (flags 0x0)
ExtInputEvent: EVENT:
ExtInputEvent: type = Absolute Valuator (13)
ExtInputEvent: deviceID = 20
ExtInputEvent: buttonMask = 0
ExtInputEvent: firstValuator = 0
ExtInputEvent: Valuator 0 = 24703
ExtInputEvent: Valuator 1 = 39286
ExtInputEvent: EVENT:
ExtInputEvent: type = Absolute Valuator (13)
ExtInputEvent: deviceID = 20
ExtInputEvent: buttonMask = 0
ExtInputEvent: firstValuator = 0
ExtInputEvent: Valuator 0 = 24527
ExtInputEvent: Valuator 1 = 38686
ExtInputEvent: Valuator 2 = 76
ExtInputEvent: Valuator 3 = 0
ExtInputEvent: EVENT:
ExtInputEvent: type = Absolute Valuator (13)
ExtInputEvent: deviceID = 20
ExtInputEvent: buttonMask = 0
ExtInputEvent: firstValuator = 0
ExtInputEvent: Valuator 0 = 24447
ExtInputEvent: Valuator 1 = 38652
ExtInputEvent: Valuator 2 = 76
ExtInputEvent: Valuator 3 = 1
ExtInputEvent: EVENT:
ExtInputEvent: type = Absolute Valuator (13)
ExtInputEvent: deviceID = 20
ExtInputEvent: buttonMask = 0
ExtInputEvent: firstValuator = 0
ExtInputEvent: Valuator 0 = 24367
ExtInputEvent: Valuator 1 = 38582
ExtInputEvent: Valuator 2 = 76
ExtInputEvent: Valuator 3 = 1
ExtInputEvent: EVENT:
ExtInputEvent: type = Absolute Valuator (13)
ExtInputEvent: deviceID = 20
ExtInputEvent: buttonMask = 0
ExtInputEvent: firstValuator = 0
ExtInputEvent: Valuator 0 = 24295
ExtInputEvent: Valuator 1 = 38504
ExtInputEvent: Valuator 2 = 76
ExtInputEvent: Valuator 3 = 1
ExtInputEvent: EVENT:
ExtInputEvent: type = Absolute Valuator (13)
ExtInputEvent: deviceID = 20
ExtInputEvent: buttonMask = 0
ExtInputEvent: firstValuator = 0
ExtInputEvent: Valuator 0 = 24183
ExtInputEvent: Valuator 1 = 38417
ExtInputEvent: Valuator 2 = 76
ExtInputEvent: Valuator 3 = 1
ExtInputEvent: EVENT:
ExtInputEvent: type = Absolute Valuator (13)
ExtInputEvent: deviceID = 20
ExtInputEvent: buttonMask = 0
ExtInputEvent: firstValuator = 0
ExtInputEvent: Valuator 0 = 24063
ExtInputEvent: Valuator 1 = 38321
ExtInputEvent: Valuator 2 = 76
ExtInputEvent: Valuator 3 = 1
ExtInputEvent: EVENT:
ExtInputEvent: type = Absolute Valuator (13)
ExtInputEvent: deviceID = 20
ExtInputEvent: buttonMask = 0
ExtInputEvent: firstValuator = 0
ExtInputEvent: Valuator 0 = 24903
ExtInputEvent: Valuator 1 = 41139
ExtInputEvent: Valuator 2 = 75
ExtInputEvent: Valuator 3 = 4
ExtInputEvent: EVENT:
ExtInputEvent: type = Absolute Valuator (13)
ExtInputEvent: deviceID = 20
ExtInputEvent: buttonMask = 0
ExtInputEvent: firstValuator = 0
ExtInputEvent: Valuator 0 = 23903
ExtInputEvent: Valuator 1 = 38147
ExtInputEvent: Valuator 2 = 76
ExtInputEvent: Valuator 3 = 1
ExtInputEvent: EVENT:
ExtInputEvent: type = Absolute Valuator (13)
ExtInputEvent: deviceID = 20
ExtInputEvent: buttonMask = 0
ExtInputEvent: firstValuator = 0
ExtInputEvent: Valuator 0 = 24703
ExtInputEvent: Valuator 1 = 39286
ExtInputEvent: Valuator 2 = 75
ExtInputEvent: Valuator 3 = 3
ExtInputEvent: EVENT:
ExtInputEvent: type = Absolute Valuator (13)
ExtInputEvent: deviceID = 20
ExtInputEvent: buttonMask = 0
ExtInputEvent: firstValuator = 0
ExtInputEvent: Valuator 0 = 24935
ExtInputEvent: Valuator 1 = 41539
ExtInputEvent: Valuator 2 = 75
ExtInputEvent: Valuator 3 = 4
ExtInputEvent: EVENT:
ExtInputEvent: type = Absolute Valuator (13)
ExtInputEvent: deviceID = 20
ExtInputEvent: buttonMask = 0
ExtInputEvent: firstValuator = 0
ExtInputEvent: Valuator 0 = 23743
ExtInputEvent: Valuator 1 = 37947
ExtInputEvent: Valuator 2 = 76
ExtInputEvent: Valuator 3 = 1
ExtInputEvent: EVENT:
ExtInputEvent: type = Absolute Valuator (13)
ExtInputEvent: deviceID = 20
ExtInputEvent: buttonMask = 0
ExtInputEvent: firstValuator = 0
ExtInputEvent: Valuator 0 = 24983
ExtInputEvent: Valuator 1 = 41843
ExtInputEvent: Valuator 2 = 75
ExtInputEvent: Valuator 3 = 4
ExtInputEvent: EVENT:
ExtInputEvent: type = Absolute Valuator (13)
ExtInputEvent: deviceID = 20
ExtInputEvent: buttonMask = 0
ExtInputEvent: firstValuator = 0
ExtInputEvent: Valuator 0 = 23575
ExtInputEvent: Valuator 1 = 37808
ExtInputEvent: Valuator 2 = 76
ExtInputEvent: Valuator 3 = 1
ExtInputEvent: EVENT:
ExtInputEvent: type = Absolute Valuator (13)
ExtInputEvent: deviceID = 20
ExtInputEvent: buttonMask = 0
ExtInputEvent: firstValuator = 0
ExtInputEvent: Valuator 0 = 25031
ExtInputEvent: Valuator 1 = 42130
ExtInputEvent: Valuator 2 = 75
ExtInputEvent: Valuator 3 = 4

[binzhai]

log of turbovnc server

TurboVNC Server (Xvnc) 64-bit v3.0.3 (build 20230227)
Copyright (C) 1999-2022 The VirtualGL Project and many others (see README.txt)
Visit http://www.TurboVNC.org for more information on TurboVNC

06/09/2023 14:53:54 Using security configuration file /etc/turbovncserver-security.conf
06/09/2023 14:53:54 Enabled security type 'none'
06/09/2023 14:53:54 Desktop name 'TurboVNC: localhost:1 ()' (localhost:1)
06/09/2023 14:53:54 Protocol versions supported: 3.3, 3.7, 3.8, 3.7t, 3.8t
06/09/2023 14:53:54 Listening for VNC connections on TCP port 5901
06/09/2023 14:53:54 Interface 0.0.0.0
06/09/2023 14:53:54 Framebuffer: BGRX 8/8/8/8
06/09/2023 14:53:54 New desktop size: 3840 x 2160
06/09/2023 14:53:54 New screen layout:
06/09/2023 14:53:54 0x00000040 (output 0x00000040): 3840x2160+0+0
06/09/2023 14:53:54 Maximum clipboard transfer size: 1048576 bytes
06/09/2023 14:53:54 VNC extension running!

06/09/2023 14:55:35 Got connection from client 192.168.68.41
06/09/2023 14:55:35 Normal socket connection
06/09/2023 14:55:35 Using protocol version 3.8
06/09/2023 14:55:35 rfbSendSecurityTypeList: advertise sectype tight
06/09/2023 14:55:35 Enabling TightVNC protocol extensions
06/09/2023 14:55:35 Advertising Tight auth cap 'NOAUTH__'
06/09/2023 14:55:35 Pixel format for client 192.168.68.41:
06/09/2023 14:55:35 32 bpp, depth 24, little endian
06/09/2023 14:55:35 true colour: max r 255 g 255 b 255, shift r 16 g 8 b 0
06/09/2023 14:55:35 no translation needed
06/09/2023 14:55:35 Enabling full-color cursor updates for client 192.168.68.41
06/09/2023 14:55:35 Enabling Desktop Size protocol extension for client 192.168.68.41
06/09/2023 14:55:35 Enabling Extended Desktop Size protocol extension for client 192.168.68.41
06/09/2023 14:55:35 rfbProcessClientNormalMessage: ignoring unknown encoding -307 (fffffecd)
06/09/2023 14:55:35 Enabling LastRect protocol extension for client 192.168.68.41
06/09/2023 14:55:35 Enabling Continuous Updates protocol extension for client 192.168.68.41
06/09/2023 14:55:35 Enabling Fence protocol extension for client 192.168.68.41
06/09/2023 14:55:35 rfbProcessClientNormalMessage: ignoring unknown encoding -1063131698 (c0a1e5ce)
06/09/2023 14:55:35 Enabling GII protocol extension for client 192.168.68.41
06/09/2023 14:55:35 rfbProcessClientNormalMessage: ignoring unknown encoding -258 (fffffefe)
06/09/2023 14:55:35 rfbProcessClientNormalMessage: ignoring unknown encoding -261 (fffffefb)
06/09/2023 14:55:35 rfbProcessClientNormalMessage: ignoring unknown encoding 1464686184 (574d5668)
06/09/2023 14:55:35 Using tight encoding for client 192.168.68.41
06/09/2023 14:55:35 Using JPEG subsampling 0, Q100 for client 192.168.68.41
06/09/2023 14:55:35 Using JPEG quality 95 for client 192.168.68.41
06/09/2023 14:55:35 Using JPEG subsampling 0 for client 192.168.68.41
06/09/2023 14:55:35 Using Tight compression level 1 for client 192.168.68.41
06/09/2023 14:55:35 Using 8 threads for Tight encoding
06/09/2023 14:55:35 Client supports GII version 1
06/09/2023 14:55:35 GII Device Create: Linux 4.9.125 with mtu3 Composi
06/09/2023 14:55:35 GII device ID = 1
06/09/2023 14:55:36 GII Device Create: eGalax Inc. eGalaxTouch P86H128
06/09/2023 14:55:36 GII device ID = 2
06/09/2023 14:55:36 New desktop size: 3768 x 2074
06/09/2023 14:55:36 New screen layout:
06/09/2023 14:55:36 0x00000040 (output 0x00000040): 3768x2074+0+0
06/09/2023 14:55:36 WARNING: Framebuffer update at 0,0 with dimensions 3840x2074 has been clipped to the screen boundaries
06/09/2023 14:55:36 WARNING: Framebuffer update at 0,0 with dimensions 3840x2160 has been clipped to the screen boundaries
06/09/2023 14:55:36 Continuous updates enabled
06/09/2023 14:55:36 Continuous updates enabled
06/09/2023 14:55:36 GII Device Create: Linux 4.9.125 with mtu3 Composi
06/09/2023 14:55:36 Device 'Linux 4.9.125 with mtu3 Composi' already exists with GII device ID 1
06/09/2023 14:55:36 GII Device Create: eGalax Inc. eGalaxTouch P86H128
06/09/2023 14:55:36 Device 'eGalax Inc. eGalaxTouch P86H128' already exists with GII device ID 2
06/09/2023 14:55:36 Continuous updates enabled
(EE) [dix] eGalax Inc. eGalaxTouch P86H128: unable to find touch point 161
(EE) [dix] eGalax Inc. eGalaxTouch P86H128: unable to find touch point 162
(EE) [dix] eGalax Inc. eGalaxTouch P86H128: unable to find touch point 163
(EE) [dix] eGalax Inc. eGalaxTouch P86H128: unable to find touch point 163
(EE) [dix] eGalax Inc. eGalaxTouch P86H128: unable to find touch point 165
(EE) [dix] eGalax Inc. eGalaxTouch P86H128: unable to find touch point 166
(EE) [dix] eGalax Inc. eGalaxTouch P86H128: unable to find touch point 167
(EE) [dix] eGalax Inc. eGalaxTouch P86H128: unable to find touch point 168
(EE) [dix] eGalax Inc. eGalaxTouch P86H128: unable to find touch point 169
(EE) [dix] eGalax Inc. eGalaxTouch P86H128: unable to find touch point 169
(EE) [dix] eGalax Inc. eGalaxTouch P86H128: unable to find touch point 171
(EE) [dix] eGalax Inc. eGalaxTouch P86H128: unable to find touch point 172
(EE) [dix] eGalax Inc. eGalaxTouch P86H128: unable to find touch point 174
(EE) [dix] eGalax Inc. eGalaxTouch P86H128: unable to find touch point 174
(EE) [dix] eGalax Inc. eGalaxTouch P86H128: unable to find touch point 177
06/09/2023 14:56:13 rfbProcessClientNormalMessage: read: Connection reset by peer
06/09/2023 14:56:13 Client 192.168.68.41 gone
06/09/2023 14:56:13 Statistics:
06/09/2023 14:56:13 key events received 0, pointer events 997
06/09/2023 14:56:13 framebuffer updates 608, rectangles 18692, bytes 18365542
06/09/2023 14:56:13 LastRect markers 556, bytes 6672
06/09/2023 14:56:13 cursor shape updates 6, bytes 3432
06/09/2023 14:56:13 Tight rectangles 18130, bytes 18355438
06/09/2023 14:56:13 raw equivalent 1311.061896 Mbytes, compression ratio 71.426348
06/09/2023 14:56:13 Device 'Linux 4.9.125 with mtu3 Composi' no longer used by any clients. Deleting.
06/09/2023 14:56:13 Device 'eGalax Inc. eGalaxTouch P86H128' no longer used by any clients. Deleting.

[dcommander]

Unfortunately it appears as if your device's behavior isn't compatible with the Linux TurboVNC Viewer's multitouch implementation. I only have one touchscreen device, an ASUS VT168, that I can use for testing such things. That device was provided by the company that funded the development of the Linux-->Linux multitouch feature in TurboVNC. The VT168 was a good enough analogue for the proprietary multitouch device that the company uses, but apparently it isn't a good enough analogue for your multitouch device. I would have to obtain a device similar to yours in order to diagnose and fix the issue and thus extend our implementation to cover that family of devices. (Regression testing to ensure that the VT168 still worked properly would also be necessary.) That isn't something I can do for free, so I am converting this into a feature request marked "funding needed."